21 July 2025
Analytical Transformation in the Cloud: Performance, Scalability and Large-Scale Security
Analytical Transformation in the Cloud: Performance, Scalability and Large-Scale Security
Challenge
In the context of digital transformation and the need to modernise its Analytics platform, a financial sector institution sought to develop a cloud-based analytical solution to replace its previous system, which relied on Oracle Exadata and MicroStrategy. This new solution had to support extremely high data volumes—approximately 2.6 billion transactions per year—while maintaining high performance, adhering to strict security standards, and ensuring the secure sharing of data with external entities.
This initiative involved two significant challenges. The first was the creation of an entirely new analytical model, called “Neighbourhoods”, which had never before been made available to the institution’s clients and was explicitly designed to support municipal and regional entities. The second was the migration of the existing “Banks” model from an On-Premises infrastructure to a cloud-based version that would be more scalable and sustainable, all while preserving its complex business logic and overcoming the technological constraints of the previous system.
Both models required rigorous security measures, optimised performance design, and visualisation within a unified interface, with particular emphasis on access management, scalability and response times.
Solution
BI4ALL designed and implemented a solution based on modern cloud technologies. The architecture includes Databricks as the data processing platform and for analytical model preparation, and Power BI Embedded as the visualisation tool, integrated into a custom web portal. The solution also incorporates hybrid analytical models, combining DirectQuery and Import modes, with optimised aggregations to ensure high performance. In terms of security, advanced policies were applied, including Row-Level Security (RLS) across both models and Object-Level Security (OLS) in the Banks model. To comply with strict privacy and data protection requirements, both models feature real-time anonymisation mechanisms, ensuring confidentiality even at the most granular levels.
Neighbourhoods Model
This marked the first time the institution made an analytical model available for municipal use. The Neighbourhoods model enables the analysis of consumption behaviour by parish, municipality and surrounding geographic areas — revealing local trends, temporal patterns, and relevant segmentations to support public policy, local commerce and tourism.
This model presented significant performance challenges, as it was the first to be implemented and dealt with highly granular data (combining geography, time, consumption category, channel, and more). Over the course of the project, a 90% reduction in response time was achieved from the initial version to the final release.
Banks Model
Unlike the previous model, the Banks model already existed in the On-Prem environment, but with complex rules that were difficult to maintain, limited scalability, and poor integration with external channels. The migration to the cloud enabled a cleaner and more robust architecture, delivering greater flexibility and scalability, as well as shifting infrastructure costs from CAPEX to OPEX.
This model analyses consumption trends among clients of Portuguese banks, filtering by channel, region, time, and category, to support strategic decision-making for banking partners. The security layer is particularly stringent, with RLS and OLS ensuring each entity only accesses the data it is authorised to view.
Benefits
The new solution delivered significant performance improvements, including 90% faster report load times in the Neighbourhoods model. By transitioning to a native cloud architecture, the platform also gained enhanced scalability and simplified maintenance, enabling the institution to respond more efficiently to growing data demands and operational needs.
In addition, the solution ensures full compliance with strict security and regulatory requirements through sophisticated access control mechanisms. It also supports the secure sharing of insights with external entities within a robust B2B framework and integrates seamlessly with a user-friendly web portal, providing a consistent and intuitive user experience.
Practical Applications
-
A municipality can use the Neighbourhoods model to adjust fiscal policies or select optimal locations for events based on local consumption trends;
-
A bank can cross-reference transaction volumes by channel and time with marketing campaigns to fine-tune payment terminal allocation or commercial strategies in near real time;
-
Processing of over 7.8 billion historical records to support advanced analytical models;
-
Support for dozens of external users per month, including municipalities and financial institutions;
-
Report page load times reduced by over 90%, now taking less than 15 seconds even for complex queries.
Example
Consider a financial institution that, as part of its digital transformation, needed to modernise its Analytics platform. The organisation therefore decided to migrate existing on-premises models to the Cloud, making them more scalable and sustainable. This new platform combines optimised performance, unified visualisation, and advanced security policies. As a result, it became possible to ensure reduced response times, controlled information sharing with external entities, and real-time data anonymisation. The institution now benefits from a robust, secure, and highly scalable solution that supports strategic decision-making.
