Hourly General Ledger updates, month-end reporting effort cut from 96 to 12 hours and zero source-system downtime, delivered by BI4ALL

The Challenge

The General Ledger (GL) is the backbone of VARO’s financial reporting: it records every transaction across assets, liabilities, revenue, expenses and equity, and its accuracy directly drives the month-end close and management decision-making. As an integrated energy company rooted in Oil & Gas and increasingly investing in Power & Utilities as part of the energy transition, VARO operates a growing and more diverse portfolio of entities and source systems — making timely, trusted financial data across the whole business essential.

Before the project, the Finance team faced several operational constraints:

• Heavy manual effort: producing the Excel report stack had to be repeated several times during each close, adding up to around 96 hours of repetitive month-end work with little strategic value.
• Spreadsheet-based reporting: Finance relied heavily on custom-built Excel workbooks, each pulling its own extracts and re-creating the same calculations, so logic was duplicated, hard to maintain and prone to inconsistent figures.
• Slow, fragile data extraction: extracts from source systems could take hours and frequently failed under high data volumes or system load, forcing restarts and delays. The extraction process also caused variable downtime, leaving source systems unavailable to the business.
• No centralized mapping: there was no single, governed mapping model for financial data across JDE, VEGA and OneStream.

Figure 1 – The challenges VARO Finance faced before the project

The Solution

VARO Finance needed more than faster extracts: it needed an automated, governed General Ledger it could trust at any point in the month, not only at close. Together with the Finance team, BI4ALL shaped the solution around four design principles:

• Automation and error reduction: automated extraction and validation replace repetitive manual work, minimizing the risk that comes with handling data by hand.
• Centralized data integration: data from every source system is integrated automatically into a single model, giving Finance a comprehensive view of the business.
• Flexibility and monitoring: source data is extracted every 15 minutes and the model refreshed every hour, so Finance can follow the accounts throughout the day instead of waiting for month-end.
• Governed mappings: a governance model for account mappings is built into the solution, so every report uses the same, owned definitions.

Figure 2 – The four design principles behind the solution

Each platform plays to its strengths along the way: Databricks handles data engineering, business rules and governance, while Microsoft Fabric serves as the semantic and reporting layer on top of VARO’s established Power BI landscape. Because both work on open Delta tables, OneLake mirroring connects them without copying data or building extra pipelines. Every transaction follows the same journey from posting to decision:

Figure 3 – The Finance journey, from posting to decision

The result is a ready-to-use Finance data product. A central General Ledger holds every posting, enriched with company, account, business unit, date and currency context, so users can slice transactions any way they need. A Trial Balance, calculated from 2012, tracks period, cumulative and month-to-date balances over time to validate accuracy and spot missing transactions. On top, purpose-built reports cover General Ledger data quality checks, cross-system reconciliation that matches postings at account and journal-entry level, and consolidation insights into management adjustments and allocations. And because Finance works heavily in Excel, BI4ALL also delivered a set of ready-made Excel workbooks with a live connection to the Fabric semantic model: users keep working in the tool they know, with pivot tables that refresh from the governed model, so there is no need to copy extracts or rebuild logic in spreadsheets.

Figure 4 – What was delivered: datasets, key measures and reports

Beyond the platform build, BI4ALL brought Finance domain expertise and drove adoption: modeling choices such as keeping General Ledger and Trial Balance analyses separate protect performance, and a user guide with query best practices helps Finance teams get answers quickly and confidently.

Under the hood, each step of the journey maps to a specific platform capability:

• Ingest (Databricks): Spark Declarative Pipelines load JDE, VEGA and OneStream data incrementally every 15 minutes into a reusable silver layer, which is also available to data science teams.
• Consolidate (Databricks): a second set of pipelines applies General Ledger business rules and centrally governed account mappings every hour. Built-in multi-currency logic keeps every amount available in both company currency and the original transaction currency, and Lakeflow Jobs orchestrates both layers, so refresh frequencies can be tuned without changing the pipelines.
• Govern (Unity Catalog): the source systems remain the authority on who can see what. A security table replicated automatically from source-system access definitions, combined with Microsoft Entra ID groups, ensures users never see more in a report than they could see in the source system, with no separate access requests to manage.
• Analyze (Microsoft Fabric): gold tables are mirrored to OneLake through shortcuts, with no data copy, and Fabric semantic models and Power BI reports give Finance self-service access, in Power BI or through Excel workbooks connected live to the semantic model.

The architecture below shows how the pieces fit together. DirectQuery from Power BI to Databricks was also evaluated; given the hourly refresh cadence, OneLake mirroring was the best fit for this phase. Access rules are currently applied in both Unity Catalog and the Fabric semantic models, and the architecture is ready to move to fully centralized governance.

Figure 5 – Solution architecture: Databricks and Microsoft Fabric better together (illustrative)

The Impact

Together, Databricks and Microsoft Fabric turned a manual, slow and error-prone process into an automated, flexible and centralized solution, with results measured against the Enterprise Business Intelligence KPI of automation % and time saving:

• Fresher, reliable data: source extraction went from hours-long, failure-prone runs to incremental loads every 15 minutes, with the consolidated General Ledger refreshed every hour.
• Less manual work: reports are now refreshed every hour and ready whenever Finance needs them, cutting total month-end reporting effort from around 96 hours to 12 hours.
• One logic, no duplication: Finance users work from live-connected Excel workbooks instead of maintaining their own extracts and calculations, so every spreadsheet shows the same governed numbers.
• Faster month-end close: instead of spending time producing manual reports, the Finance team now focuses on the month-end close activities that matter.
• Zero source-system downtime: the previous extraction process caused variable downtime that left source systems unavailable; incremental loads now run with no downtime at all.
• Higher data quality: manual processes used to introduce inconsistencies and errors; with extraction, validation and calculations now automated, those sources of error are gone.
• Self-service, secure access: reporting access mirrors source-system access, so users see exactly what they are entitled to, and nothing more.
• One consolidated view: dozens of users across three legal entities now work from a single GL model spanning three source systems.

Figure 6 – Before and after: what Databricks and Microsoft Fabric each brought

What’s next

When the project was delivered, Databricks metric views were not yet generally available. As a next step, VARO and BI4ALL plan to assess defining core financial metrics once as Databricks metric views, governed centrally in Unity Catalog and consumed consistently from Fabric, Power BI and other tools. This would also allow access rules to be managed in a single place in Unity Catalog, rather than maintained in both the lakehouse and the reporting layer.