A Power BI report can look flawless and still fall short within its first few weeks of use. It is slow to open, a total does not match the figure the finance team knows, or nobody can explain how a KPI is calculated. These problems rarely show up in a demo; they surface only once the report is already being used to make decisions.

For a long time, the way to prevent them was manual review: a more experienced team member would open the model and check, point by point, what was right and what needed fixing. This approach works, but it depends on who is reviewing and how much time they have. In this article, we explain how we moved to carrying out this validation against defined criteria and with our own tools, and what that changes for the team and for the client.

What usually goes wrong in a delivered report

Most problems are not in what you see on screen, but in the semantic model: the layer behind the report that holds the tables, the relationships between them and the calculations behind every number.

The most common failures fall into four groups:

Slow reports. The model loads columns that no visual uses. Tables are joined by bi-directional or many-to-many relationships, so every filter forces Power BI to do more calculations. Dates are stored together with the time, which increases the size of the model. Each filter takes seconds to respond, and users end up exporting the data to Excel.

Numbers that don’t add up. There are duplicate measures under different names, divisions that throw an error when the denominator is zero, and year-on-year comparisons built without a properly configured calendar table. When a figure doesn’t match another report or the source system, users start checking the numbers elsewhere and the report loses its value.

Models that are hard to maintain. Table and measure names are unclear, technical keys are visible to users, calculations have no description and fixed values are hard-coded into formulas. Every change the client requests takes longer and carries more risk, especially when the person making it is not the one who built the model.

No documentation. Documentation is usually left until the end of the project, when time is already running out. Knowledge of the model stays with whoever developed it, and the client receives a report that works but that they cannot explain.

As the team and the number of projects grew, reviewing everything by hand for every delivery stopped being viable. A medium-sized model has dozens of tables and hundreds of columns and measures. Two people reviewing the same model often reach different conclusions, and what was checked on one project is not recorded for the next. We built Quality Guard to solve this.

The criteria we check

Quality Guard is the set of tools our Power BI Centre of Excellence uses to ensure report quality. It has three components: the Medallion Rules, which define the criteria; the BPA Rules, which check the model automatically; and Smart PBI, a set of scripts that meet those criteria from the very start of development.

The Medallion Rules are a list of best practices organised into four areas: data modelling, report design, DAX (the language used to write calculations) and the configuration of the Power BI Service, the platform where reports are published and shared.

Within each area, the rules are split into three tiers: Gold, Silver and Bronze. Gold rules are those no report should fail, Silver rules are strongly recommended, and Bronze rules are improvements that become more important in larger or more complex models. This split helps set priorities when project time is limited.

The table below shows an example Gold rule in each area and what it guarantees for the people using the report.

With a shared list, developers and reviewers apply the same criteria. When a rule cannot be met, because of limitations in the data source or a client request, the exception is accepted as long as it is justified. Even so, every rule still has to be checked in every model. That check is carried out with the BPA Rules.

How Quality Guard automates the checks

The BPA Rules (Best Practice Analyzer) turn part of the Medallion Rules into automated checks. They are 71 rules run in Tabular Editor, an external tool used by model developers alongside Power BI. In a few seconds, the whole model is checked against these rules and the result is a list of everything that does not comply.

The table below shows how some of these rules map to the failures described at the start of the article.

Each rule has a severity level from 1 to 3, and the highest-severity issues must be resolved before delivery. For 19 rules, Tabular Editor can apply the fix automatically, for example to hide technical keys or apply the correct number format.

The results are collated in a Power BI report that shows warnings by severity, business area and model, making it possible to track how each model evolves over the course of the project. The analysis is still launched by a team member, but the rules are always the same, regardless of who runs it.

Figure 1: Summary page of the BPA Rules results report.

Smart PBI comes into play before validation. It is a set of scripts that create, in a few minutes, elements almost every model needs: the calendar table, period-on-period comparison calculations and connections to the most common data sources. Because these scripts already follow the Medallion Rules, a model built with them reaches the validation stage with fewer issues. The calendar table comes ready configured, and there is no need for dozens of measures repeating the same time calculations for each KPI.

Documentation as a by-product, not an extra task

The fourth failure described at the start of the article, the lack of documentation, is addressed by Doc4PBI. The tool generates documentation from the semantic model itself, which already contains all the information needed: tables, columns, measures, relationships and calculations. There is therefore no need to write it by hand at the end of the project.

There are two versions. The Power BI Desktop version uses native Power BI functions to create tables with this information inside the model, which update with every data refresh. The Microsoft Fabric version documents several models at once and also shows the dependencies between calculations, that is, which measures are used by others.

At the validation stage, Doc4PBI gives reviewers a complete view of the model in a single report: which measures lack a description, which objects are hidden and how the tables relate to each other. At the end of the project, the same documentation is handed over to the client at no extra effort. How the tool works is explained in detail in an earlier article, “Doc4PowerBI: Documenting Your Power BI Models Without Writing a Single Line of Code”.

Figure 2: Doc4PBI documentation page, generated from the semantic model.

Quality Guard in practice

We applied Quality Guard to an internal report used by our Centre of Excellence to track the team’s activities, assets and case studies. The model had been built with the Smart PBI calendar, but the BPA Rules had never been run against it. The first analysis flagged 296 warnings. After the fixes, that number fell to 164. Most were resolved with Tabular Editor’s automatic fixes and a single configuration change in Power BI.

The three fixes with the biggest impact were:

Auto date tables. Power BI was creating four hidden date tables, which took up space and duplicated the work of the existing calendar. Switching this option off removed around 20 warnings on its own.

Measure formatting. Numeric measures were given a thousands separator, so values appear the same way across all visuals.

Technical columns. Technical keys were hidden from users and several columns were adjusted so that Power BI reads less data on each page.

The nine high-severity warnings were kept on purpose. They relate to measures that return text, such as dynamic titles, and to a field parameter, cases where the rule does not apply. They are exactly the kind of justified exception the Medallion Rules allow for. Around half of the remaining warnings are objects without a description, which can be dealt with in a later phase.

What changes for the team and the client

For the team, reviews always follow the same criteria and checking a model takes minutes. Anyone joining a project midway finds an organised, documented model and gets up to speed faster.

For the client, reports respond faster and the numbers match across pages and across reports. The model can be maintained by another team, internal or external, and documentation is part of the deliverable.

Conclusion

With Quality Guard, a report’s quality no longer depends on who reviews it or how much time is left at the end of the project. The Medallion Rules define the criteria, the BPA Rules check the model and Doc4PBI generates the documentation, so every model is validated in the same way before it reaches the client. Problems are found during development, while they are still easy to fix.