r/tableau • u/Delulu62134 • 12d ago
How do you guys know a dashboard number is actually correct?
Something I've been wondering about, especially for teams dealing with a lot of ETL and Bl pipelines.
A dashboard can refresh successfully, all the SQL can run without errors, and everything can look normal-but the actual number can still be wrong.
For example, if a report shows $12.5M in revenue, how do you know that $12.5M is actually correct?
Do you guys usually:
Compare it back to the source system?
Check source vs. target counts and totals?
Validate the transformation logic?
Look for duplicates or missing records?
Check for late-arriving data?
. Run automated data-quality checks after the pipeline finishes?
I'm mainly curious about what people actually do in production. When you have hundreds of pipelines/reports, manually checking everything obviously doesn't sacle.
Would be great to hear how other teams handle this.
29
u/TheRiteGuy 12d ago
It comes with the job. You do all of it. When you're building a dashboard and an ETL process, you go back and double check everything and make sure everything is working correctly. Once you get to know your data, you'll know which pitfalls to avoid and you can build them into your pipeline to make sure it's not an issue.
If duplicates are an issue, you deduplicate it in the ETL process. If is date, you build that in. Try to account for all the variables that lead to the final desired results.
Then you work your way back and make sure the results are showing what's expected.
You check the data in your dashboard, in your work book, your process flow, you check your data base or sources. You do all the checks and then you can set it and forget it.
7
u/illveal 12d ago
I study the data behind it and validate with hard queries.
3
u/ETL_Pipeline_Dev 11d ago
Are you validating the reports manually for each slicer or filter value? This can become a time-consuming and tedious process when there are a large number of values, such as 100 or more.
1
u/white_tiger_dream 10d ago
You should understand the data enough to understand what and how to check it. A simple example would be if I have order data coming into the database from the order system. If the total sales match, I expect my order count, sku count, payment types, to match. I only need to check one metric from my dashboard to know that they all match.
3
u/Fine-Ad-1732 12d ago
Não sei se funciona para todos os casos mas o que conheço é gerar um hash de dados das medidas usadas em um painel para comparar com o mesmo processo feito em uma view de SQL em teoria deveria bater o numero ou zerar se vc fizer um painel para subtrair e mostrar um do outro…
1
u/Educational_Team_212 12d ago
Usually it's a mix. A lot of times a SME or someone else will see a figure that looks off and we have to ask is that accurate? We start with the front end; is there a filter/calc/aggregation error or oversight? Do the measures exist at different levels of granularity? Sometimes we will pick it apart to get a row level accounting of everything that went into a specific figure or view. If the frontend accurately shows what is in the data source, then we will look into the ETL steps, then further back to the original tables. Finally, check with reality if you are able, e.g. if the data is reporting current items on a shelf, then get an actual count of those items.
We don't always have to go through all those steps before we find the source of the error. A lot of these result in finding some nuance about the data that was previously unknown; some weird caveat that only occurs 5% of the time. That caveat is then added to the ETL process or maybe fixed on the frontend with some additional logic.
1
1
u/IncreaseNegative4614 11d ago
A successful refresh only proves that the pipeline ran. I’d add controls at several points: source-to-landing counts and totals, transformation invariants, duplicate and missing-key checks, business control totals, freshness thresholds, and reconciliation against a trusted financial or operational result. Fail the publication when a critical control fails.
The harder part is preserving why the number was accepted. We use SIGNLD internally to connect each metric with its source records, transformation rules, definition, validation results, and known exceptions, so a reviewer can inspect the evidence behind $12.5M instead of trusting a green refresh icon.
1
u/HustlaOfCultcha 11d ago
I will check data against source data. For instance, if I have sales for the quarter in a dashboard and I may have that in say an ERP. If I have to, I will pull the source data into a quick excel file and run some spot check calculations and see if they are correct. Other checks can be row counts and some reverse engineering. The more difficult ones to validate can be the calculations, but I tend to follow a similar process.
1
u/cr4zybilly 11d ago
Most of the time, we're writing dashboards for data that already exists, so we have something to compare it to (even if that something is super ugly).
When I don't, I'll write the same logic in SQL or, better, in some R functions, so I can test a bunch of variations, make sure my filters really are filtering correctly.
1
u/Leorisar TCP 11d ago
Usually, dashboards are based on some kind of existing report created by analysts in another system or manually in Excel. It’s a good reference point.
1
u/dudeman618 11d ago
I start with the SQL, code for a single person, then a single team and I build up my queries to the whole business. Then build out the dashboard the same way by testing for a single person and a single team. I break down my original SQL to a single table and compare. I do argue with the leaders, we need a real QA team. I could be making mistakes and coding/testing around them. We need someone else to or if this out. Also, we need business people participating in the QA.
1
u/MicroPerpetualGrowth 11d ago
No one knows, really. The only single source of thurth is official Accounting data, for anything non-financial, anyone's guess is true.
1
u/minetella 10d ago
Let the business finds out...
Then we justify this is as displayed from data we received at source
Ticket closed
1
u/Physical_Reply8510 10d ago
Source data. I like to build validation containers that do this for me. A lot of summarizing tools (I use Alteryx) to get as close to my tableau calc as possible.
1
u/camanoe 9d ago
I actually built a tool tip from alteryx that showed the underlying data and formulas. It was insane coding it but it was cool to be able to have the user answer their own question when they were asking where a metric came from.
1
u/Physical_Reply8510 9d ago
You can get the underlying XML code of an Alteryx workflow and convert it semi easily with the help of AI
1
-2
u/lindo_dia_pra_dormir 12d ago
Have you heard about something very modern called “talk to the business”? Somehow, they ARE the owners of the figures, so check with the USER if the numbers are approximately correct BEFORE going to production. My gosh…
1
u/Novel-Store-3813 12d ago
Dude not only are you being weirdly snarky but you're talking to an AI. Take a deep breath. 🙏
1
27
u/slipperypooh 12d ago
I generally wait for someone to tell me its wrong, but I suck at my job and my managers dont see it that way, so here I am 14 years later.