DQOps

Fixing source data issues

As the first step, the Data Owner should check whether the problem is present in the source platform, such as an OLTP database, or if it is only in the target platform (data warehouse or data lake). If the source data is correct, the Data Owner contacts the Data Engineering Team to review the data pipelines.

However, if there is a problem with the source data, the Data Owner tries to solve it with the Data Producer, even involving external data suppliers or business users. If the problem cannot be resolved, the Data Owner may create a list of acceptable data quality exceptions.

Learn more about the efficient data quality process in our Ebook(https://dqops.com/best-practices-for-effective-data-quality-improvement/).

Assign the initial thresholds. The threshold represents just the expectations and beliefs about the current data quality status. The Data Owner or the Data Engineering Team may believe that there are no invalid rows, so the rule to count the number of invalid rows should be "equals 0".

The correct values will be validated in later steps. The thresholds should later be adjusted to a more reasonable value. The default alerting thresholds are raising data quality issues at the "error" severity level.

Learn more about the efficient data quality process in our Ebook(https://dqops.com/best-practices-for-effective-data-quality-improvement/).

Best practices for effective data quality improvement - DQOps Download DQO eBook to learn best practices for effective data quality improvement. Reach 100% data quality score.

We would like to invite you to the next meetup organised by Dqo.ai!

👉Does your data meet all required data quality dimensions?
👉Would you like to safely analyze the data quality on different clouds?
👉Do you spend too much time on managing data quality?

If you are interested in the answers to these questions, this meetup is sure to meet your expectations!🔍

Data Quality process: How to meet data quality dimensions?, Tue, Nov 29, 2022, 3:00 PM | Meetup We are happy to invite you to our meetup Data Quality process: How to meet data quality dimensions?🔍 We will have two hosts: 👉Piotr Czarnas - Experienced manager and sof