datacamel.pl

Zapraszamy na stronę:
www.datacamel.pl

.pl

DataCamel - Tworzymy i wspieramy Hurtownie Danych i Systemy BI Tworzymy i zarządzamy - hurtownie danych, systemy BI. Automatyzujemy procesy biznesowe. Wyciągamy ukryte wnioski z Twoich danych.

Data Quality in Insurance Company

First of all, let me ask you a question. What is the most vulnerable part of Data Quality process in corporation?

Some answers may be around technical aspects, like: ‘It is difficult to evaluate data’. Certainly that is true. Mostly when volume require Big Data techniques and infrastructure. There is no doubt that this is big project, require highly trained professionals and modern infrastructure.

So, it is the correct answer to question that was asked? No. Please, give me two minutes for brief introduction to Data Quality Process, then the answer will be easier to accept.

There are multiple steps in this process. Most of them are pretty obvious when you start to think about Data Quality. Let me focus on two most important ones. First is awareness of data quality and fair discovery of consequences of poor quality of organization’s data. As my experience shows, people are aware of this consequences but on different levels. Data Stewards are focus on processing quotes and some data quality issues mostly, add precious time to process single case. Process leaders in back office hears about Data Quality when someone wrongly entered data made huge impact on insurance policy, so client or front office discovered it (before or after there is becoming financial impacts due to huge payment or missing coverage for other institutions) . As I mentioned here other institutions, there are regulations and government institutions that obligate Insurance Company for proper DQ management. Regulations are last option, but sometimes government needs to repair some problems in industry sectors. Therefore, Top Management sooner or later will be into DQ process within company.

Great, so where is the issue there? How to cope with multiple expectations and lack of big picture? How to prepare Data Quality Process Owner for this task, when DQ process requires to work from bottom to top, from Data Steward that type data into forms (or client directly is filling data but understanding of data definitions is different between company and client) to ETL processes that feed Data Warehouse for Analytical Applications? There is no simple answer for that.

Secondly, there is Data Remediation Process. I have been to many discussions where parties made simple solution to deal with any Data Quality issue with single statement. Let’s start evaluate data during entering process (client forms or operational tool currently used in company). Then, there is always similar list of problems: we have more than one system in company. This checks will kill system performance. We do not own internal code to make those changes. Cost of technical implementations will be huge (Actually, there is a cost of bad Data Quality in organization. No one knows it on those meetings), etc. I stop here.

Even when we have Data Monitoring and perfectly prepared Exception Report (that includes wrong and expected values, with clear link to spot where it should be corrected, and by whom), it is not easy task to correct data in source system.

Finally, I can reveal the answer. Organizations don’t know who is actual Data Owner, processes are so distributed, forms were filled by employees that are no longer here, or we cannot even tell what is the correct data, as no one will ask again client for policy that is closed.

If you start to think about Data Quality within your organization, let’s help you with our experience. This process will be fully covered by my team. We will join you in this discussion with Board of Directors, Top Management or even with Technicians that manages your business apps.

PS. There is no better dashboard for monitoring DQ KPI’s than DQ Heat Map. This is a single tool for presenting DQ insights. This is quick check of critical company data elements.

Data Warehouse in Medium e-Commerce Company

Running a company may be seen as repeatable process of making business decisions. Let’s consider medium size e-commerce company. Obviously, there are different departments and processes within. As we may expect, there is not single application to cover all of needs and expected usability for organization. In this article, I would like to cover how to connect data into single source, allowing conformed data elements that allows BI applications efficiently visualize organization data, and finally to make accurate decisions on top of it. That’s is the underlying story for setting up a mature Data Warehouse (DWH) in any company.

There are few reasons why companies start building DWH. When company is growing, it is starting to produce more and more data in new areas of interests. Main factor then is to separate daily operations from analytics. This will free core systems from performing large scale reporting. DWH then will run, right after business hours, main ETL process (Extract, Transform and Load) to fed up data into own storage. This will not affect performance of operational systems when employees are using it. Since data feeding process will end, data warehouse has full night to prepare data for analytics and reporting. This allows developers to slice and dice data with very deep dive into data. With proper storage of facts and dimensions, BI applications will be free to go with most complex reports without paying in processing time. Moreover, growing company also start to introduce another operational systems when new processes starts. Data warehouse should be ready to add another source into ETL process. Therefore, Architect role is to chose suitable model (Kimball vs Inmon, Star vs Snowflake, etc.) to allow balance between flexibility and scalability.

Although, DWH may be seen as back up for company operational data, I will not cover that in this article. Moreover, there is also problem when DWH encounter multiple data source and how to conform it into single point of truth. Let me skip those topics, I will cover them in another articles.

E-commerce company is focusing mostly on sales: retail (B2C) or B2B. Sales is core for e-commerce company, but there are multiple other business areas that add value, or even give competitive advantage.

Company could consider implementing holistic ERP system. In real world examples, ERP is mostly introduced when manufacturing process is part of the business. In other case when products are purchased from another organizations, ERP is replaced by CRM (as vendors in that case are similar to customers). So, CRM in our example is first data source system. Then, there is Logistic and Warehouse for products. We assume that some products are imported, and some of them is good to have in stock, as this will give us more flexibility and credibility as supplier for B2B. This process will be in another smaller system, tailor made for keeping low cost of rotation and stock. Another part is Accounting and Finance, where Accounting has own structures. This will be connected with CRM to generate invoices and send it to clients. Then, there is also Analytics, that build data warehouse for BI system, and deliver data to Finance, Marketing and Management (Top Managements and Operational Leaders as well). All parties are somewhere as clients or suppliers for Enterprise DWH.

Building DWH in our example company structure, will definitely encounter problem of joining data from different sources. We have three systems, but DWH should present dimensional model across all of them. This will start conformance process. This topic will be covered in another article. There is also data quality issue, that has been describe in here: Data Quality in Insurance Company. Although, DWH were described just as an idea, I will finish it here. You need to wait for more technical blog to come.