The goal to build a data pipeline, is to better follow up on the business strategy. To bring the way value is delivered to the customers in line with the strategy. To refine the processes with which the customers and employees interact. We do this by capturing the signals that are generate by the process and plot the evolution of well chosen measuring points in intuitive dashboards.
The signals should be captured in a deliberate way in accessible software applications. Next the data is processed in a data warehouse to facilitate visualizations. Then the insights from the visuals are used to take decisions on how to improve the processes. And so on. In a continuous cycle. The circle of data.
Everything starts and ends with the customer. The purpose of your organization is to help customers to solve a problem they have, to satisfy a need they feel. A solution they are willing to pay for. This is the value proposition of the business: to generate income by offering a solution to the customers problem.
The question the customer asks is used as input, the solution that is offered is the output of a process where different actions and steps are taken to achieve this result. On different points in the process measuring points can be defined of things you want to follow up. For instance the time between a customer's question and the response, or the number of departments that are involved in putting together the response.
By going through the different steps in the process, signals are produced. In some cases these signals are kept on paper, or in the minds of the employees. These days the signals are usually captured in a software application that supports the process. In the context of the use of data to follow up on strategy this is the moment data is created: the moment a signal is captured in a software systems and is saved to an underlying database. Technically this means you register a certain transaction with certain properties in a relational data model.
An example of a step in the process where data is created in a software system, is a request for a quote from a prospective customer that is registered in a CRM system. Or the invoice of a purchase of a gift for a customer that is booked into the accounting software.
In order to use the data in these systems to follow up your strategy a number of conditions have to be met.
First the data has to be of sufficient quality. A good understanding of the people who know the customers and the process best. Make sure you understand what the mean when talking about important concepts like 'Customer', 'Product', 'Order', 'Delivery', 'Success'. On first sight this seems crystal clear, but when you go into it you will soon discover a lot of different interpretations are possible. It is important to come to a common understanding to make clear how the input in the systems should be done. To a certain extend certain rules can be programmed into the systems to avoid faulty input.
Second the data has to be captured. For a lot of processes it's technically not necessary to store the data after the process has been successfully completed. Sometimes too little detail is kept to distill valuable insights. Also by overwriting data, instead of register the changes, information is lost.
Third the data has to be accessible. It has to be possible to get the data out of the software systems to use it in analysis. With the rise of SaaS solutions in the cloud, this is not always as easy as you would expect in the current day and age due to limited API capabilities. When you had to install the software on your own infrastructure it was often easier to access your data than it is now, where you have to depend on the maturity of the API's the SaaS vendors are offering. Make sure you do your research in advance when choosing a software application!
Often the software applications we mentioned here above have dashboards of their own to follow up what's going on. The problem with these dashboards, is that they only use their own data. There is huge value in combining the data from different systems is an integrated data model that reflects what's actually going on in the organization over the borders of the applications. A good data model should allow you to follow up how the organization is doing as a whole, not just in it's separate parts.
To follow up on the measuring points you defined in the processes, so how the customer can be provided with a better solution for his problem, is done in a visual way by presenting the data in intuitive charts and tables. This is called a dashboard. The most important measuring points are an indication on the performance of the organization we call Key Performance Indicators (KPI's).
So why am I telling you all this? I help people make sense of their data. This is the framework I use to discover where the risks and opportunities are in your organization. During a leadership alignment session I will go through these different levels where data is relevant in different ways. Depending on the maturity of the organization I meet you where you are and together we figure out which one of the tracks of the data strategy roadmap suits you best at this time.
