How can we enable an Enterprise to make Data Driven Business Decisions
Detailing the role of a Business Analyst in Data Projects
There has been considerable hype and hoopla about ‘Data Driven Business Decision Making’ recently. In fact, many of the Industry experts claim that Decision making powered by Data is going to become the necessity for every business, in order to stay relevant and competitive.
And, almost all the Industries ranging from Retail, Finance to Healthcare, have responded to this new data phenomenon extremely well, by investing heavily in Data, for mining and deriving insights out of it. The role of an Engineering Service provider is, to help the enterprise to collect and analyse the data and present it in an insightful manner, thereby enabling the enterprise to make better strategic decisions. But because of the complexity involved in both the Business and Technical side of things, Engineering Service providers are looking up to the Business Analysts (BAs), for scaling up to this challenge. The expectation is that, along with the business context, a BA needs to have some context on the technology, in order to have a better impact within the team and also with the Stakeholders.
Topics I intend to cover in this article are, What is a data project in general, How is it different from a typical software project, and How to kick-start and manage a data project effectively. And these questions will be covered mostly from the perspective of a Business Analyst.
What is a Data Project?
Like any other software project, even a Data project starts from the Business problem (Vision) identification, followed by the exploration of the underlying business and user needs (Goals), and finally coming up with a solution that addresses the root cause of the problem (Bets). Once the bets are identified, for solutionising some of the ideas like real time user insights or prediction of a variable, there comes the need to process, store and analyse the data effectively. This is when the Data project journey begins.
A data project will be of two types mostly,
I) Creating a Data Platform: If the solution requires data collection and processing from multiple data sources in near real time, then we might require to come up with a platform, for enabling better data streaming and storage for the insight derivation, and
II) Data Analytics: Once the data is processed and normalised, we need to come up with the predictive analytics model on top of the Data stored, to garner business insights.
Below flow depicts the typical data flow for getting business insights,
Let us see this through an example,
Business Vision: I want to improve my Company’s market share.
This can then be translated to the below Goals:
- Increase the customer base
- Increase the sales of my product
The Objectives can then be broken down to multiple smaller Bets,
- Increasing the brand awareness of the products
- Enabling salesmen to be more effective in selling products
- Spotting new industry trends ahead of other players and bringing more innovations to our product portfolio
- Maintaining good customer relationships thereby responding to customers really fast and ensuring that the demand for the product is always met.
In the above bets, if we take the first two pointers, it’s really straight forward. And, we can implement that by creating a system, to help the marketing people in running product campaigns, and by giving the sales people the right set of information about the customers for effective lead conversion.
But, if we take the last two pointers, spotting new trends requires monitoring the buying behaviour and customer expectations consistently (Velocity). Similarly, for maintaining customer relationships, we need to keep track of all the transactions (Volume), customer feedback and use that to forecast the demand. And, the challenging part here is that, we may require data not only from our systems but also from other external systems too (Variety).
Bringing an effective system to manage the 3 Vs of Data — Volume, Velocity and Variety forms the crux of a Data Project (Data Engineering). On top of which intelligent analytics will be applied through Machine learning, for the users to predict the trends (Data Analytics) and maintain a good balance between the demand and supply.
How is the Data Project different from a typical software project?
Data project is very much similar to any other software project. One major difference is that, in a typical software project, we typically keep the user at the center and come up with features around them. But in a data project, along with the user we may also need to have the required data in the center and pivot solution on top of that.
To explain this in detail:
Software Project: We keep the user at the center, gather all possible personas and their corresponding goals, breakdown the goals to features and then prioritise the features that are of high value to the important personas first, implement the MVP, get user feedback and iteratively develop the product.
Data Project: We gather the list of data insights that are required by stakeholders to achieve their goals, prioritise the important insights that will add more value to the user, start analysing the existing data source, come up with a data platform, if the existing data won’t satisfy the needs, develop a predictive model to come up with insights, and iteratively develop the model by increasing the accuracy of its prediction.
How to kick-start and manage a Data Project effectively?
1Doing Business and Data Analysis: Start the project with meticulous planning. Once the key business insights are prioritised, analyse whether the current set of data sources are sufficient for coming up with predictive insights required. If not, ensure the list of components that are required as part of the platform is defined, to process and analyse the data. For this phase, BAs should pair with the Technical members effectively. Outcome of this stage could be a proposed architecture with high level components identified.
Typical Components in a Data project:
2Coming up with a Data Journey: Once the needed insights are identified, the BA should come up with a Data Journey across the components, like how the data will be ingested, processed, exposed and analysed for those Insights. We can then use the same for coming up with the Epics and Story sizing.
If we draw an analogy to the User Journey, user steps with a specific goal will be considered as Feature (Epic), and each step can then be broken down into user activities (User Stories). Whereas, In a Data Journey, each component that is required to come up with an insight can be considered as Feature (Epic) and each component can then be broken down into multiple categories based on the data source (User Stories).
For example: In order to get the insights on New trends on the buying behaviour and customer expectations
We need data from Point of Sales (POS) systems, Customer feedback in the company’s social media page and Customer Service system for tracking the complaints.
3Coming up with Epics and Story slicing: Once the data sources are identified, we will start with Epic and Story slicing. In this case, components like Data Ingestion, Data Processing (Raw / Processed), Data Interfacing, Data Analytics and Data Insights can be considered as Epics.
And under the Data Ingestion Epic, assume that the different data can be ingested through different protocols — POS (through HTTP), Customer Service system (through MQTT) and Social media page (through Batch Jobs), we can then categorise them into Stories. On a side note, You can also consider having each insight or the data source as an Epic. Apply whichever works best for the project.
Ground rule: Slice the stories in a way that you can get the data for insights soon.
4Prioritising and Release planning: Once the stories are carved out, the BA needs to involve business stakeholders for prioritising the stories based on the value. Upon which, we need to work together with the team for the estimates and come up with a proposed roadmap plan. Story grooming and release planning sessions are conducted on a regular basis, during which the BA needs to ensure that the team gets a good understanding of what outcome is expected out of each component like why we are ingesting a particular data, and why we are coming up with a Data interfacing option. So that the stories are estimated with precision, and the roadmap is arrived based on the business value and the team velocity.
5Story Writing: Once the roadmap plan is defined, the BA will come up with the Iteration-wise plan accordingly. And before each iteration, Story needs to be detailed with the expected outcome, the functional background and the success criteria. Although, user insights is the last step in journey, We can use ‘As a — I want — So that’ format for specifying the expected outcome and functional background as part of each story.
And, for specifying the success criteria for each story, we can use Gherkin Scenarios — ‘Given / When / Then’.
6Showcase and Feedback loop: Once we complete the related set of stories in an iteration, let’s say for predicting the New trends on the buying behaviour and customer expectations, we need to do certain things. I) Having a showcase with the stakeholders to ensure the generated insights are useful for them and they were able to make decisions out of it, II) Capture the feedback in the form of backlogs and prioritising them in upcoming iterations, and III) We also need to be watchful of the Machine Learning model’s accuracy, along with the stakeholder’s feedback, we need to ensure the accuracy is also getting improved iteratively.
Takeaways:
I want to conclude this article with a couple of takeaways for you,
I) Since the Data Project is tech heavy in nature, there is a high chance that the technical members may get immersed in the solution and lose sight of the intended goal of the project over time. So, it is the responsibility of the BA, to ensure the team is in alignment with the business goals throughout the project, by acting as a torch bearer for the team to highlight the intended purpose of the solution and highlight the potential outcome often.
II) Similarly, along with the business need, the BA should at least have a good clarity on the “What” and the “Why” of the different technical components like the Data platform and the Analytical model, if not the “how” or the implementation aspect, which will be handled by the technical team anyways. By having a good technical understanding, we can have better communication and coordination with the team and the stakeholders. Thereby, ensuring the project is delivered with utmost quality matching the business requirements.
