Continuous delivery of Analytics

Screen Shot 2016-09-18 at 16.48.40.png

 

I am biased towards Analytics not only because it is my bread and butter but also my passion. But seriously, Analytics is the most important factor that helps drive businesses forward by providing insights into sales, revenue generation means, operations, competitors and customer satisfaction.

wud-slovakia-2015-datadriven-design-jozef-okay-8-638Analytics being paramount to businesses, the placement of it is still a matter of dispute. The organisations that get it right and are using data to drive their businesses, understand fully well that Analytics is neither a part of IT nor a part of business. It is somewhere in between, an entity in itself.

The insights generated from Analytics is all about business drivers:

  • Performance of the product (Product Analytics)
  • How well is the product perceived by customers (Customer Experience)
  • Can the business generate larger margins without increasing the price of the product (Cost Optimisation)
  • What is the bounce rate and what causes bounce (Funnel Analytics)
  • Getting to know the target audience better (Customer Analytics)

While the above insights are business related and require a deep understanding of the product, online marketing knowledge, data stickiness mastery and product management skills, there is a huge IT infrastructure behind the scenes to be able gather the data required and generate the insights.

To be able to generate the business insights required to drive online and offline traffic or increase sales, organisations need to understand their targeted customer base better. Understanding customer behaviour or product performance entails quite a number of technical tasks in the background:

  • Logging events on the website or app such as registration, add to cart, add to wish list, proceed to payment etc. (Data Pipelines)
  • Having in place a scalable data storage and fast computing infrastructure, which requires knowledge about the various layers of tech stack
  • Utilising machine learning and AI to implement Predictive Analytics and recommendations
  • Implementing data visualisation tools to distribute data easily throughout the organisation to facilitate data driven decision making and spread data literacy

As is the case, Analytics cannot be boxed into either Tech or Business. It is a conjoined effort of both business and tech to understand the business requirements and translate the same into technically implementable steps. Many organisations make the mistake of involving Analytics at the end stage of product or concept development, which is almost a sure shot fiasco. Analytics needs to be involved at every step of a product development or customer experience or UX design or data infrastructure to make sure that the events, the data points that lead to insights, are in place from the beginning.

Delivering Analytics solutions is a collaborative effort that involves DevOps, data engineers, UX designers, online marketeers, social media strategists, IT strategists, Business Analysts, IT/Data architects and data scientists. A close co-operation between tech and business leads to continuous delivery of smarter and faster automations, enhanced customer experience and business insights.

Build. Measure. Evaluate. Optimise. Reevaluate.

 

 

Advertisement

Recommendation Systems

Recommendation systems have changed the way people shop online, find books, movies or music, news articles go viral or find friends and work mates on Linkedin. The recommendation systems analyze the browsing patterns on websites, ratings or most popular items at that point of time or the products saved in ones virtual basket to recommend products. Similarly, the common interests, work skills or common geographical locations are used to predict people, that you might want to connect with on social media sites.

Behind such personalized recommendation systems lie big data platforms including software, hardware and algorithms that analyze customer behavior and push recommended products, in real time. The big data platforms handle both data and event data distribution and computation. Data can pertain to how customers or customers similar to the one in question, have rated products in the past while event data could be tracking mouse clicks that trigger events for example viewing a product and sometimes both of the above need to be combined to be able to predict a customer’s choice. Hence, the recommendation system architecture caters to data storage for offline analysis as well as low latency computational needs and a combination of the two.

The data platform architecture needs to be robust enough to ingest continuous real time data streams into scalable systems like Hadoop HBASE or any other big data data storage infrastructure like AWS Redshift. Apache Kafka is usually used as the messaging system for the real time data stream in combination with Apache Storm. Due to high throughput data redundancy needs to be taken care of, in case of failures. If the real time computation needs to take into account customer data like previous purchase history, preferences, products already bought , segmentation based on socio economic demographics or data from ERP, CRM, in that case either all the systems have to be available online to be able to blend the data in real time or the customer detail data could be mashed up, offline to create Single Customer View and queried in combination with the real time event data.

The valueable assests of any organisation are customers,products and now, data. Machine learning algorithms combine the three assets together to leverage business gains and predictive analytics is imperative in being proactive to customer needs. Some of the algorithms used for recommendation engines are content-based filtering, collaborative filtering, dimensionality reduction, Kmeans and matrix factorization techniques. The challenge is not the data storage, with wide availability of highly scalable data storage platforms, but the speed with which the data needs to be analyzed in case of recommendation systems. The best approach is to combine mostly precomputed data with fresh event data using pre modelled algorithms to push personalised recommendations to the customer interface.