How to become big data – data analyst

Anyone who works in the tech industry is aware of the rising demand of Analytics/ Machine learning professionals. More and more organisations have been jumping on to the data driven decision making bandwagon, thereby accumulating loads of data pertaining to their business. In order to make sense of all the data gathered, organisations will require Big Data Analysts to decipher the data.

  Data Analysts have traditionally worked with pre formatted data, that was served by the IT departments, to perform analysis. But with the need for real time or near-real time Analytics to serve end customers better and faster, analysis needs to be performed faster, thereby making the dependency on IT departments a bottleneck. Analysts are required to understand data streams that ingest millions of records into databases or file systems, Lambda architecture and batch processing of data to understand the influx of data.

Also analysing larger amounts of data requires skills that range from understanding the business complexities, the market and the competitors to a wide range of technical skills in data extraction, data cleaning and transformation, data modelling and statistical methods.

Analytics being a relatively new field, is struggling to resource the market demands with highly skilled Big Data Analysts. Being a Big Data Analyst requires a thorough understanding of data architecture and the data flow from source systems into the big data platform. One can always stick to a specific industry domain and specialize within that, for example Healthcare Analytics, Marketing Analytics, Financial Analytics, Operations Analytics, People Analytics, Gaming Analytics etc. But mastering the end-to-end data chain management can lead to plenty of opportunities, irrespective of industry domain.

The entire Data and Analytics suite includes the following gamut of stages:

• Data integrations – connecting disparate data sources
• Data security and governance – ensuring data integrity and access rights
• Master data management – ensuring consistency and uniformity of data
• Data Extraction, Transformation and Loading – making raw data business user friendly
• Hadoop and HDFS – big data storage mechanisms
• SQL/ Hive / Pig – data query languages
• R/ Python –  for data analysis and mining programming languages
• Data science algorithms like Naive Bayes, K-means, AdaBoost etc. – Machine learning algorithms for clustering, classification
• Data Architecture – solutionizing all the above in an optimized way to deliver business insights

The new age data analysts or a versatile Big Data Analyst is one who understands the complexity of data integrations using APIs or connectors or ETL (Extraction, Transformation and Loading), designs data flow from disparate systems keeping in mind data security and quality issues, can code in SQL or Hive and R or Python and is well acquainted with the machine learning algorithms and has a knack at understanding business complexities.

Since Big Data and Analytics is constantly evolving, it is imperative for anyone aiming at a career within the same, to be well versed with the latest tech stack and architectural breakthroughs. Some ways of doing so:

• Following knowledgeable industry leaders or big data thought leaders on Twitter
• Joining Big Data related groups on LinkedIn
• Following Big Data influencers on LinkedIn
• Attending events, conferences and seminars on Big Data
• Connecting with peers within the Big Data industry
• Last but not the least (probably the most important) enrolling in MOOC (Massive Open Online Course) and/ or Big Data books

Since Analytics is a vast field, encompassing several operations, one could choose to specialise in parts of the Analytics chain like data engineers – specializing in highly scalable data management systems or data scientists specializing in machine learning algorithms or data architects – specializing in the overall data integrations, data flow and storage mechanisms. But in order to excel and future proof a career in the world of Big Data, one needs to master more than one area. A data analyst who is acquainted with all the steps involved in data analysis from data extraction to insights is an asset to any organization and will be much sought after!

Advertisement

Continuous customer experience — service designing

Clayton M. Christensen, one of the world’s top experts on innovation and growth, in his book – Competing Against Luck elaborates on why some companies excel at satisfying customer needs while others fail. He calls it the “Jobs to be done theory” which essentially is a means to identify customer needs and build products and service around those needs, instead of pushing a product and expecting the customer to fall for the bait. Customers want a certain product or service to solve a void in their lives and if the product/service not only fills that gap but also resolves multiple other needs and adds value, then even better.

Enhancing the customer experience requires organisations to understand different service interactions that customers experience and the potential for value addition within the same. The product/service provider should not confine their design thinking to their immediate customer but also envision the needs of the extended customer, the consumer of the customer (B2B2C). A seamless customer experience entails identifying the gaps in the customer needs across the touch-points and connecting the dots. Most of the times customers perceive the gaps in the services when they are handed from one division to another, within an organisation, making the behind the scene silos obvious.

Customer journey landscape

Creating continuous or seamless customer experiences should begin with journey maps centered around the journey the customers indulge in while navigating through interactions and touch-points, across multiple devices. A customer journey is never a linear path as the way customers engage with a product or service can be manifold. Charting the fragments of the journey on a single map, makes it easier to design the entire landscape including the front and backstage bits of the customer journey. A well researched concept plotted as journey map leads to ideation which can be refined in iterations culminating in service blueprints. Every part of the journey map jointly contributes to the entire customer experience. Tech enabled business innovation can enhance each customer interaction to yield a holistic contented experience.

Tech enabled touch points — The use of blockchain in supply-chain not only prevents fraud but could aid the customer experience in terms of story telling or driving sustainability where customers can trace the fair trade ecological sourcing of products. Smart labels and smart tags on wine bottlesand clothes aid the B2B customers with supply-chain and logistics Analytics while making the end user’s experience smarter with IoT solutions. The consumer shopping for a piece of clothing can continue to remain engaged with the service provider by engaging with the smart wardrobe apps that allow refurbishing or recycling the garments, suggest new trends based on the consumer’s preferences, help in maintaining an inventory of the wardrobe. The challenge, however, for the service provider is to be able to gather product/service usage data and provide AI driven services within the realms of data privacy and compliance. The understanding of patterns or deviations in the patterns of the product usage leads to innovating new products or services or business models like a contractual business model or partnerships with other service providers, which jointly make a service appealing, cementing the discontinuities. The tech know-how enables the design execution but the prelude is visualizing stories as part of the customer journey map.

New business models that cater to both B2B and B2C customers using data driven approaches to enhance customer experience

It is important to consider loss aversion and recyclability while designing products/services. As more and more people embrace minimalism as a way of life, sales figures dwindle. Refurbishing, recycling or donating to charity, as mentioned in the circular economy should also be considered as part of the product/service design as the number of environment conscious consumers rises.

To serve the customers a non-fragmented customer experience fueled by design thinking at every interaction demands a robust ecosystem of business acumen, big data solutions, IoT, augmented reality and blockchain implementations, actionable insights and KPIs, abolishing organisational silos and an ambidextrous leadership. Designing the front and backstage touch points that support the journey by orchestrating connections across the different interaction points with a service design mindset delights the customer by hitting the spot!

Four steps to becoming a Data-Driven organisation

Not a day goes by when our LinkedIn news feed is not flooded with the mentions of AI and Machine Learning benefitting and changing the ways of mankind, like never before. This hype surrounding AI, Machine learning has resulted in most organisations jumping on the bandwagon without proper evaluation. A couple of years ago, the term Big Data enjoyed a similar hyped status but it has been losing it’s lustre to all the talk about AI and Machine Learning, lately.

The truth, however, is that, AI and Big data need to coexist and converge. Merely collecting and storing data in huge amounts will prove futile, unless AI and Analytics are used to generate meaningful insights that help businesses, enhance customer experience or increase revenue influx.

Making an organisation Data-Driven will take time and will happen in stages. While there are no sure shot ways to create a Data-Driven organisation, below are some ways that could lead to a change:

1. Strategy – It all starts with a clearly defined strategy in place, stating the Whys, Hows, Whos and Whens. A clear strategy helps in raising awareness across the organisation, about the topic in focus (data in this case) and creates a sense of urgency around the change process. It is imperative that the entire organisation understands the importance and implications of a data-driven organisation, thus encouraging people to update their skill sets and raise their level of data awareness. An all round data strategy should not only include the technology required for execution but the kind of competence and people skills and the sort of conducive atmosphere required for a data-driven organisation to thrive.
2. People – Just as there are different kinds of skills required within a Marketing or a Software organisation, there are different skill sets for the different job roles within a data organisation. But due to the hype surrounding Machine Learning and AI while companies lack the practical knowledge in data know-how, the tendency is to either hire the wrong people or assign the wrong tasks to the right people! Not everyone has to be a data scientist in the data organisation. There will be people required to work on data architecture, data infrastructure, data engineering, data science and the Business Analysts. These could very well be the same person, if the organisation is lucky enough. But it is unfair to hire a data engineer and assign him/her the task of building Predictive models or hiring a data Scientist to be told to develop BI reports. Strategists will have to spend the time required to understand the nuances of skills and expertise required in a data organisation but it will be worth it, to retain and grown the talent pool required for a Data-driven organisation.
3.  Patience – Creating a Data-driven organisation will require ample amounts of patience and perseverance. If data has not been involved in the decision making process, earlier,  then the data is most probably not in a state that can be used readily or maybe there is no or not enough data to begin with! In that case, it has to start with gathering the data required to achieve the business goals. Transaction systems have a very different database design than the data storage mechanisms used for Analytics purposes, which entails a design and architecting process before being able to analyse the data. Moreover, as Analysts dig into the transaction data, they surely will encounter non-existence of relevant data, data retrieval issues and unearth data quality issues and data integration problems due to the existence of data silos. In a data-driven organisation, all data sources are integrated to provide a single enterprise version of truth, irrespective of Customer data or Sales or Marketing data. A data platform, integrating all business data sources, ensuring quality and data integrity and security is a time-consuming process. Organisations will have to take this lead time into consideration when strategizing a Data-driven decision making approach.
4. Organisational Culture – The purpose of a Data-driven organisation is to empower employees by means of data and information sharing to enable the organisation to collectively achieve the business goals. This approach requires employees to be data aware and not use gut feelings to make decisions and this could be a whole new approach for many. This new way of working requires organisational change management, educating people to use facts and figures to arrive at conclusions and make decisions. If an organisation is fairly data aware, in the sense that metrics are used to measure certain processes, in order to turn Data-driven , the organisation has to take steps to use data proactively (read Predictive Analytics) and not just summarise events that happened. The CDOs/ CMOs need to drive data awareness by showcasing quick wins and success cases of Data-driven approaches, as a means to use data as the foundation in every decision making process.

Some organisations may take longer to implement a Data-driven culture than others but there is no way an organisation can become Data-driven, just like that, one fine day! If the CDOs can gauge that the organisation has a longer incubation period then it is good to start with raising data awareness and introducing a BI/ Datawarehousing team. It is not recommended to directly leap on to AI, hiring data scientists, to be then left in a lurch if the organisation and the infrastructure are pretty rudimentary to handle their expertise.

A Data-driven organisation culture starts with the right strategy in place, followed by the right people and technology, evaluating and optimising the entire process, intermittently.

Continuous delivery of Analytics

 

I am biased towards Analytics not only because it is my bread and butter but also my passion. But seriously, Analytics is the most important factor that helps drive businesses forward by providing insights into sales, revenue generation means, operations, competitors and customer satisfaction.

Analytics being paramount to businesses, the placement of it is still a matter of dispute. The organisations that get it right and are using data to drive their businesses, understand fully well that Analytics is neither a part of IT nor a part of business. It is somewhere in between, an entity in itself.

The insights generated from Analytics is all about business drivers:

• Performance of the product (Product Analytics)
• How well is the product perceived by customers (Customer Experience)
• Can the business generate larger margins without increasing the price of the product (Cost Optimisation)
• What is the bounce rate and what causes bounce (Funnel Analytics)
• Getting to know the target audience better (Customer Analytics)

While the above insights are business related and require a deep understanding of the product, online marketing knowledge, data stickiness mastery and product management skills, there is a huge IT infrastructure behind the scenes to be able gather the data required and generate the insights.

To be able to generate the business insights required to drive online and offline traffic or increase sales, organisations need to understand their targeted customer base better. Understanding customer behaviour or product performance entails quite a number of technical tasks in the background:

• Logging events on the website or app such as registration, add to cart, add to wish list, proceed to payment etc. (Data Pipelines)
• Having in place a scalable data storage and fast computing infrastructure, which requires knowledge about the various layers of tech stack
• Utilising machine learning and AI to implement Predictive Analytics and recommendations
• Implementing data visualisation tools to distribute data easily throughout the organisation to facilitate data driven decision making and spread data literacy

As is the case, Analytics cannot be boxed into either Tech or Business. It is a conjoined effort of both business and tech to understand the business requirements and translate the same into technically implementable steps. Many organisations make the mistake of involving Analytics at the end stage of product or concept development, which is almost a sure shot fiasco. Analytics needs to be involved at every step of a product development or customer experience or UX design or data infrastructure to make sure that the events, the data points that lead to insights, are in place from the beginning.

Delivering Analytics solutions is a collaborative effort that involves DevOps, data engineers, UX designers, online marketeers, social media strategists, IT strategists, Business Analysts, IT/Data architects and data scientists. A close co-operation between tech and business leads to continuous delivery of smarter and faster automations, enhanced customer experience and business insights.

Build. Measure. Evaluate. Optimise. Reevaluate.

 

 

Growth Hacker’s Marketing

Marketing is being disrupted and no more run by only traditional non-technical marketeers. Marketing is supported by a wide range of – call it reporting, dashboarding, marketing analytics, marketing automation processes. Moreover, the startup scene is very exciting and a hot bed for innovation. Most startups spring into action sans a huge funding. The startups will have to grow exponentially, boasting a substantial customer base to be able to entice investors. Enter the growth hacker – with a single minded goal, growth!

Typically, the UX team designs the UX strategy, the product team develops the product, the coder codes in order to deliver the product and the marketeer tries selling the product. But with the new age disruptive marketing, the UX team, product team, code development team and the marketing team will have to work very closely, trying and testing every trick in the book to elevate growth. A growth hacker is a bit of all the above.

A growth hacker is more of a full stack employee armed with Swiss knife like multiple skill sets, analytical abilities being top rated. Growth hacking is primarily a focus within the startups, the budget being a constraint, lesser number of employees expected to contribute more. But with time, enterprise companies will adapt to growth hacking means of increasing revenue generation. Growth hacking is based on data, analyzing data to improve the business processes, to sell more, to convert more, to gain new customers and retain existing customers. Growth hacking does not entail data reporting only for the purpose of data visualization, it uses data to derive at hypotheses and reasoning to better understand and improve internet marketing.

So what’s growth hacking all about? Growth hacking is about

• Improving user experince by A/B testing to reduce bounce rate
• Content Marketing
• Designing, implementing and analyzing sales funnel to reduce drop rates
• Search Engine Optimization
• Channelizing all it takes to increase conversion rate
• Using analytics to track click stream data about consumer’s online behavior
• Analyzing past online or shopping behavior to be able to predict consumer’s probable behavior at the next visit
• Social Media marketing – paramount for startups on shoe string budgets. Using Facebook, Twitter APIs to analyze the demographics of consumers sharing and liking the products, consumer opinion in social media and competitor analysis
• Being able to analyze consumers that are likely to churn and the reasons behind, which can be addressed. Analyzing the response data from campaigns targeted at reducing churn, to measure campaign effectiveness.
• Improving omnichannel advertising and using analytics to analyze data to conclude the channel that yields most and finding potential market opportunities

From the above list, growth, data and analytics are evidently the point of convergence for growth hacking. Growth hackers have to be inherently curious, tenacious, analytical and above all innovative. Growth hacking is an an art, not just number crunching or coding. It is the ability to see beyond code, to be able to analyze the implications of new features or every change in any part of the business processes that drive growth.

As Sean Ellis says, a “growth hacker’s true compass is north.”

Free Wi-Fi a boom for retailers?

Image courtesy http://www.curtincollege.edu.au/blog/

With the number of smart phone users on the rise every minute,consumers having more choices than ever, businesses have got to get innovative in order to attract new and retain existing customers. On a recent trip abroad, where I had my data roaming turned off, I realised the importance of retailers offering free Wi-Fi! This got me thinking about the ways in which free Wi-Fi could boost sales and increase customer engagement.

• Free Wi-Fi sure drives traffic! Consumers would throng to retail outlets offering Wi-Fi availability. spend more time in stores,  which could lead to conversion. On the contrary offering no Wi-Fi could drive traffic away.
• Having access to internet is a way to quickly check products on offer in the store, finding online discount coupons that can be encashed at the store, try out products in the store but order similar products (in variations) online thereby reducing bounce rate and comparing prices online. All of this leads to an overall better consumer experience and boosts customer retention.
• Consumers act as brand ambassadors on social media liking, sharing and checking in at the retail outlets. The number of check-ins at a particular store speaks about it’s popularity, the same applies to consumers sharing and complimenting products on offer in the stores, on facebook, twitter and instagram. Consumer referrals are a great way of attracting more traffic to both the online and the physical stores.

The crux however lies in the easy and quick connectivity. If the retailers boast about free Wi-Fi but have a cumbersome process connecting to the hotspot then this could actually backfire.

A great mobile reception and easy connectivity to Wi-Fi – happy customers & better sales!

The data value chain

The Consumer Lifecycle

The terms “Data driven” and “Big Data” are the buzz words of today, hyped definitely, but the implications and potential are real and huge! Tapping into the enormous amount of data and associating this data from multiple sources creates a data chain, proving valueable for any organisation. Creating a data value chain consists of four parts: collection, storage, analysis, and implementation. With data storage getting cheaper, the volume and variety of data available to be exploited is increasing exponentially. But unless businesses ask the right questions and better understand the value that the data brings in and be sufficiently informed to make the right decisions, it does not help storing the data. For example, in marketing, organisations can gather data from multiple sources about acquiring a customer, about the customer’s purchasing behaviour, customer feedback on different social media, about the company’s inventory and logistics of product delivery. Analyzing this stored data can lead to substantial number of customers being retained.

A few of the actionable insights can be as follows:

• Improving SEO (search engine optimization), increasing the visibility of the product site and attracting more customers
• CRO (Conversion rate optimization) i.e. converting prospects into sales, by analzying the sales funnel. A typical sales funnel is Home page > search results page > product page > proposal generation and delivery > negotiation > checkout
• Better inventory control systems, resulting in faster deliveries
• Predicting products that a consumer might be interested in, from the vast inventory, by implementing good recommendation algorithms that scan through the consumer behaviour and can predict their preferences
• If some of the above points are taken care of, customer loyalty can increase manifold, based on the overall experience during the entire consumer lifecycle.

Data blending which leads to a Single Customer View and Actionable Insights

Often the focus lies on the Big data technology rather than the business value of implementing big data projects. Data is revolutionising the way we do business. Organisations, today, are inundated with data. To be able to make sense of the data and create a value chain, there has to be starting point and the customer is a good starting point. The customer’s lifecycle with experiences at every touch point defines business growth, innovation and product development. The big data implementations allow blending data from multiple sources leading to a holistic single view of customer, which in turn gives rise to enlightening insights. The data pretaining to customer, from multiple sources, like CRM/ERP/Order Management/Logitics/Social/cookie trackers/Click traffic etc., should be stored, blended and analysed to gain useful actionable insights.

In order to be able to store the gigantic amount of data, organisations have to invest in robust big data technologies. The earlier BI technologies that we had do not support the new forms of data sources such as unstructured data and the huge volumes, variety & velocity of data. The big data architecture consists of the integration from the data sources, the data storage layer, the data processing layer where data exploration can be performed and/or topped with a data visualization layer. Both structured and unstructured data from various sources can be ingested into the big data platform, using Apache Sqoop or Apache Flume, real-time interactive analyses can be performed on massive data sets stored in HDFS or HBase using SQL with Impala, HIVE or using statistical programming language such as R. There are very good visualization tools, such as Pentaho, Datameer, Jaspersoft that can be integrated into the Hadoop ecosystem to get visual insights. Organisations can offload expensive datawarehouses to low cost and high storage enterprise big data technology.

Edited image from Hortonworks

Irrespective of the technical implementation, business metrics such as increasing revenue, reducing operational costs and improving customer experience, should always be kept in mind. The manner in which the data is analyzed could create new business opportunites and transform businesses. Data is an asset and investing in a value chain, from gathering to analyzing, implementing, analyzing the implementations and evolving continuously, will result in huge business gains.

Streamlining the process of processing

The customer expectations are very different, now. Decisions need to be taken in real time, to convert a prospective customer into committing. In an age, where customer seeks instant gratification, organisations that have a longer time-to-market due to cumbersome internal processes, customer loyalty is hard to win. For example, a customer visits your physical store, if you offer a discount at the very first visit, the chances that the customer will revisit your store are high. On the other hand, if you are merely noting customer behaviour which then has to pass through unwieldy processes, later, to mete out a discount coupon, the second time the customer visits your store… if at all, is a thing of the past. The advanced analytics systems now, are able to handle data influx from multiple disparate systems, cleanse and house in the dmp (data management platforms), ready to be queried in real time to cater to predictive and actionable insights, on the fly.

However, if the business methodologies used are not complimenting this speed of data processing, the business will still suffer. The widely used, Lean methodology preaches creating more value for customers with fewer resources. Anything that does not yield value should be eliminated. But organisations need to adapt to only the best of the best practices. Following methodologies by the book, on the contrary, causes bottlenecks. To be able to leverage more out of the Business Analytics systems and solutions, the processes and tools, both, need to be streamlined to create customer satisfaction. A lot of the business intelligence projects take too long to deliver and are inflexible, resulting in the functional business teams procuring BI tools which promise quick wins. The problem with such data discovery tools, apart from creating data silos, are that they lack data governance, hinder data sharing at an enterprise level and increase licensing costs.

It is not a solution to have no business process at all. There needs to be accountability and that comes from business processes. It is a continuous iterative process to find the right balance between processes and the speed of delivering value to keep the costs low and increase the profitability of any business. One size does not fit all and it applies to organisations, as well. Methodologies/processes need to be tweaked, tuned and tailor made for each company. Organisations that try to implement Lean/Agile/Scrum but fail are because they lose the customer focus, some companies do not have a clear strategy in place with employees being assigned foggy responsibilities and lack of communication and this in turn results in the focus shifting from the task at hand to the nitty gritties of such project management methods.

To avoid pitfalls, a clear business strategy needs to be defined specifying business goals in order to maximise gains. The next step is to trim all the processes that lead to this gain.