How to become big data – data analyst

Anyone who works in the tech industry is aware of the rising demand of Analytics/ Machine learning professionals. More and more organisations have been jumping on to the data driven decision making bandwagon, thereby accumulating loads of data pertaining to their business. In order to make sense of all the data gathered, organisations will require Big Data Analysts to decipher the data.

  Data Analysts have traditionally worked with pre formatted data, that was served by the IT departments, to perform analysis. But with the need for real time or near-real time Analytics to serve end customers better and faster, analysis needs to be performed faster, thereby making the dependency on IT departments a bottleneck. Analysts are required to understand data streams that ingest millions of records into databases or file systems, Lambda architecture and batch processing of data to understand the influx of data.

Also analysing larger amounts of data requires skills that range from understanding the business complexities, the market and the competitors to a wide range of technical skills in data extraction, data cleaning and transformation, data modelling and statistical methods.

Analytics being a relatively new field, is struggling to resource the market demands with highly skilled Big Data Analysts. Being a Big Data Analyst requires a thorough understanding of data architecture and the data flow from source systems into the big data platform. One can always stick to a specific industry domain and specialize within that, for example Healthcare Analytics, Marketing Analytics, Financial Analytics, Operations Analytics, People Analytics, Gaming Analytics etc. But mastering the end-to-end data chain management can lead to plenty of opportunities, irrespective of industry domain.

The entire Data and Analytics suite includes the following gamut of stages:

• Data integrations – connecting disparate data sources
• Data security and governance – ensuring data integrity and access rights
• Master data management – ensuring consistency and uniformity of data
• Data Extraction, Transformation and Loading – making raw data business user friendly
• Hadoop and HDFS – big data storage mechanisms
• SQL/ Hive / Pig – data query languages
• R/ Python –  for data analysis and mining programming languages
• Data science algorithms like Naive Bayes, K-means, AdaBoost etc. – Machine learning algorithms for clustering, classification
• Data Architecture – solutionizing all the above in an optimized way to deliver business insights

The new age data analysts or a versatile Big Data Analyst is one who understands the complexity of data integrations using APIs or connectors or ETL (Extraction, Transformation and Loading), designs data flow from disparate systems keeping in mind data security and quality issues, can code in SQL or Hive and R or Python and is well acquainted with the machine learning algorithms and has a knack at understanding business complexities.

Since Big Data and Analytics is constantly evolving, it is imperative for anyone aiming at a career within the same, to be well versed with the latest tech stack and architectural breakthroughs. Some ways of doing so:

• Following knowledgeable industry leaders or big data thought leaders on Twitter
• Joining Big Data related groups on LinkedIn
• Following Big Data influencers on LinkedIn
• Attending events, conferences and seminars on Big Data
• Connecting with peers within the Big Data industry
• Last but not the least (probably the most important) enrolling in MOOC (Massive Open Online Course) and/ or Big Data books

Since Analytics is a vast field, encompassing several operations, one could choose to specialise in parts of the Analytics chain like data engineers – specializing in highly scalable data management systems or data scientists specializing in machine learning algorithms or data architects – specializing in the overall data integrations, data flow and storage mechanisms. But in order to excel and future proof a career in the world of Big Data, one needs to master more than one area. A data analyst who is acquainted with all the steps involved in data analysis from data extraction to insights is an asset to any organization and will be much sought after!

Advertisement

Analytics – Implications on Digitization

Digital is all about data, contrary to the prevalent method of creating Analytics as a silo all by itself. Analytics should be seen as one of the fundamental underlying processes that support the core business processes like product development, marketing, sales, customer relationship, finance and innovation. Data and Analytics provide value to core processes, for continuous improvement.

Most organisations are keen on innovation. Innovation could entail new market opportunities and could be an entirely new value proposition, discovered on a strategy canvas. But innovation could also be a by-product of a business process improvement. Such opportunities can only arise when business processes are tracked, measured and analyzed. Organisations that indulge in hypothesis driven product development or mass marketing could benefit by introducing  a data driven approach to the above processes, thereby uncovering the customer needs and product usage. Businesses may launch products with a certain outcome in mind, but sales, social media feedback and web analytics data may have another story to tell. It is in this story, that new opportunities can be unearthed. Understanding customer behavior is a way of discovering new marketing and/or product/service development opportunities.

Many organisations investing heavily in digitization, charting customer journeys, aimed at improving customer experience across all touch points, seemingly forget to make Analytics an integral part of this process.  The key to understanding  major business drivers like customer retention, ROMI, growth, customer engagement, monetization, finding new customer segments depend on deciphering the business data generated.

Analytics, therefore should be embedded in all business processes to capture the way the end customers perceive products or services or marketing and branding efforts made by any organisation. Analyzing the business data from existing processes could possibly give rise to future business prospects. To tread on a path of continuous improvement and innovation, companies will have to make Analytics a fundamental part of every business strategy.

Four steps to becoming a Data-Driven organisation

Not a day goes by when our LinkedIn news feed is not flooded with the mentions of AI and Machine Learning benefitting and changing the ways of mankind, like never before. This hype surrounding AI, Machine learning has resulted in most organisations jumping on the bandwagon without proper evaluation. A couple of years ago, the term Big Data enjoyed a similar hyped status but it has been losing it’s lustre to all the talk about AI and Machine Learning, lately.

The truth, however, is that, AI and Big data need to coexist and converge. Merely collecting and storing data in huge amounts will prove futile, unless AI and Analytics are used to generate meaningful insights that help businesses, enhance customer experience or increase revenue influx.

Making an organisation Data-Driven will take time and will happen in stages. While there are no sure shot ways to create a Data-Driven organisation, below are some ways that could lead to a change:

1. Strategy – It all starts with a clearly defined strategy in place, stating the Whys, Hows, Whos and Whens. A clear strategy helps in raising awareness across the organisation, about the topic in focus (data in this case) and creates a sense of urgency around the change process. It is imperative that the entire organisation understands the importance and implications of a data-driven organisation, thus encouraging people to update their skill sets and raise their level of data awareness. An all round data strategy should not only include the technology required for execution but the kind of competence and people skills and the sort of conducive atmosphere required for a data-driven organisation to thrive.
2. People – Just as there are different kinds of skills required within a Marketing or a Software organisation, there are different skill sets for the different job roles within a data organisation. But due to the hype surrounding Machine Learning and AI while companies lack the practical knowledge in data know-how, the tendency is to either hire the wrong people or assign the wrong tasks to the right people! Not everyone has to be a data scientist in the data organisation. There will be people required to work on data architecture, data infrastructure, data engineering, data science and the Business Analysts. These could very well be the same person, if the organisation is lucky enough. But it is unfair to hire a data engineer and assign him/her the task of building Predictive models or hiring a data Scientist to be told to develop BI reports. Strategists will have to spend the time required to understand the nuances of skills and expertise required in a data organisation but it will be worth it, to retain and grown the talent pool required for a Data-driven organisation.
3.  Patience – Creating a Data-driven organisation will require ample amounts of patience and perseverance. If data has not been involved in the decision making process, earlier,  then the data is most probably not in a state that can be used readily or maybe there is no or not enough data to begin with! In that case, it has to start with gathering the data required to achieve the business goals. Transaction systems have a very different database design than the data storage mechanisms used for Analytics purposes, which entails a design and architecting process before being able to analyse the data. Moreover, as Analysts dig into the transaction data, they surely will encounter non-existence of relevant data, data retrieval issues and unearth data quality issues and data integration problems due to the existence of data silos. In a data-driven organisation, all data sources are integrated to provide a single enterprise version of truth, irrespective of Customer data or Sales or Marketing data. A data platform, integrating all business data sources, ensuring quality and data integrity and security is a time-consuming process. Organisations will have to take this lead time into consideration when strategizing a Data-driven decision making approach.
4. Organisational Culture – The purpose of a Data-driven organisation is to empower employees by means of data and information sharing to enable the organisation to collectively achieve the business goals. This approach requires employees to be data aware and not use gut feelings to make decisions and this could be a whole new approach for many. This new way of working requires organisational change management, educating people to use facts and figures to arrive at conclusions and make decisions. If an organisation is fairly data aware, in the sense that metrics are used to measure certain processes, in order to turn Data-driven , the organisation has to take steps to use data proactively (read Predictive Analytics) and not just summarise events that happened. The CDOs/ CMOs need to drive data awareness by showcasing quick wins and success cases of Data-driven approaches, as a means to use data as the foundation in every decision making process.

Some organisations may take longer to implement a Data-driven culture than others but there is no way an organisation can become Data-driven, just like that, one fine day! If the CDOs can gauge that the organisation has a longer incubation period then it is good to start with raising data awareness and introducing a BI/ Datawarehousing team. It is not recommended to directly leap on to AI, hiring data scientists, to be then left in a lurch if the organisation and the infrastructure are pretty rudimentary to handle their expertise.

A Data-driven organisation culture starts with the right strategy in place, followed by the right people and technology, evaluating and optimising the entire process, intermittently.

Six Great Marketing Hacks For Startups on a Shoestring Budget

 

Startups are built around the idea of a product, with entire teams focused on crafting the perfect app experience. But once a product is built and ready for an audience, or if it needs enough traction to secure a round of funding, what should happen next?  The era of “build it and they will come” is over –  to spread the word about the product and harness an audience, marketing is key. Venture capitalists usually look for functional products that customers are already using and a plan to continue growing before investing. Whether your goal is to bootstrap or to raise venture capital for your startup, you won’t get anywhere if you can’t both build it and sell it.

In order to secure funding, startups are required to boast a substantial customer base and in order to be able to acquire customers, marketing is essential. But the good news is that marketing is not all about expensive marketing automation tools and hiring digital marketing specialists. Not only is social media marketing a boon for startups, there are a few more hacks to market the frugal way!

Let’s get started with some of the ways to get the word out about a startup product:

1. A picture is worth a thousand words

Pictures tell a story that are otherwise difficult to articulate.  When it comes to customer engagement, Instagram rules. People sharing the same interest connect through hashtags. Using catchy feel-good hashtags that are associated with the product give a boost to the promotion. For example, if promoting the launch of a toothpaste a peppy hashtag like #Riseandshine will win the toothpaste company many followers as it strikes a chord. Posting often and relevant images — not just about the product but images of related events, products and emotions — will keep the customer engaged and interested. Posting on Instagram does not of course involve money.

French photo printing company Cheerz lets customers easily print their mobile phone pictures from Instagram or Facebook in formats such as Posters, or Magnets. Cheerz use creative and feel good campaigns on Instagram targeting holiday seasons or festivities, to promote their product to print personal images meanwhile inspiring and offering inspiration about home decor with the pictures involved.

Engaging with customers to make them feel part of the product is a sure shot way to win some accolades and loyalty. Starbucks has implemented customer engagement on Instagram by resharing images posted by its fans, earning them goodwill. Here’s an example of Starbucks reaching out to an Instagram user, asking permission to use a great shot of its classic red holiday mug next to a Christmas tree as its Facebook cover image.

A big brand like Mercedes Benz has also turned to Instagram to attract potential customers through a brilliant Instagram campaign to promote the GLA sports utility vehicle by letting the targeted segment of consumers customise and design their new car. The Instagram campaign #gla_build_your_own allows the customers to create their own version of their coveted new car by choosing colour, wheels and roofs. The campaign resulted in increasing the site visits and brand awareness, manifold.

2. Social Media Marketing

Facebook and twitter are powerful mediums of marketing. Getting customers engaged is the key, by replying to their comments and retweeting. It is, however, imperative to analyze traffic generated by Facebook and twitter to get to know the audience that is interested in the product.

Creating Facebook ads using Ads Manager to target the right audience using parameters to define the appropriate age, demographics, interests and behaviors is quite a low cost marketing gimmick. Facebook also provides an easy way to retarget customers who have abandoned shopping carts or have shown an interest in a product but have not taken the leap to purchase, this post elaborates the steps involved in retargeting customers.

Facebook for Businesses provides many easy to use features for marketing, at pretty low costs. Facebook marketing enables startups to target the right audience by creating custom audiences and boosting posts to increase outreach, geo targeting by promoting products using location based marketing and Facebook also provides insights to be able to measure campaign effectiveness. Here are some great success stories that have been able to create brand awareness and generate revenue using Facebook marketing.

Youtube video advertising is again a great way to spread the word about your product at low costs and it includes features like targeting through customer segmentation and analytics to measure and analyze the traffic..

Analyzing Facebook and Twitter data generates great insights about consumer demographics and sentiments. In order to analyze Facebook and Twitter, open source code like R and Python are freely available on the internet, which when connected to Facebook and Twitter APIs can extract and analyze data regarding customer names, age, geographic location, number of likes, shares , comments, popular hashtags associated with the tweets. One does not have to be an ace programmer to be able to connect to these APIs using R and Python, there are numerous blogs and websites, stating step by step code snippets to connect to Facebook and Twitter APIs to extract and analyze data. Here is an example of steps to connect, fetch data from Facebook and analyze it.

Not only can the Facebook, Instagram, LinkedIn, Pinterest and Twitter APIs be used to generate online customer behavior and help target the befitting audience, the APIs can also be used to conduct competitive analysis by analyzing hashtags that are associated with a competitor’s products.

Social media management product Hootsuite offers a free version to manage up to 3 social networks from a single easy-to-use interface. Basic Analytics, Reports and basic scheduling capabilities that make the life easier for a marketeer, at no cost. Hootsuite also makes it easier to use influencer marketing to promote products by tracking online conversations for hashtags or keywords to spot influencers. Influencers could be buyers themselves but could also very well be bloggers or writers with social influence. When influencers express an interest in a product the outreach is impactful.

3. Utilising trial versions of marketing tools

Trial versions of most marketing tools, are available free of cost for customers, for a limited period. This is not a long term arrangement but while the startup is struggling to get itself noticed, at the same time trying to keep the costs low, it is a blessing. Marketing tools like Marketo, Hubspot, Adobe, Tableau all have trial versions of their tools that can be downloaded for free for a period of 15-30 days, not to forget the free version of Google Analytics. Utilizing the trial version tools also serve as an opportunity to evaluate marketing tools available in the market, that are most suited for the job at hand.

Webhose provides access to data from several sources such as news sites, social sites, blogs and from several different technical platforms with quick integration capabilities, requisites that expedite the new age data driven marketer’s job. Webhose comes with a free trial period which can be utilised to analyze multichannel data sources.

4. Free Market Survey tools

Launching a new product entails verifying that a potential market for the product exists. And if similar products already exist in the market, then it is worth checking information used to identify potential customer segments, opportunities and problems faced to further optimize marketing efforts. Free market survey products such as SurveyMonkey help startups to gather consumer insights and feedback  to optimize their product.

San Francisco based Happy Goat Caramel gained insights about which factors of their product mattered most to their customers by using information gathered through SurveyMonkey. The feedback gathered also helped Happy Goat make strategic decisions about their product and pricing in order to accomplish their growth aspirations.

Market research data that can be fetched from web crawlers or market research companies helps companies gather information about marketing campaigns designed by their competitors. Data regarding the competitor’s ad spend, methods used, SEO techniques can be helpful in creating ad copy optimization.

While working for a media giant, I was involved in a marketing effort where the aim was to increase the market share for the company in regards to online advertising. We gathered data about the big buyers and their ad spend behavior for example if they invested mostly in mobile advertising or print, the kind of advertising – native or branded video, if the method used for ad buying was programmatic or through media agencies, from Market Research companies. This information about customer ad spend gave us an edge over the competitors by being able to target the big spenders in a much more personalised manner through marketing campaigns, thereby increasing the share of wallet. This example can be used for any business case, using market research data to figure out ways to have an edge over competitors marketing strategy can yield only good results.

Data regarding consumer behaviour, preferences, trends, interesting segments are gathered by market research companies, which are usually available, on purchase. But a few government agencies, websites and nonprofit organizations make their market research data open to the general public, enabling SWOT analysis (analysis of Strength, weaknesses, opportunities and threats for a business) saving startups, on meagre budgets, additional expenses.

5. Crowdsourcing content and outreach

Improbable as it may seem but it is very much happening. Startups can engage customers by running contests for the wittiest hashtag and then getting the winner featured on their social media channels. Who does not like fame? Similarly, startup organizations can conduct surveys on Facebook, Twitter and LinkedIn to gain insights about the general opinion about the product. As far as advertising is concerned, customers would feel empowered and a part of the product development journey if they contribute to social media advertising. Considering the toothpaste launching company as an example, starting a campaign by asking customers to post pictures that they can associate with #riseandshine will not only drive more web traffic and create brand awareness, but also contribute to great brand storytelling.

Chaordix Crowd Intelligence process and platform facilitates hundreds of thousands of high-value customer submissions, comments from social media platforms to gather feedback on new products, services and marketing campaigns. Chaordix’s small-business-centric Pro plan costs only $99 per month.

This marketing gig by Unilever to promote Magnum ice creams by letting customers design their own ice creams was a huge hit with customers posting images on social media with a hashtag #magnumstockholm, acting as brand ambassadors.

When it comes to crowdsourcing content marketing – there are numerous talented bloggers and writers that wish for nothing more than a platform to showcase their writing skills, liasoning with such skillful writers to roll out great content on the sites is a win-win situation for all parties involved.

Refuga, a travelpreneur site has a team of crowdsourced writers spread across the globe, writing content for them, not only to showcase their writing to a worldwide audience of entrepreneurs but also winning a free trip per year for adequate amount of high quality articles in return. It is a mutually beneficial collaboration and has worked out well for me, personally.

6. Marketing swag

To create brand familiarity, giving away branded merchandise at events or as contest prizes or as an incentive is a great idea. But again, it does not necessarily imply a huge cost. Surely there exist startup companies that have tote bags, personalised mugs or USB sticks as their products. Collaborating with such companies to co-sponsor advertising and promote brand awareness, while sharing costs, is a wise thing to do.

New Relic is a SaaS (Software as a Service) application performance management that provides comprehensive, real time visibility into web and mobile applications. New Relic uses marketing swag in the form of their Data Nerd t-shirt which acts as a motivator for their buyers to try the software and deploy it. And of course the subsequent tweets and Instagram images of the t shirts only add to the website traffic volume and brand awareness.

There could be countless other ways to market products on a shoestring that I will add to the list as and when I get brainwaves. Please feel free to share your ideas, views or tips about marketing on a limited budget.

Data integration is not a choice!

Every organization irrespective of industry has several business processes, each business process being supported by several IT products. Each of these IT products have an insurmountable amount of information that can generate insights which are paramount for any organization. Businesses that have been around for a while have obsolete processes and legacy systems that support the same. A typical organization independent of industry has transaction processing systems, CRM systems, ERP, billing and business analytics solutions. Each solution in itself is a silo if not integrated with the rest of the solutions. Granted that each of these solutions harbour valuable information but the the information residing in each system does not generate a holistic view of the business.

Integrating the silos is a Herculean task, or so it may seem, if the solutions are outdated and do not support APIs, plug-ins and adapters. Most CRM, ERP, Marketing automation products, lately are equiped with some form of connector, enabling data blending. If an organization has systems that do not support the above, then it is wise to migrate or upgrade the solutions to versions compatible with data extraction. Migrating legacy systems is a rocky road but the trade off being elimination of data silos. Often the implementation cycle of new software solutions are so long that the idea becomes outdated even before the roll out. Ofcourse there exist solutions with shorter time-to-market, for example data analytics platform that are run on Spark have a faster implementation cycle and are scalable, providing the flexibility that growing businesses need.

It was not long ago that marketing and data analytics borders got blurred due to new business needs. This has resulted in complex technological challenges. Not all businesses have the budget and resources to invest in migrating and upgrading most of the legacy systems. But in order to appease todays demanding customers, data integration is the key. No customer would like to remember or rummage through their homes to find old reciepts or mails when they call the customer care for a service or to complain. They would very much expect that on identifying themselves, the customer care representative not only solves their grievances but also comes up with suggestions to improve their customer lifecycle, which can be only attained by integrating data from disparate systems to gain a 360 degree view of the customer journey. Data integration, thus is a matter of being in business or out.

To start with, businesses should identify each data silo that exists and the function that each of them fulfill. (There maybe exist examples of one business process that is fulfilled by several software solutions. If an organization lacks data governance, then the number of redundant solutions and products can be plenty.) Listing and mapping business processes to softare solutions clarifies the current architecture. The next process is

• To identify the to-be roadmap
• Map solutions that support data blending, to each of the business process 

The solutions that are adapted for new age businesses require to embody the following characteristics:

• Easy to implement
• Short implementation time
• Compatability with a wide range of disparate systems
• Easy to implement data security and access rights
• Scalable
• Forward compatible

Businesses need technology that support business gain and growth and the ever changing rules of the game (read disrutption).