Techaisle Analyst Insights

Predictive Analytics (PA) is emerging as an important tool in the area of business decision. Predictive Analytics primarily deals with making a forecast based on several inputs. In this and the blogs that follow I will share my experiences with Predictive Modelling (PM), with a view to contributing to the current knowledge base that exists in the Predictive Analytics World.

In the world of business most predictive analytical tools are quantitative where numeric data is used for building an input-output model. The output is the prediction for specific inputs. For example: A 10% increase in advertising in January will result in an increase of 1% sale in May is a typical output from predictive analytics.

Common Mistake of Predictive Modelers: Assumption of linearity

Predictive Models are largely based on statistical techniques. Multiple Linear Regression (MLR) model is what most users will confront when they look at predictive models. This model works in the background whether one is using a multiple time series or multi-level modelling.

Multiple Linear Regression Models are developed based on a crucial assumption: the output is linearly dependent on the inputs. But all experience shows that in most business situations the assumption of linearity is not valid. Hence the statistical models have a poor fit and low predictive capability. In addition, the business world also suffers from Black Swan problems that no modelling can manage with any level of confidence.

The net effect of a linearity assumption, which is ubiquitous in almost all statistical modelling, and the resultant poor fit and low predictive capability has led to frustrated user community. Hence, a business executive looks at models with suspicion and trusts ‘gut’ to make decisions.

Predicament

The predicament of Predictive Modellers’ is: How do we get away from the linearity assumption on which almost all statistical tools are based, but it is known that this assumption is a poor, in fact a very poor, approximation of the real world behaviour?

The story of our approach to modelling starts from this predicament that we have been in, along with all others, and the path that we cut out to get out of it.

Dr. Cooram Ramacharlu Sridhar (Doc)
Managing Director and Advisor, Segmentation & Predictive Modeling

Before you attempt any modelling you should first look at the inputs and outputs that you want to go in to your modelling. Here is the matrix:

What you need to do is to make a laundry list of the variables (inputs) that affect the output. Typically in a marketing company one would look at sales as the output and a whole lot of variables as inputs. Let me look at a few examples for these cells.

1.       Measurable-Controllable Variables

GRPs of your brand through TV advertising are measurable and controllable.

2.       Measurable-Not-Controllable

Inflation is measurable but not controllable

 3. Not-measurable – Not Controllable

The amount of investments made by your competition in dealer incentives is neither easy to measure accurately nor can you have any control. But this activity impacts the sales of your brand.

4. Not Measurable-Controllable

Not measurable generally refers to qualitative issues which are quite often measured by a pseudo variable, for example: Quality of your salesperson.

In your business environment if the majority of your input variables are in Cells 1 and 2, and you feel that these make a big impact, then modelling will be successful. If not, and many variables are in Cells 3 and 4, modelling will not be a success.

Most companies do not undertake this simple preliminary exercise of classifying the variables that impact their business and then hit potholes throughout the design testing and implementation.

Unclassified variables are veritable landmines. Watch out for them.

Dr. Cooram Ramacharlu Sridhar (Doc)
Techaisle

Underutilization and the complexity of managing growing data sprawl have spawned several trends during the last several years. Data-as-a-Service (DaaS) is one such trend which represents an opportunity to improve IT efficiency and performance through centralization of resources. DaaS strategies have increased dramatically in the last few years with the maturation of technologies such as data virtualization, data integration, MDM, SOA, BPM and Platform-as-a-service.

Within the corner offices of business heads, data scientists and analysts several questions are being asked:

• How to deliver the right data to the right place at the right time?

 

• How to “virtualize” the data often trapped inside applications?

 

• How to support changing business requirements (analytics, reporting, and performance management) in spite of ever changing data volumes and complexity?

In the early years most of DaaS initiatives were limited to financial services, telecom, and government sectors. However, in the past 24 months, we have seen a significant increase in adoption in the healthcare, insurance, retail, manufacturing, eCommerce, and media/entertainment sectors. This is because of massive amalgamation of extracting continuous insights from structured and unstructured data, liberation of data restricted and protected within silos to the enterprise level and the express desire to conduct real-time analytics.

Businesses are looking to solve tough data and process integration challenges as they once again begin to invest in new business capabilities. Data as a Service (DaaS) is based on the concept that the fragmented transaction, product, customer data can be provided on demand to the user regardless of geographic or organizational separation of provider and consumer. Additionally, the emergence of PaaS and service-oriented architecture (SOA) has rendered the actual platform on which the data resides also irrelevant.

Data as a Service (DaaS) has many use cases:

1. Providing a single version of the truth;

 

1. Integration of data from multiple systems of record

 

1. Enabling real-time business intelligence (BI),

 

1. Federating views across multiple domains;

 

1. Improving security and access;

 

1. Integrating with cloud and partner data and social media;

 

1. Delivering real-time information to mobile apps

Data as a Service (DaaS) brings the notion that data related services can happen in a centralized place – aggregation, quality, cleansing, enriching and offering it to different systems, applications or mobile users, irrespective of where they were. DaaS is a major enabler of the Master Data Management (MDM) concept.

Master Data Management is the Holy Grail in data management.  The focus for most businesses is on the single version of the truth or Golden Source “Product”, “Customer”, “Transaction” and “Supplier” data.  This is because:

• Fragmented inconsistent product data slows time-to-market, creates supply chain inefficiencies, results in weaker than expected market penetration, and drives up the cost of compliance.

 

• Fragmented inconsistent Customer data hides revenue recognition, introduces risk, creates sales inefficiencies, and results in misguided marketing campaigns and lost customer loyalty.

 

• Fragmented and inconsistent Supplier data reduces efficiency; negatively impacts spend control initiatives, and increases the risk of supplier exceptions.

MDM provides the plumbing that enables DaaS solutions. This plumbing allows for:

• Agility & Time to Market – Customers can move quickly due to the consolidation of data access and the fact that they don’t need extensive knowledge of the underlying data. If customers require a slightly different data structure or has location specific requirements, the implementation is easy because the changes are minimal.

 

• Cost-effectiveness – Providers can build a base with data experts and outsource the presentation layer, which makes for very cost-effective report and dashboard user interfaces and makes change requests at the presentation layer much more feasible.

 

• Data quality – Access to the data is controlled via data services, which tends to improve data quality, as there is a single point for updates. Once those services are tested thoroughly, they only need to be regression tested, if they remain unchanged for the next deployment.

 

• Cloud like Efficiency, High availability and Elastic capacity. These benefits derive from the virtualization foundation —one gets efficiency from high utilization of sharing physical servers, availability from clustering across multiple physical servers, and elastic capacity from the ability to dynamically resize clusters and/or migrate live cluster nodes to different physical servers.

We find that there is a common process that is appearing within the mid-market and customer customers focused on enabling and MDM strategy. It is the data logistics chain consisting of data acquisition, data stewardship, data aggregation and data servicing.

There is a sudden and dramatic shift in how data is handled in businesses as they are shifting away from a hierarchical, one-dimensional enterprise data warehouse initiative with fixed data sources to a fragmented network. This phenomenon has caused ripple effects throughout the old data logistics network.  Data-as-a-Service (DaaS) at its core is addressing this problem of fragmentation soundly enabled by MDM.

 

Techaisle's Big Data study of 3,360 businesses shows that mid-market businesses typically started their big data journey in one of four ways. However, the highest success rate (determined by reaching a successful implementation of a big data project within six months of initiation) was achieved when an external consultant or organization was brought in to develop proof of concept, advice on database architecture and ultimately develop the big data analytics solution.

Once a decision was made to embark on a big data deployment project, the mid-market organization tended to quickly align behind the initiative. They did realize that big data was not a typical cloud application deployment where independent department purchases could be made, nor was it infrastructure deployment where only IT could be involved. Big data required a new type of alignment between business heads, namely, Marketing, Finance, IT and a completely new set of players known as data scientists or data analysts.

Study shows that businesses are moving from “whack-a-mole” analytics to “business perspectives” to get newer insights into their operations and better knowledge about their customers as they rethink their marketing strategies because mobility, social media, and other transactional services have increased the number avenues for connections with their customers. There are many different tactical objectives for deploying big data projects but the top among them are sentiment monitoring, generating new revenue streams & improving predictive analytics. And businesses are expecting some clear cut benefits from big data analytics such as increased sales, more efficient operations, improved Customer service.