George Kouvaras - 10 January 2019
One of the most popular features of the KPI Performance report in Real Customer Insights (RCI) is the “Start Analysis” button, also known as “what’s changed analysis”.
This feature addresses the missing element in any anomaly detection system – the isolation and identification of the root causes of anomalies.
Understanding what lies behind anomalies
Once an anomaly is detected, it will be highlighted on the KPI performance chart. We can also alert you on anomalies, so you know about big changes in KPI performance as soon as they happen. You can then use the “what’s changed analysis” to answer key questions such as:
- Is it a global issue (i.e. across the whole site)?
- Is it a specific timing point or traffic volume-related issue (e.g. has response start time slowed down site-wide)?
- Can we pinpoint the segments which seem most responsible for the observed anomaly? (Note that a segment may contain one or more elements, e.g. users running Chrome on Windows 10.)
- And, most importantly, does this anomaly really matter? In other words, is it having an effect on my KPIs?
How it works
“What’s changed analysis” encompasses a range of machine learning methods in order to identify, describe, and quantify the properties that significantly influenced the anomaly. It works by comparing the actual, observed values of each metric with the expected ones and quantifying this deviation.
KPI performance and what's changed analysis in Real Customer Insights
The basic elements of this analysis include:
- How much the volume has changed (to account for, e.g. when a slow segment increases in volume or a new, fast segment appears).
- A score function to quantify the change from the expected.
- A clustering algorithm that puts together similar segments.
All of this means that Eggplant’s “what’s changed analysis” enables better understanding about any anomaly, distinguishing the signal from the noise and providing clients with real, actionable insights.
Try Real Customer Insights for free.