Uncover Hidden Trends with Anomaly Detection Techniques
Let us know all about Anomaly Detection. How can it be utilized?
Dec 12, 2022
What is Anomaly Detection?
Anomaly detection is the process of identifying unexpected patterns in data, which is used in various industries such as finance, manufacturing, and bio. Retentics implemented this system in order to spot abnormal patterns in customer retention and alarms the user whenever anomaly is detected.
There are 3 types of anomaly detection methods depending on dataset and presence of labels :1. Supervised Anomaly Detection (if normal and abnormal samples, and labels exist)2. Semi-Supervised Anomaly Detection (if only normal samples and labels exist)3. Unsupervised Anomaly Detection (Assuming samples are normal, if labels don’t exist)
Since we do not have any labels of anomalies, Retentics uses one of Unsupervised Anomaly Detection models, Autoencoder, to identify abnormal changes in customer retention related metrics.
What is an autoencoder?
Simply speaking, the autoencoder is a neural network that copies input to output; it compresses the input and reconstructs the data back to make the input and output identical. As the autoencoder trains, it calculates the reconstruction loss, which is the difference of the input and the output. Then, we check the distribution of reconstruction loss and decide the threshold that will define the anomaly. Ultimately, we make a prediction with the test set and calculate the reconstruction loss. If the reconstruction loss of the test set is smaller than the threshold, we can assume the test set is normal. On the other hand, if the reconstruction loss of the test set is greater than the threshold, we can assume the test is anomalous.
Why do we use the Autoencoder?
Some may ask if the autoencoder is really necessary. They may say why not use rule-based anomaly detection, which we decide a threshold for each company and assume as anomaly if the change exceeds the threshold. This may sound like a better option because we do not have to train a model and can check numerous numbers of combinations with simple calculation. However, based on clients, industry, channels and regions, data will have distinct characteristics. Hence, a single number may not be enough to define an anomaly. Therefore, Retentics uses the Autoencoder to apply different thresholds to each combination.
How does Retentics use Anomaly Detection?
As mentioned above, Retentics uses the Autoencoder to detect abnormal patterns in customer retention related metrics. It will automatically go over all combinations, spot anomalies if there are any, and alarms the user. With these alarms, the user can analyze the cause of the anomaly and come up with actions. Moreover, Retentics allows users to rate each alarm to filter out unhelpful cases the next time the user runs the analysis.
Share article