手淘再推新利器Holmes：一站式智能化异常检测平台-InfoQ

指标监控关乎稳定性，但随着数据量的增加、指标的复杂周期性和模式变化的动态性，基于阈值/同比环比的规则难以适用，而且复杂的领域知识导致为每条指标配置相应的规则费时费力，无法应用在大规模数据监控上。在监控的有效性方面，传统的规则报警无法智能识别季节性，也经常受到噪声/抖动数据的干扰而导致误报，固定的规则以及阈值更无法进行提前预警。

前言

在上面的背景下，淘系技术质量团队打造了一款基于 AI 算法的异常检测平台，Holmes 是一款智能化、轻量级、易接入、可扩展的异常检测平台，使用基于 AI 的异常检测算法，替代传统的规则监控方案。解决规则告警系统准确率低、时效性低、规则配置复杂与耗费人力等诸多问题。

Holmes 目前已经在阿里内进行开放，覆盖淘宝、千牛、优酷、钉钉、淘宝直播等多个应用，核心指标量 200+，整体准确率在 90%以上，算法调用量高达 1000 万余次，有效进行故障的提前预警 20 余次。

平台理念

Holmes 通过 AI 算法对业务指标进行检测和预测，从而实现智能化报警，弥补了规则监控的不足，减少误报和漏报情况。提供一站式的异常检测服务，满足时序数据的实时检测，提供多种接入方式，利用机器学习和深度学习方式，定制化学习模型，有效提升报警覆盖率，以智能化方式进行稳定性的监控和保障。

特点：
学习历史数据，分析当前指标曲线趋势是否异常
基于以往数据，进行预测未来指标走势
优势：
算法检测代替规则检测
告警准确率高
更早发现异常情况
可适应业务发展带来的趋势变化

解决的异常场景：

系统架构

配置化流程

通过 4 步简单配置进行指标的接入和算法选择，轻松开启智能异常检测。

算法概览

Holmes 融合了多种检测、预测的时序算法，检测异常响应速度快，预测数据走势准，二者的完美结合，奠定了异常检测的基础能力，同时平台也支持扩展算法的能力，以适应新增场景的特殊需求。

检测算法：

✎ Statistic

In statistics, the 68-95-99.7 rule is a shorthand used to remember the percentage of valuesthat lie within a band around the mean in a normal distribution with a width of two, four andsix standard deviations, respectively; more accurately, 68.27%, 95.45% and 99.73% of the valueslie within one, two and three standard deviations of the mean, respectively.

✎ Ewma

In statistical quality control, the EWMA chart (or exponentially weighted moving average chart)is a type of control chart used to monitor either variables or attributes-type data using the monitored businessor industrial process’s entire history of output. While other control charts treat rational subgroups of samplesindividually, the EWMA chart tracks the exponentially-weighted moving average of all prior sample means.

✎ Polynomial

In statistics, polynomial regression is a form of regression analysis in which the relationshipbetween the independent variable x and the dependent variable y is modelled as an nth degree polynomial in x.

✎ IsolationForest

The IsolationForest ‘isolates’ observations by randomly selecting a feature and thenrandomly selecting a split value between the maximum and minimum values of the selected feature.

✎ XGBoost

XGBoost is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way. The same code runs on major distributed environment (Hadoop, SGE, MPI) and can solve problems beyond billions of examples.

预测算法：

✎ Tensorflow-LSTM

Long Short-Term Memory layer - Hochreiter 1997.预测效果图：

✎ Facebook-prophet

Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.