Alerts with a Dynamic Baseline

Dynamic Baseline Alerts Now Automatically Find the Best Algorithm for You

We utilized a single algorithm when we first launched Dynamic Baseline Alerts late last year, and it covered a lot of ground and functioned well in several situations.

Since then, we’ve talked to consumers and done even more math to develop new methods to improve Dynamic Baseline Alerts. We weren’t interested in doing the arithmetic for the sake of doing it. Our dedicated applied intelligence engineering team was tasked with identifying techniques for solving real-world challenges for our customers, all while working with large amounts of data. For more than 15,000 clients, this means monitoring over a billion events and data per minute.

As always, our goal is to take care of as much of the labor as possible so you can concentrate on your systems and customers. Some solutions still require you to choose the seasonality of a metric or a specific algorithm for a particular statistic. We can now take care of everything for you with our most recent enhancements.

Seasonality was discovered automatically.

Seasonality is the underlying periodic pattern of a time series. Our first iteration of baselines relied on a seasonality system to detect patterns connected to the weekday, hour of the day, and minute of the hour. It allowed us to discover cyclical trends and regular usage patterns that vary by day and hour (for example, people using a website during business hours Monday through Friday). That’s a lot of ground to cover!

However, many other sorts of seasonality can occur, and we wanted to encourage them as well. It is where auto-discovered seasonality comes in. The applied intelligence engineering team developed a signal processing technique known as Fast Fourier Transforms (FFTs) to overcome this. FFTs can be used to find the underlying frequency in a time series. Our systems use FFTs to find good candidates for seasonality—cycles that don’t match the time of day, such as something that happens every three hours—and then compare them to historical metric data to see if they perform better than the default seasonality.

​​Every time, the Ensemble algorithm selects the best algorithm.

We devised a mechanism to identify the optimum fit when introducing auto-discovered seasonality. The time-series data’s recency, trend, and seasonality are all components in our base algorithm. However, for some data streams, a different approach may yield a better prediction. We now select the method best matches that particular time series using our new unsupervised ensemble system.

The ensemble selector assesses the performance of the different algorithms every minute and chooses the one with the best results. We use exponential decay to look at earlier data and weight performance toward a more recent version. The MASE (mean absolute scaled error) statistical method is used by our evaluator to determine the optimum fit. (See our blog post How We Find the Best Algorithms for Dynamic Baseline Alerts for more information on MASE.)

We are currently comparing four options: triple exponential smoothing with found seasonality, triple exponential smoothing with default seasonality, double exponential smoothing (just recency and trend variables), and single exponential smoothing (recency only).

Surprisingly, a more straightforward approach, such as single exponential smoothing, is frequently a better fit than the “fancy” triple exponential smoothing. The seasonality factor in triple exponential smoothing can exacerbate noise unrelated to the data’s behavior for data with no discernible seasonality.

For example, we found that Holt-Winters (triple exponential smoothing with default seasonality) had the best match in a sample of several thousand metrics time series, followed by the considerably simpler single exponential smoothing. We came in third with the newly established learned seasons.

Additionally, we can use our autonomous ensemble selection to add additional algorithms anytime we identify an opportunity to improve accuracy.

The applied intelligence engine’s math nerds continually seek new methods to better our systems. As a pure SaaS firm, we enjoy that we can ship a tremendous new enhancement as soon as we finish it so that all of our clients can benefit right away.

Enteros

About Enteros

Enteros offers a patented database performance management SaaS platform. It proactively identifies root causes of complex business-impacting database scalability and performance issues across a growing number of RDBMS, NoSQL, and machine learning database platforms.