Configure Timeseries Anomaly Detection

Objective

This document provides instructions on how to enable anomaly detection using time series analysis on the metrics of your application. The Volterra Time Series Anomaly (TSA) detection is supported for the Request Rate, Error Rate, Latency, and Throughput (RELT) metrics. The Volterra TSA is performed using advanced machine learning upon enabling through configuration. To know more about the TSA concepts, see Behavioural Firewall.

The TSA detection monitors and alerts about the following types of abnormal traffic patterns

  • Unusually large spikes (DoS attack and genuinely high traffic)
  • Sudden drops — may indicate reachability issues
  • Seasonality patterns — these are periodic patterns
  • Missing periodic peaks — may indicate problems with client application
  • Unexpected peaks or drops

Using the instructions provided in this document, you can enable the TSA detection for metrics of your application and monitor the related detected anomalies in the Volterra service mesh.


Prerequisites


Configuration

The following image describes the configuration work-flow for enabling TSA detection for your application metrics:

seq tsa
Figure: Work-flow for Enabling TSA

Configuration Sequence

The following table presents the sequence of activities in enabling the TSA detection:

Activity Description
Create App Type Create app type and configure the TSA features.
Create App Settings Select metrics and components such as nodes, edges, or virtual hosts for TSA and associate them with the app type.
Monitor Anomalies and Alerts Monitor the service mesh or virtual host to check for anomalies detected and reported by TSA.

Create App Type

To enable anomaly detection for your application services, it is required to first enable TSA for those services using the app type object.

The app type object is created in the shared namespace. The virtual hosts of that app type in different namespaces need to be assigned with the label of the app type object.

Perform the following to create app type and enable generating the anomaly model.

Step 1: Log into the VoltConsole and navigate to app type configuration.

Change to the shared namespace and select Security from the configuration menu and AI & ML -> App Types from the options. Click Add app type to start app type creation.

nav atype
Figure: Navigate to App Type Configuration

Step 2: Configure app type object settings.

Enter the configuration in the app type object creation form as per the following guidelines:

  • Enter a name for the app type. This is the value for the app type label to be assigned to the virtual hosts for which the TSA needs to be enabled.
  • Click Add features in the Features section and select a type for the AI/ML Feature Type from the drop-down list as per the following guidelines:

    • Select Business Logic Markup for enabling analysis on interactions between the services.
    • Select Timeseries Analysis for enabling analysis on RELT metrics.
    • Select Per API Request Analysis for enabling detection per API request.

Note: You can add all the features using the Add feature option.

  • Click Add app type to complete creating the app type object.

apptype cnf
Figure: App Type Feature Configuration


Assign App Type Label to Virtual Hosts

After creating the app type, it is required to assign the app type label to the virtual hosts for which you want to enable TSA detection.

Note: Enabling TSA detection for all virtual hosts in a namespace requires you to apply the app type label to all virtual hosts in that namespace.

Perform the following to assign the app type label to your virtual hosts.

Step 1: Log into the VoltConsole and navigate to virtual host management.

Change to your application namespace and select Manage from the configuration menu and Virtual Hosts from the options. Click ...->Edit to edit virtual host configuration.

vhost edit
Figure: Navigate to Virtual Host Edit Configuration

Step 2: Assign the app type label.
  • Select ves.io/app_type for the Labels field and type.

at label
Figure: App Type Label Selection

  • Type the name of the app type object created in the previous step and click Assign Custom Value to add the app type label.

label value
Figure: App Type Label Addition

  • Click Save changes to apply the label to the virtual host.

Create App Settings

After creating an app type with the TSA feature enabled, it is required to associate it with the metrics and sources for which the anomaly detection is required. This is done by configuring the app settings object.

The metrics are RELT metrics and sources are of the following types:

  • Services
  • Service interactions
  • Virtual hosts

Perform the following to create the app type object.

Step 1: Navigate to app settings configuration and start app setting object creation.

Change to the namespace where your application deployment is created and virtual hosts are configured. Select Security from the configuration menu and App Settings from the options under the AI & ML field. Click Add app setting to start app setting creation.

nav app setting
Figure: Navigate to App Setting Configuration

Step 2: Enter basic configuration for the app setting object.
  • Enter a name for the app setting.
  • Optionally, click Select app type ref in the App Types section and select an app type object. Click Select app type ref to add the app type to the configuration.
  • Optionally, select the anomaly types for the Anomaly Types field. You can select multiple types from the drop-down list.

app setting basic
Figure: App Setting Basic Configuration

Step 2: Add app type settings.

Click Add app type settings and enter the configuration as per the following guidelines:

  • Click Select app type ref in the App Types section and select the app type object created in the previous chapter. Click Select app type ref to add the app type to the configuration.
  • Click Add metric selector in the Timeseries Analysis Setting field and configure the following:

    • Select a source for the Metrics Sources field from the drop down.
    • Select a metric for the Metrics field from the drop down.

Note: You can add multiple metric selectors using the Add metric selector option.

This sample shows app type settings for sources as all services and metrics as request rate and error rate.

app type settings
Figure: App Setting App Type and Metric Selector Configuration

Note: You can add multiple app type settings using the Add app type settings option.

  • Click Add app setting to complete creating the app setting object.

Monitor Anomalies and Alerts

TSA detection happens as per your selection of sources in app settings and app type configuration. You can monitor the anomalies using the metrics or alerts or both. The TSA gets detected and displayed for service mesh or virtual host or both depending on your TSA configuration.

Step 1: Log into the VoltConsole and navigate to service mesh.

Change to your application namespace and select Mesh from the configuration menu and Service Mesh from the options. Click on your application from the displayed list to load its service mesh monitoring.

nav sm
Figure: Navigate to Service Mesh

Step 2: Load the service mesh metrics view.

The service mesh loads service graph by default. Click Metrics tab to load the metrics view.

The metrics view presents trend for your service metrics for a default or configured time period.

When the TSA is enabled for metrics, a shadow is shown over the metrics bars. This is called as Confidence interval. The confidence interval indicates that the metric value crossing this interval is treated as an anomaly. Such instances are marked in red color bars. Hover over or click on any bar to display the metric and confidence interval values.

sm metrics
Figure: TSA Enabled Service Mesh Metrics

Step 3: Load the service mesh alerts view.

The service mesh loads service graph by default. Click Alerts tab to load the alerts view.

Active alerts are displayed by default. Select All option to display all alerts for default interval of an hour. You can also change time interval using the Last 1 hour drop down. The value TSA Alert for the Group field indicates that the alert is an anomaly. The TSA alerts are generated for sustained anomalies.

Click > for any alert entry to load details in the JSON format.

sm alerts
Figure: Service Mesh TSA Alerts

Step 4: Navigate to the virtual host monitoring.

Change to your application namespace and select Virtual Hosts from the configuration menu and HTTP Load Balancers from the options. Click on your load balancer from the displayed list to load its monitoring view. Virtual host dashboard is loaded by default.

Step 5: Load the virtual host metrics view.

The virtual host dashboard is loaded by default. Click Metrics tab to load the metrics view.

The metrics view presents trend for your virtual host metrics for a default or configured time period.

When the TSA is enabled for metrics, a shadow is shown over the metrics bars. This is called as Confidence interval. The confidence interval indicates that the metric value crossing this interval is treated as an anomaly. Such instances are marked in red color bars. Hover over or click on any bar to display the metric and confidence interval values.

vh metrics
Figure: TSA Enabled Virtual Host Metrics

Step 5: Load the virtual host alerts view.

Click Alerts tab to load the alerts view.

Active alerts are displayed by default. Select All option to display all alerts for default interval of an hour. You can also change time interval using the Last 1 hour drop down. The value TSA Alert for the Group field indicates that the alert is an anomaly. Click > for any alert entry to load details in the JSON format.

vh alerts
Figure: Virtual Host TSA Alerts


Concepts


API References