Time-series Anomaly Detection

Objective

This document provides instructions on how to enable anomaly detection using time series analysis on the metrics of your application. The Volterra Time Series Anomaly (TSA) detection is supported for the Request Rate, Error Rate, Latency, and Throughput (RELT) metrics. The Volterra TSA is performed using advanced machine learning upon enabling through configuration. To know more about the TSA concepts, see Behavioural Firewall.

The TSA detection monitors and alerts about the following types of abnormal traffic patterns:

  • Unusually large spikes (DoS attack and genuinely high traffic)
  • Sudden drops — may indicate reachability issues
  • Seasonality patterns — these are periodic patterns
  • Missing periodic peaks — may indicate problems with client application
  • Unexpected peaks or drops

Note: The time series analyses include learning time of day usage patterns.

Using the instructions provided in this document, you can enable the TSA detection for metrics of your application and monitor the related detected anomalies in the Volterra service mesh.


Prerequisites


Configuration

The following image describes the configuration work-flow for enabling TSA detection for your application metrics:

seq tsa
Figure: Work-flow for Enabling TSA

Configuration Sequence

The following table presents the sequence of activities in enabling the TSA detection:

Activity Description
Create App Type Create app type and configure the TSA features.
Create App Settings Select metrics and components such as nodes, edges, or load balancers for TSA and associate them with the app type.
Monitor Anomalies and Alerts Monitor the service mesh or load balancer to check for anomalies detected and reported by TSA.

Create App Type

To enable anomaly detection for your application services, it is required to first enable TSA for those services using the app type object.

The app type object is created in the shared namespace. The load balancers of that app type in different namespaces need to be assigned with the label of the app type object.

Perform the following to create app type and enable generating the anomaly model.

Step 1: Log into the VoltConsole and navigate to app type configuration.

Change to the Shared namespace and select Security from the configuration menu and AI & ML -> App Types from the options. Click Add app type to start app type creation.

nav atype new
Figure: Navigate to App Type Configuration

Step 2: Configure app type object settings.

Enter the configuration in the app type object creation form as per the following guidelines:

  • Enter a name for the app type. This is the value for the app type label to be assigned to the load balancers for which the TSA needs to be enabled.
  • Click Add features in the Features section and select a type for the AI/ML Feature Type from the drop-down list as per the following guidelines:

    • Select API Discovery for enabling analysis on interactions between the services.
    • Select Timeseries Analysis for enabling analysis on RELT metrics.
    • Select Per API Request Analysis for enabling detection per API request.
    • Select User Behavior Analysis for enabling user behavior analysis.

Note: You can add all the features using the Add feature option.

  • Optionally, select Enable learning from redirect traffic option for the Learn from Traffic with Redirect Response field in the Business Logic Markup Setting section.
  • Click Add app type to complete creating the app type object.

apptype cnf new
Figure: App Type Feature Configuration


Assign App Type Label to Load Balancers

After creating the app type, it is required to assign the app type label to the load balancers for which you want to enable TSA detection.

Note: Enabling TSA detection for all load balancers in a namespace requires you to apply the app type label to all load balancers in that namespace.

Perform the following to assign the app type label to your load balancers.

Step 1: Log into the VoltConsole and navigate to load balancer management.

Change to your application namespace and select Manage -> Load Balancers from the configuration menu and HTTP Load Balancers from the options. Click ...->Edit for the load balancer for which the app type label needs to be assigned.

lb edit
Figure: Navigate to load balancer Edit Configuration

Step 2: Assign the app type label.
  • Select ves.io/app_type for the Labels field and type.

at label new
Figure: App Type Label Selection

  • Type the name of the app type object created in the previous step and click Assign Custom Value to add the app type label.

label value new
Figure: App Type Label Addition

  • Click Save and Exit to apply the label to the load balancer.

Create App Settings

After creating an app type with the TSA feature enabled, it is required to associate it with the metrics and sources for which the anomaly detection is required. This is done by configuring the app settings object.

The metrics are RELT metrics and sources are of the following types:

  • Services
  • Service interactions
  • load balancers

Perform the following to create the app type object.

Step 1: Navigate to app settings configuration and start app setting object creation.

Change to the namespace where your application deployment is created and load balancers are configured. Select Security from the configuration menu and App Settings from the options under the AI & ML field. Click Add App Setting to start app setting creation.

nav asetting new
Figure: Navigate to App Setting Configuration

Step 2: Enter configuration for the app settings object.
  • Enter a name for the app setting.
  • Go to Application Type Feature Configuration section. Click on the AppType field and select the created napp type object from the drop-down list.
  • Click Configure under the Timeseries Analysis Setting field. Click Add item in the Metric Selectors section of the time series analysis setting page.
  • Select an option for the Metrics Sources field from the list of options.

    • Select All Services for enabling metric analysis for all services.
    • Select All Service Interactions for enabling analyais for all service interactions between source and destination services.
    • Select All Virtual Hosts for enabling metric analysis for all virtual hosts.
  • Select an RELT metric for the Metrics field from the list of options.

Note: You can add multiple metric selectors using the Add item option.

tsa metrics
Figure: Time Series Configuration for App Settings

Step 3: Complete app settings object creation.

Click Add app setting to complete creating the app setting object.

app settings final
Figure: App Settings Object Creation

Note: You can add multiple app settings using the Add item option.


Monitor Anomalies and Alerts

TSA detection happens as per your selection of sources in app settings and app type configuration. You can monitor the anomalies using the metrics or alerts or both. The TSA gets detected and displayed for service mesh or load balancer or both depending on your TSA configuration.

Step 1: Log into the VoltConsole and navigate to service mesh.

Change to your application namespace and select Mesh from the configuration menu and Service Mesh from the options. Click on your application from the displayed list to load its service mesh monitoring.

nav sm new
Figure: Navigate to Service Mesh

Step 2: Load the service mesh metrics view.

The service mesh loads service graph by default. Click Metrics tab to load the metrics view.

The metrics view presents trend for your service metrics for a default or configured time period.

When the TSA is enabled for metrics, a shadow is shown over the metrics bars. This is called as Confidence interval. The confidence interval indicates that the metric value crossing this interval is treated as an anomaly. Such instances are marked in red color bars. Hover over or click on any bar to display the metric and confidence interval values.

sm metrics new
Figure: TSA Enabled Service Mesh Metrics

Step 3: Load the service mesh alerts view.

The service mesh loads service graph by default. Click Alerts tab to load the alerts view.

Active alerts are displayed by default. Select All option to display all alerts for default interval of an hour. You can also change time interval using the Last 1 hour drop down. The value Timeseries-Anomaly for the Group field indicates that the alert is an anomaly. The TSA alerts are generated for sustained anomalies.

Click > for any alert entry to load details in the JSON format.

sm alerts new
Figure: Service Mesh TSA Alerts

Note: See TSA Alerts for information on time-series related alerts.

Step 4: Navigate to the load balancer monitoring.

Change to your application namespace and select Virtual Hosts from the configuration menu and HTTP Load Balancers from the options. Click on your load balancer from the displayed list to load its monitoring view. load balancer dashboard is loaded by default.

Step 5: Load the load balancer metrics view.

The load balancer dashboard is loaded by default. Click Metrics tab to load the metrics view.

The metrics view presents trend for your load balancer metrics for a default or configured time period.

When the TSA is enabled for metrics, a shadow is shown over the metrics bars. This is called as Confidence interval. The confidence interval indicates that the metric value crossing this interval is treated as an anomaly. Such instances are marked in red color bars. Hover over or click on any bar to display the metric and confidence interval values.

vh metrics
Figure: TSA Enabled load balancer Metrics

Step 5: Load the load balancer alerts view.

Click Alerts tab to load the alerts view.

Active alerts are displayed by default. Select All option to display all alerts for default interval of an hour. You can also change time interval using the Last 1 hour drop down. The value Timeseries-Anomaly for the Group field indicates that the alert is an anomaly. Click > for any alert entry to load details in the JSON format.

vh alerts new
Figure: load balancer TSA Alerts

Note: See TSA Alerts for information on time-series related alerts.


Concepts


API References