Create a Kubernetes Site

Objective

This guide provides instructions on how to deploy a Volterra Site as a pod on a Kubernetes (K8s) cluster. For more information on Volterra sites, see Volterra Site. A Volterra Site deployed as a pod on a K8s cluster acts as a K8s ingress controller with built-in application security. It also enables the VoltMesh features, such as discovery of services of the K8s cluster, publish of other site's services on this site, publish of this site's discovered services on other sites, etc.

Deploying a site as K8s is supported for the following:

  • Azure Kubernetes Service (AKS)
  • Amazon Elastic Kubernetes Service (Amazon EKS)
  • Google Kubernetes Engine (GKE)
  • Minikube

Note: The Volterra Site as Kubernetes supports only VoltMesh functionalities and does not support VoltStack. The supported Kubernetes version is 1.21.

Using the instructions provided in this guide, you can perform the following:

  • Deploy a site of the nodes
  • Decommission the site from the K8s cluster

Prerequisites

The following prerequisites apply:

  • The following requirements are for the managed K8s environment:

    • Managed K8s (EKS/AKS/GKE): minimum 4 vCPUs and 8 GB of memory per node.
    • Minikube: minimum 6 GB of memory per node.

Note: The driver should support HugePages for managed K8s. For example, for macOS you can use the minikube start --driver=<driver> command to start Minikube with the driver that supports HugePages. For example, use the minikube start --driver=virtualbox --memory 8192 command to start Minikube with virtualbox driver.

  • Kubernetes StorageClass with enabled Dynamic Persistent Volume Provisioner (PVC) with a minimum 1 GB space.

Note: Use the kubectl get storageclass command to check if dynamic PVC is enabled for your K8s StorageClass. The output with an entry with (default) in the name indicates that K8s storage class is enabled with dynamic PVC.


Configuration

The following image shows the workflow of deploying a Volterra Site as a pod on an existing K8s cluster:

CnfSeqCE
Figure: Configuration Sequence For Site as K8s Pod

The following video shows the deployment workflow:

Configuration Sequence

Deploying a site as a pod on an existing K8s cluster requires you to perform the following sequence of actions:

  1. Create a site token.
  2. Prepare a manifest file with the parameters required for site provisioning.
  3. Deploy the Volterra Site using the kubeconfig of the K8s cluster and the manifest file.
  4. Perform site Registration.

Create a Site Token

Create a site token or use an existing token. If you are configuring a multi-node site, use the same token for all nodes.

Step 1: Log into VoltConsole and navigate to site tokens.
  • Click on the System namespace.
  • Select Manage -> Site Management -> Site Tokens.

NameSpaceNavig
Figure: Site Tokens

Step 2: Generate a new site token.
  • Click Add site token to create a new token. This loads the Add site token form.
  • In the Name field, enter the token name.
  • In the Description field, enter a description for the token.
  • Click Add site token.

CreateSiteToken
Figure: Site Token Form

Step 3: Note down the new token.
  • Find the token previously created or choose an existing token from the list of tokens displayed.
  • Click > to expand the token details in JSON format and note down the value of the uid field.

SiteTokenUID
Figure: UID Field


Prepare the Manifest File

The manifest file contains a YAML schema used for descriptor information to support deployment of Kubernetes for a Volterra Site.

Step 1: Create the manifest file.
Step 2: Edit the configuration.
  • Edit the configuration in the Vpm section of the manifest file per the following guidelines:

    • In the ClusterName field, type your cluster name.
    • In the Latitude and Longitude fields, type the latitude and longitude values.
    • In the Token field, type the site token.
  • Save your changes.

This image provides a manifest file example:

Manifest
Figure: Manifest Configuration Example

Note: You can also set the cluster name, latitude, and longitude when you register a site.


Deploy Volterra Site

Step 1: Deploy the Volterra Site using the kubeconfig file for the K8s cluster and manifest file.
  • Type kubectl --kubeconfig=<kubeconfig-of-existing-k8s-cluster> apply -f <manifest>.yml.
kubectl --kubeconfig=<kubeconfig-of-existing-k8s-cluster> apply -f <manifest>.yml

This example displays the sample output of the kubectl command:

namespace/ves-system created
daemonset.apps/volterra-ce-init created
serviceaccount/vpm-sa created
role.rbac.authorization.k8s.io/vpm-role created
rolebinding.rbac.authorization.k8s.io/vpm-role-binding created
clusterrolebinding.rbac.authorization.k8s.io/ver created
configmap/vpm-cfg created
statefulset.apps/vp-manager created
Step 2: Verify that the K8s pod for the Volterra Site was created.
  • Verify that the pod with the vp-manager-0 under the NAME column indicates that the site pod was created.
kubectl get pods -n ves-system -o=wide
NAME                     READY   STATUS    RESTARTS   AGE     IP            NODE                                NOMINATED NODE   READINESS GATES
ves-system    volterra-ce-init-5wsmb                 1/1     Running   0          32m   10.240.0.5   aks-nodepool1-29573508-vmss000001   <none>           <none>
ves-system    volterra-ce-init-sbjqp                 1/1     Running   0          32m   10.240.0.4   aks-nodepool1-29573508-vmss000000   <none>           <none>
ves-system    vp-manager-0                           1/1     Running   0          29m   10.244.1.3   aks-nodepool1-29573508-vmss000001   <none>           <none>
Step 3: Check if the registration request was created.

You can check the request status from the VoltConsole or the kubectl command-line tool.

  • To check with VoltConsole:

    • Log into VoltConsole and then navigate to the System namespace.
    • Click Manage -> Site Management -> Registrations -> Pending Registrations.
  • To check with kubectl:

    • Type kubectl logs vp-manager-0 -n ves-system.
kubectl logs vp-manager-0 -n ves-system

This example provides the sample output of the command:

  "message": "Config isn't available. Current state: NEW, registration 3c507977-a1d6-44da-b470-5e06eb055d06 must be manually APPROVED and then automatically ADMITTED. Object status: \u0026StatusType{ObjectStatus:\u0026ves_io_schema4.StatusType{Status:,Reason:Registration created, waiting for approval,Code:0,},CurrentState:NEW,}"

Register the Site

After the Volterra Node is installed, it must be registered as a Site in VoltConsole.

Note: The USB allowlist is enabled by default. If you change a USB device, such as a keyboard after registration, the device will not function.

Single-node Site Registration

Step 1: Navigate to the site registration page.
  • Log into VoltConsole and then navigate to the System namespace.
  • Click Manage -> Site Management -> Registrations.

AcceptReg
Figure: Site Registration

Step 2: Complete site registration.
  • Under Pending Registrations, find your node name and then click the blue checkmark. Pending Registration
  • In the form that appears, fill in all required fields with the asterisk symbol (*).
  • Enter a latitude value and a longitude value.

  • Enter other configuration information, if needed.
  • Click Save and Exit.
Step 3: Check site status and health.

It may take a few minutes for the site health and connectivity score information to update.

  • In the System namespace, click Sites -> Site List.
  • Click on your site name. The Dashboard tab appears, along with many other tabs to inspect your site.
  • Click the Site Status tab to verify the following:

    • The Update Status field has a Successful value for the Volterra OS Status section.
    • The Update Status field has a Successful value for the Volterra Software Status section.
    • The Tunnel status and Control Plane fields under the RE Connectivity section have up values.

Multi-node Site Registration

Step 1: Navigate to the site registration page.
  • Log into VoltConsole and then navigate to the System namespace.
  • Click Manage -> Site Management -> Registrations.

AcceptReg
Figure: Site Registration

Step 2: Accept the registration requests.

Registration requests are displayed in the Pending Registrations tab.

  • Click Accept to accept the registration requests from the master-0, master-1, and master-2nodes.
  • Enter the same values for the following parameters for all the registration requests:

    • In the Cluster name field, enter a name for the cluster. Ensure that all master nodes have the same name.
    • In the Cluster size field, enter 3. Ensure that all master nodes have the same cluster size.
  • Enter all mandatory fields marked with the asterisk (*) character.
Step 3: Check site status and health.

It may take a few minutes for the site health and connectivity score information to update.

  • In the System namespace, click Sites -> Site List.
  • Click on your site name. The Dashboard tab appears, along with many other tabs to inspect your site.
  • Click the Site Status tab to verify the following:

    • The Update Status field has a Successful value for the Volterra OS Status section.
    • The Update Status field has a Successful value for the Volterra Software Status section.
    • The Tunnel status and Control Plane fields under the RE Connectivity section have up values.

Verify Volterra Services

Verify that Volterra services were started after site registration.

  • Type kubectl get pods -n ves-system -o=wide.
kubectl get pods -n ves-system -o=wide

This example provides the sample output of the command:

NAME                           READY   STATUS      RESTARTS   AGE   IP           NODE                                NOMINATED NODE   READINESS GATES
etcd-5779568655-td4wq          2/2     Running     0          41m   10.244.1.4   aks-nodepool1-29573508-vmss000001   <none>           <none>
etcd-defrag-1587094200-58jjb   0/1     Completed   0          33m   10.244.1.5   aks-nodepool1-29573508-vmss000001   <none>           <none>
ver-0                          13/13   Running     3          41m   10.244.0.8   aks-nodepool1-29573508-vmss000000   <none>           <none>
volterra-ce-init-5wsmb         1/1     Running     0          75m   10.240.0.5   aks-nodepool1-29573508-vmss000001   <none>           <none>
volterra-ce-init-sbjqp         1/1     Running     0          75m   10.240.0.4   aks-nodepool1-29573508-vmss000000   <none>           <none>
vp-manager-0                   1/1     Running     2          72m   10.244.1.3   aks-nodepool1-29573508-vmss000001   <none>           <none>

The site deployed as a K8s pod appears in VoltConsole where you can deploy VoltMesh services, like the other sites.


Perform Scaling for a Site

You can perform scale-up or scale-down of site nodes by changing the replicas of the vp-manager pods.

Note: Scale-up for the sites also creates the VER pods, as there can be only one VER pod per node.

Step 1: Update the replicas using the kubeconfig file of your K8s cluster.
  • Type kubectl --kubeconfig=<kubeconfig-of-existing-k8s-cluster> edit statefulset/vp-manager -n ves-system.
kubectl --kubeconfig=<kubeconfig-of-existing-k8s-cluster> edit statefulset/vp-manager -n ves-system
Step 2: Verify that the new pods have started.
  • Type kubectl get pods -o=wide -n ves-system.
kubectl get pods -o=wide -n ves-system

This example assumes that the replicas are set to 2:

NAME                           READY   STATUS      RESTARTS   AGE   IP            NODE                                NOMINATED NODE   READINESS GATES
etcd-5bcbfc8689-8fx4g          2/2     Running     0          19h   10.244.1.23   aks-nodepool1-29573508-vmss000001   <none>           <none>
etcd-defrag-1587184200-dg765   0/1     Completed   0          42m   10.244.1.52   aks-nodepool1-29573508-vmss000001   <none>           <none>
ver-0                          13/13   Running     2          16h   10.244.0.12   aks-nodepool1-29573508-vmss000000   <none>           <none>
ver-1                          13/13   Running     0          17m   10.244.1.53   aks-nodepool1-29573508-vmss000001   <none>           <none>
volterra-ce-init-5wsmb         1/1     Running     0          26h   10.240.0.5    aks-nodepool1-29573508-vmss000001   <none>           <none>
volterra-ce-init-sbjqp         1/1     Running     0          26h   10.240.0.4    aks-nodepool1-29573508-vmss000000   <none>           <none>
vp-manager-0                   1/1     Running     0          12m   10.244.1.54   aks-nodepool1-29573508-vmss000001   <none>           <none>
vp-manager-1                   1/1     Running     2          15m   10.244.0.14   aks-nodepool1-29573508-vmss000000   <none>           <none>

Decomission a Site

Decommissioning a site requires you to de-register the site from VoltConsole, and then delete that site.

Step 1: Navigate to the list of registered sites.
  • Log into VoltConsole and select Manage -> Registration from the options.
  • Click the Other Registrations tab.
Step 2: Perform decomissioning.
  • Find your site and then click ... -> Decommission.
  • Click Decommission in the confirmation window to confirm the de-registration of the site.
Step 3: Delete the resources for the site.
  • Use the kubectl tool to delete resources.
kubectl --kubeconfig=<kubeconfig-of-existing-k8s-cluster> delete -f <vpm-manifest>.yml
Step 4: Delete the decommissioned site.
  • Open VoltConsole and select Sites -> Site List.
  • Select your site from the list and click ..., and then click Edit to open the site edit form.
  • Click Delete Site.
  • Click Delete in the confirmation window to complete the operation.

Concepts


API References