Offline Installation - Private registry

Before starting the installation, make sure the requirements are met.

Configure the registry

If your cluster is in an air-gapped network or is unable to reach the Akamas image repository, you need to copy the required images to your private registry.

The procedure described here leverages your local environment to upload the images. Thus, to interact between the Akamas and private registry, it requires Docker to be installed and configured.

Transfer the Docker images

The offline installation requires you to pull the images and migrate them to your private registry. In the following command replace the chart version to download the related list of images:

curl -sO  http://helm.akamas.io/images/1.5.4/image-list

Once the import is complete, you must re-tag and upload the images. Run the following snippet, replacing <REGISTRY_URL> with the actual URL of the private registry:

NEW_REGISTRY="<REGISTRY_URL>"

while read IMAGE; do
    REGISTRY=$(echo "$IMAGE" | cut -d '/' -f 1)
    REPOSITORY=$(echo "$IMAGE" | cut -d ':' -f 1 | cut -d "/" -f2-)
    TAG=$(echo "$IMAGE" | cut -d ':' -f 2)

    NEW_IMAGE="$NEW_REGISTRY/$REPOSITORY:$TAG"
    echo "Migrating $IMAGE to $NEW_IMAGE"

    docker pull "$IMAGE"
    docker tag "$IMAGE" "$NEW_IMAGE"
    docker push "$NEW_IMAGE"
done <image-list

This process could last several minutes, once the upload is complete, you can proceed with the next steps.

Create the configuration file

Akamas on Kubernetes is provided as a set of templates packaged in a chart archive managed by Helm.

To proceed with the installation, you must create a Helm Values file, called akamas.yaml in this guide, containing the mandatory configuration values required to customize your application. The following template contains the minimal set required to install Akamas:

akamas.yaml

# Akamas customer name. Must match the value in the license (required)
akamasCustomer: <CUSTOMER_NAME>

# Akamas administrator password. If not set a random password will be generated
akamasAdminPassword: <ADMIN_PASSWORD>

# The URL that will be used to access Akamas, for example 'http://akamas.kube.example.com' (required)
akamasBaseUrl: <INSTANCE_HOSTNAME>


global:
  imageRegistry: gitlab-runner-new.dev.akamas.io

elasticsearch:
  image: gitlab-runner-new.dev.akamas.io/akamas/elastic/elasticsearch
  
kibana:
  image: gitlab-runner-new.dev.akamas.io/akamas/elastic/kibana
  
airflow:
  images:
    airflow:
      repository: gitlab-runner-new.dev.akamas.io/akamas/airflow_service
      tag: 2.8.0
    pgbouncer:
      repository: gitlab-runner-new.dev.akamas.io/akamas/airflow_service
      tag: ~
    pgbouncerExporter:
      repository: gitlab-runner-new.dev.akamas.io/akamas/airflow_service
      tag: ~
  webserver:   
    extraInitContainers:
      - name: wait-logstash
        image: gitlab-runner-new.dev.akamas.io/akamas/utils:0.1.5
        command:
          - "sh"
          - "-c"
          - "until ./wait-for-it.sh -h logstash -p 9600 -t 120 -e _node/pipelines -j '.pipelines|length' -r 10 ; do echo Waiting for Logstash; sleep 10; done; echo Connected"
        resources:
          limits:
            cpu: 100m
            memory: 50Mi
          requests:
            cpu: 10m
            memory: 50Mi
  scheduler:
    podAnnotations:
      k8s.akamas.com/imageName: gitlab-runner-new.dev.akamas.io/akamas/airflow_service
    env:
      - name: CONTAINER_NAME
        value: airflow
      - name: SERVICE
        value: airflow
      - name: LOGTYPE
        value: airflow
      - name: IMAGE_NAME
        value: gitlab-runner-new.dev.akamas.io/akamas/airflow_service
      - name: AIRFLOW_CONN_HTTP_SYSTEM
        value: "http://:@system:8080"
      - name: AIRFLOW_CONN_HTTP_CAMPAIGN
        value: "http://:@campaign:8080"
      - name: AIRFLOW_CONN_HTTP_ORCHESTRATOR
        value: "http://:@orchestrator:8080"
      - name: KEYCLOAK_ENDPOINT
        value: "http://keycloak:8080"

    extraInitContainers:
      - name: wait-logstash
        image: gitlab-runner-new.dev.akamas.io/akamas/utils:0.1.5
        command:
          - "sh"
          - "-c"
          - "until ./wait-for-it.sh -h logstash -p 9600 -t 120 -e _node/pipelines -j '.pipelines|length' -r 10 ; do echo Waiting for Logstash; sleep 10; done; echo Connected"
        resources:
          limits:
            cpu: 100m
            memory: 50Mi
          requests:
            cpu: 10m
            memory: 50Mi

Replace in the file the following placeholders:

• CUSTOMER_NAME: customer name provided with the Akamas license

• ADMIN_PASSWORD: initial administrator password

• INSTANCE_HOSTNAME: the URL that will be used to expose the Akamas installation, for example https://akamas.k8s.example.com when using an Ingress, or http//:localhost:9000 when using port-forwarding. Refer to Accessing Akamas for the list of the supported access methods and a reference for any additional configuration required.
• REGISTRY_URL: the URL for the private registry used in the transfer process above

Configure the authentication

This section describes how to configure the authentication to your private registry. If your registry does not require any authentication, skip directly to the installation section.

To authenticate to your private registry, you must manually create the Secret required to pull the images. If the registry uses basic authentication, you can create the credentials in the namespace by running the following command:

kubectl create secret docker-registry registry-token \
  --namespace akamas \
  --docker-server=<REGISTRY_URL> \
  --docker-username=<USER> \
  --docker-password=<PASSWORD>

Otherwise, you can leverage any credential already configured on your machine by running the following command:

kubectl create secret docker-registry registry-token \
  --namespace akamas \
  --from-file=.dockerconfigjson=<PATH/TO/.docker/config.json>

Define Size

Akamas can be installed in three sizes Small, Medium, and Large as explained in the cluster prerequisite section. By default, the chart installs the Small size. If you want to install a specific size add the following snippet to your values.yaml file.

Medium

#Medium
airflow:
  config:
    core:
      parallelism: 102
  scheduler:
    resources:
      limits:
        cpu: 2500m         
        memory: 21000Mi    
      requests:
        cpu: 1000m         
        memory: 21000Mi   

Large

#Large
airflow:
  config:
    core:
      parallelism: 202
  scheduler:
    resources:
      limits:
        cpu: 2500m         
        memory: 28000Mi    
      requests:
        cpu: 1000m         
        memory: 28000Mi    
telemetry:
  parallelism: 50

Start the installation

If the host you are using to install akamas can reach helm.akamas.io you can follow the instructions in the online installation guide. Otherwise, follow the instructions below to download the chart content locally.

From a machine that can reach the endpoint, run the following command to download the chart:

helm pull --repo http://helm.akamas.io/charts --version '1.5.4' akamas

The command downloads the latest version chart version as an archive named akamas-<version>.tgz. The file can be transferred to the machine where the installation will be run. Replace akamas/akamas with the download package in the following commands.

If you wish to see and override the values that Helm will use to install Akamas, you may execute the following command.

helm show values akamas-<version>.tgz

Now, with the configuration file you just created (and the new variables you added to override the defaults), you can start the installation with the following command:

helm upgrade --install \
  --create-namespace --namespace akamas \
  -f akamas.yaml \
  akamas akamas-<version>.tgz

This command will create the Akamas resources within the specified namespace. You can define a different namespace by changing the argument --namespace <your-namespace>

An example output of a successful installation is the following:

Release "akamas" does not exist. Installing it now.
NAME: akamas
LAST DEPLOYED: Thu Sep 21 10:39:01 2023
NAMESPACE: akamas
STATUS: deployed
REVISION: 1
NOTES:
Akamas has been installed

NOTES:
Akamas has been installed

To get the initial password use the following command:

kubectl get secret akamas-admin-credentials -o go-template='{{ .data.password | base64decode }}'

Check the installation

To monitor the application startup, run the command kubectl get pods. After a few minutes, the expected output should be similar to the following:

NAME                           READY   STATUS    RESTARTS   AGE
airflow-6ffbbf46d8-dqf8m       3/3     Running   0          5m
analyzer-67cf968b48-jhxvd      1/1     Running   0          5m
campaign-666c5db96-xvl2z       1/1     Running   0          5m
database-0                     1/1     Running   0          5m
elasticsearch-master-0         1/1     Running   0          5m
keycloak-66f748d54-7l6wb       1/1     Running   0          5m
kibana-6d86b8cbf5-6nz9v        1/1     Running   0          5m
kong-7d6fdd97cf-c2xc9          1/1     Running   0          5m
license-54ff5cc5d8-tr64l       1/1     Running   0          5m
log-5974b5c86b-4q7lj           1/1     Running   0          5m
logstash-8697dd69f8-9bkts      1/1     Running   0          5m
metrics-577fb6bf8d-j7cl2       1/1     Running   0          5m
optimizer-5b7576c6bb-96w8n     1/1     Running   0          5m
orchestrator-95c57fd45-lh4m6   1/1     Running   0          5m
store-5489dd65f4-lsk62         1/1     Running   0          5m
system-5877d4c89b-h8s6v        1/1     Running   0          5m
telemetry-8cf448bf4-x68tr      1/1     Running   0          5m
ui-7f7f4c4f44-55lv5            1/1     Running   0          5m
users-966f8f78-wv4zj           1/1     Running   0          5m

At this point, you should be able to access the Akamas UI using the endpoint specified in the akamasBaseUrl, and interact through the Akamas CLI with the path /api.

If you haven't already, you can update your configuration file to use a different type of service to expose Akamas' endpoints. To do so, pick from the Accessing Akamas the configuration snippet for the service type of your choice, add it to the akamas.yaml file, update the akamasBaseUrl value, and re-run the installation command to update your Helm release.

Installing telemetry providers

During online installation, a set of out-of-the-box telemetry providers are automatically installed. For offline installation, this step has to be executed manually. To install the telemetry providers required for your environment proceed to Integrating Telemetry Providers section.