> For the complete documentation index, see [llms.txt](https://docs.akamas.io/akamas-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.akamas.io/akamas-docs/managing-akamas/upgrade/migration-procedures.md).

# Migration Procedures

### From 3.6 to 4.0

The upgrade from Akamas 3.6 to 4.0 requires:

* upgrading the database to a new major version
* upgrading the CSV telemetry provider configuration

Those upgrades both involve some manual steps.

#### Database upgrade

Here's the guide for the database upgrade

<details>

<summary>DB upgrade - Docker version</summary>

{% hint style="info" %}
Make sure you are logged in to the host running the Akamas instance before running the following commands.
{% endhint %}

**Stop the services**

Stop the Akamas services except the databases:

```bash
cd akamas
docker compose down
docker compose up -d database airflow-db kong-database
```

**Export AWS credentials and login to AWS docker repo**

```bash
export AWS_ACCESS_KEY_ID=AAAAAAAAAAA ## use your AWS access key ID
export AWS_SECRET_ACCESS_KEY=bbbbbbbbbbbbbbbbbb ## use your AWS secret access key id
aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 485790562880.dkr.ecr.us-east-2.amazonaws.com

```

**Extract the database passwords**

```bash
docker compose config | python3 -c 'import sys, yaml; s=yaml.safe_load(sys.stdin)["services"]; [print(k+"_DB_PASSWORD="+s[v]["environment"]["POSTGRES_PASSWORD"]) or print(k+"_DB_USER="+s[v]["environment"].get("POSTGRES_USER", "postgres")) for k, v in {"AKAMAS": "database", "AIRFLOW": "airflow-db"}.items()]' > pass.env
```

**Dump the database**

```bash
export PG_IMAGE='485790562880.dkr.ecr.us-east-2.amazonaws.com/akamas/master-db:1.13.0'

mkdir -p backup
source pass.env

echo " *** Dumping akamas services"
docker run --rm \
    --network akamas \
    -u "$(id -u):$(id -g)" \
    -v $(pwd)/backup:/backup \
    -e PGHOST=database \
    -e PGUSER=${AKAMAS_DB_USER} \
    -e PGPASSWORD=${AKAMAS_DB_PASSWORD} \
    "$PG_IMAGE" \
    pg_dumpall --clean --if-exists --exclude-database=postgres -f /backup/akamas_dump.sql

echo " *** Dumping airflow"
docker run --rm \
    --network akamas \
    -u "$(id -u):$(id -g)" \
    -v $(pwd)/backup:/backup \
    -e PGHOST=airflow-db \
    -e PGUSER=${AIRFLOW_DB_USER} \
    -e PGPASSWORD=${AIRFLOW_DB_PASSWORD} \
    "$PG_IMAGE" \
    pg_dump --clean --if-exists -d airflow -f /backup/airflow_dump.sql
```

**Clean up old database**

```bash
echo " *** Removing old containers"
docker rm -fv database airflow-db kong-db
echo " *** Removing old volumes"
docker volume rm -f akamas_airflow-db-data akamas_database-data akamas_kong-data
```

**Start the updated database**

Update `docker-compose.yml` to use the new database image. Back up the old file and replace the image version using `sed`:

```bash
OLD_VERSION=$(python3 -c 'import sys, yaml; print(yaml.safe_load(sys.stdin)["services"]["database"]["image"].split(":")[1])' < docker-compose.yml)
echo "Current database image version: $OLD_VERSION"
cp docker-compose.yml docker-compose.yml.bak$(date +%s)
sed -i "s|master-db:${OLD_VERSION}|master-db:${PG_IMAGE##*:}|g" docker-compose.yml
diff -u docker-compose.yml.bak* docker-compose.yml
```

Verify the update and start the new database:

```bash
docker compose pull
docker compose up -d database
docker logs -f database
```

Then wait for the message `database system is ready to accept connections` to appear then press CTRL+C.\
NOTE: if the expected message doesn't pop up after a couple of minutes and you see the message `database system was shut down`, instead, some random bug occurred. In this case press CTRL+C then stop the database with `docker compose down`. Then relaunch the last 2 commands above (`docker compose up -d database` and `docker logs -f database` and wait again for the message `database system is ready to accept connections` to appear then press CTRL+C.

**Restore the data**

```bash
docker run --rm \
    --network akamas \
    -v $(pwd)/backup:/backup \
    -e PGHOST=database \
    -e PGUSER=${AKAMAS_DB_USER} \
    -e PGPASSWORD=${AKAMAS_DB_PASSWORD} \
    "$PG_IMAGE" \
    psql -X -f /backup/akamas_dump.sql

docker run --rm \
    --network akamas \
    -v $(pwd)/backup:/backup \
    -e PGHOST=database \
    -e PGUSER=${AKAMAS_DB_USER} \
    -e PGPASSWORD=${AKAMAS_DB_PASSWORD} \
    "$PG_IMAGE" \
    psql -X -f /backup/airflow_dump.sql
```

**Restart Akamas**

Replace `docker-compose.yml` with the latest Akamas 4.0 version, as described in Install the Akamas Server, and restart the remaining services:

```bash
docker compose pull
docker compose up -d
```

</details>

<details>

<summary>DB upgrade - Kubernetes version</summary>

{% hint style="info" %}
Make sure you are using the correct namespace for the Akamas installation. To switch namespace, run `kubectl config set-context --current --namespace <akamas>`, replacing `<akamas>` with your namespace name.
{% endhint %}

**Check `preStop` hooks**

Ensure the current chart already supports `preStop` hooks for Postgres. Run:

```bash
kubectl get statefulset/database -o jsonpath='{.spec.template.spec.containers[].lifecycle}'
```

If the output contains `preStop`, like in the example below, your chart already supports graceful shutdown and no patching is needed.

```json
{"preStop":{"exec":{"command":["/bin/sh","-c","PGUSER=postgres pg_ctl stop -m fast"]}}}
```

If the output is empty or missing the `preStop` entry, patch the statefulset with:

```bash
kubectl patch statefulset database -p '{"spec":{"template":{"spec":{"terminationGracePeriodSeconds":60,"containers":[{"name":"postgresql","lifecycle":{"preStop":{"exec":{"command":["/bin/sh","-c","PGUSER=postgres pg_ctl stop -m fast"]}}}}]}}}}'
kubectl wait statefulset/database --for=jsonpath='{.status.availableReplicas}=1'
```

**Stop the services**

Stop the Akamas services:

```bash
kubectl scale deployment --all --replicas 0
kubectl scale statefulsets -l 'app.kubernetes.io/name notin (postgresql)' --replicas 0
```

Charts `1.6.4` and later stop all pods except the database automatically. For older charts, scale the database back up manually:

```bash
kubectl scale statefulset database --replicas 1
kubectl wait statefulset/database --for=jsonpath='{.status.availableReplicas}=1'
```

**Dump the database**

Dump the database into a dedicated volume using the job defined in the attached `pg16_dump.yaml` file. It creates a 10Gi PersistentVolumeClaim to store the backup. If your database needs larger storage, update the PVC definition in the YAML file.

```bash
kubectl apply -f pg16_dump.yaml
kubectl wait job/pg-dump --for=jsonpath='{.status.ready}=1' -o template='{{"initContainer complete\n"}}'
kubectl logs job/pg-dump -f --all-containers=true
```

Once the dump completes, scale down the database:

```bash
kubectl scale statefulset database --replicas 0
```

**Clean up the datadir**

Verify the database is scaled down before proceeding:

```bash
kubectl wait statefulset/database --for=jsonpath='{.status.replicas}=0' --timeout=5s && echo Ok || echo 'ERROR: database is still running'
```

Then clean up the datadir using the attached `pg16_cleanup.yaml` file:

```bash
kubectl apply -f pg16_cleanup.yaml
kubectl wait job/pg-cleanup --for=jsonpath='{.status.ready}=1' -o template='{{"initContainer complete\n"}}'
kubectl logs job/pg-cleanup -f --all-containers=true
```

{% hint style="info" %}
You can inspect the content of the backup volume using the `pg-debug` pod

```bash
kubectl apply -f pg16_debug.yaml
```

{% endhint %}

**Upgrade the database**

Once the cleanup completes, patch the statefulset with the new image:

```bash
export PG16_IMAGE='16.6.0-debian-12-r2'
kubectl patch statefulset database -p '{"spec":{"template":{"spec":{"containers":[{"name":"postgresql","image":"485790562880.dkr.ecr.us-east-2.amazonaws.com/akamas/bitnami/postgresql:'${PG16_IMAGE}'"}]}}}}'
kubectl scale statefulset/database --replicas=1
kubectl wait statefulset/database --for=jsonpath='{.status.availableReplicas}=1'
kubectl logs statefulset/database --all-containers=true
```

**Restore the database**

Restore the database using the attached `pg16_restore.yaml` file:

```bash
kubectl apply -f pg16_restore.yaml
kubectl wait job/pg-restore --for=jsonpath='{.status.ready}=1' -o template='{{"initContainer complete\n"}}'
kubectl logs job/pg-restore -f --all-containers=true
```

**Restart the Akamas services**

To complete the upgrade, restart the Akamas services:

```bash
kubectl scale deployment --all --replicas 1
kubectl scale statefulsets --all --replicas 1
```

**Upgrade the Akamas release**

Once verified that the new database is working correctly, upgrade Akamas using the chart associated with the latest 4.0 release, as described in Install Akamas:

```bash
helm upgrade --install \
    --create-namespace --namespace akamas \
    --repo http://helm.akamas.io/charts \
    --version '<1.8.0>' \
    -f akamas.yaml \
    akamas akamas
```

**Final cleanup**

Once you have verified that the instance was upgraded successfully and works correctly, you can delete the backup volume.

```bash
kubectl delete pvc pg-dump
```

</details>

#### CSV telemetry provider configuration upgrade

This step upgrades the configuration (adding new config parameters hostname, username and password) of the CSV telemetry provider. If you never used this provider in your studies (or systems) and have no CSV telemetry instance defined in any system we recommend the faster way, that is:

<details>

<summary>Recommended procedure when no CSV telemetry instances present</summary>

An important prerequisite for this procedure is to startup services beforehand (refer to final part of #from-3.6-to-3.7), since you need telemetry servicew up and running.

Login with Akamas CLI to your environment with:

```
 akamas login
```

Then issue the following command:

```
akamas uninstall tp "CSV File"
```

**VERY IMPORTANT:** if a message appears telling `It will also delete the following telemetry-instances:`

like in the following example:

```
$> akamas uninstall tp "CSV File" 
This command will delete telemetry provider 'CSV File'.
It will also delete the following telemetry-instances:
                 id                    workspace           system                     name          
====================================================================================================
0f13e9b9-c110-46a8-9bb1-c53fe2e6c199   default     renaissance               CSV File telem instance
e86a3e01-11a9-4a35-bcf1-ead3e492afeb   work1       customerpay-bff-service   CSV File               
b22a72e0-7013-4ac0-84a3-7cc337ba0d73   work2       Mongo                     CSV File 2           
2c257de0-faca-445b-b7ac-81044c239efb   work3       renaissance               CSV File telem instance

Do you want to proceed? [y/N]:
```

just STOP this procedure by pressing ENTER key and refer to alternate docker or kubernetes procedures below.\
If there are no telemetry instances to delete, you may proceed with the deletion of the telemetry provider (press 'y' then ENTER), then you can recreate the new telemetry provider by creating a new file named `csv-file-provider-official.yaml`

with these exact contents:

```
name: CSV File
description: Telemetry provider to collect metrics from CSV files
dockerImage: 485790562880.dkr.ecr.us-east-2.amazonaws.com/akamas/telemetry-providers/csv-file-provider:3.3.0
```

then issuing the following command:

```
akamas install tp csv-file-provider-official.yaml
```

</details>

These, instead, are the alternate procedure for docker and kubernetes case, for use when you already have CSV telemetry instances defined in your systems. They have the advantage that you don't need Akamas services to be up and running (you only need database, already running)

<details>

<summary>Alternate procedure for Docker case</summary>

Login with ssh to the machine where akamas is installed. Then issue command:

```bash
cd akamas
```

Retrieve telemetry DB password with this command:

```bash
docker compose exec telemetry bash -c "echo \$SPRING_DATASOURCE_PASSWORD  "
```

Connect to database container shell with:

```bash
docker compose exec -it database bash
```

Connect to postgres with:

```bash
psql -U akamas_telemetry -p -d telemetry_service -p 5432
```

then type in DB password (retrieved above) when asked.

Then issue SQL command:

```sql
update provider set config = '[{"required": false, "paramName": "logLevel", "paramType": "STRING_ENUM", "protected": false, "validationParams": {"validStrings": ["INFO", "DETAILED"]}}, {"required": true, "paramName": "remoteFilePattern", "paramType": "STRING", "protected": false}, {"required": true, "paramName": "hostname", "paramType": "IP_OR_FQDN", "protected": false, "requiredAlternates": ["address"]}, {"required": false, "paramName": "port", "paramType": "INTEGER", "protected": false, "validationParams": {"max": 65563, "min": 1}}, {"required": true, "paramName": "username", "paramType": "STRING", "protected": false}, {"required": true, "paramName": "password", "paramType": "STRING", "protected": true, "requiredAlternates": ["auth", "key"]}, {"required": true, "paramName": "key", "paramType": "STRING", "protected": true, "requiredAlternates": ["password", "auth"]}, {"required": true, "paramName": "authType", "paramType": "STRING_ENUM", "protected": false, "validationParams": {"validStrings": ["password", "key"]}, "requiredAlternates": ["password", "key"]}, {"required": false, "paramName": "protocol", "paramType": "STRING_ENUM", "protected": false, "validationParams": {"validStrings": ["scp", "sftp"]}}, {"required": false, "paramName": "componentColumn", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "componentName", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "timestampColumn", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "timestampFormat", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "filenamePattern", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "fieldSeparator", "paramType": "STRING", "protected": false, "validationParams": {"regex": "[,;\t]"}}, {"required": false, "paramName": "labelColumns", "paramType": "STRING_LIST", "protected": false}]' where name = 'CSV File';
```

</details>

<details>

<summary>Alternate procedure for Kubernetes case</summary>

Properly configure your kubectl configuration in order to connect to the correct cluster and namespace holding your Akamas installation.

Retrieve telemetry DB password with:

```bash
kubectl get secret database-user-credentials -o go-template='{{ .data.telemetry | base64decode }} '
```

Connect to database pod shell with:

```bash
kubectl exec -it database-0 -- bash
```

Connect to postgres with:

```bash
psql -U akamas_telemetry -p -d telemetry_service -p 5432
```

then type in DB password (retrieved above) when asked.

Then issue SQL command:

```sql
update provider set config = '[{"required": false, "paramName": "logLevel", "paramType": "STRING_ENUM", "protected": false, "validationParams": {"validStrings": ["INFO", "DETAILED"]}}, {"required": true, "paramName": "remoteFilePattern", "paramType": "STRING", "protected": false}, {"required": true, "paramName": "hostname", "paramType": "IP_OR_FQDN", "protected": false, "requiredAlternates": ["address"]}, {"required": false, "paramName": "port", "paramType": "INTEGER", "protected": false, "validationParams": {"max": 65563, "min": 1}}, {"required": true, "paramName": "username", "paramType": "STRING", "protected": false}, {"required": true, "paramName": "password", "paramType": "STRING", "protected": true, "requiredAlternates": ["auth", "key"]}, {"required": true, "paramName": "key", "paramType": "STRING", "protected": true, "requiredAlternates": ["password", "auth"]}, {"required": true, "paramName": "authType", "paramType": "STRING_ENUM", "protected": false, "validationParams": {"validStrings": ["password", "key"]}, "requiredAlternates": ["password", "key"]}, {"required": false, "paramName": "protocol", "paramType": "STRING_ENUM", "protected": false, "validationParams": {"validStrings": ["scp", "sftp"]}}, {"required": false, "paramName": "componentColumn", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "componentName", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "timestampColumn", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "timestampFormat", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "filenamePattern", "paramType": "STRING", "protected": false}, {"required": false, "paramName": "fieldSeparator", "paramType": "STRING", "protected": false, "validationParams": {"regex": "[,;\t]"}}, {"required": false, "paramName": "labelColumns", "paramType": "STRING_LIST", "protected": false}]' where name = 'CSV File';
```

</details>

After using one of the two alternate procedures, you MUST upgrade the CSV Telemetry Provider to latest version 3.3.0 (3.2.0 version that was shipped with 3.6.x will not work properly when using new features). For this, you need to start up services beforehand (refer to #from-3.6-to-3.7). Then you can upgrade the CSV telemetry provider by:\
Creating a new file named `csv-file-provider-official.yaml` with these contents:

```
name: CSV File
description: Telemetry provider to collect metrics from CSV files
dockerImage: 485790562880.dkr.ecr.us-east-2.amazonaws.com/akamas/telemetry-providers/csv-file-provider:3.3.0
```

and run:

```
akamas install tp -u csv-file-provider-official.yaml
```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.akamas.io/akamas-docs/managing-akamas/upgrade/migration-procedures.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.