# Migration procedures

## From 3.6 to 3.7

The upgrade from Akamas 3.6 to 3.7 requires upgrading the database to a new major version. This upgrade involves some manual steps.

<details>

<summary>Docker version</summary>

{% hint style="info" %}
Make sure you are logged in to the host running the Akamas instance before running the following commands.
{% endhint %}

**Stop the services**

Stop the Akamas services except the databases:

```bash
cd akamas
docker compose down
docker compose up -d database airflow-db kong-database
```

**Export AWS credentials and login to AWS docker repo**

```bash
export AWS_ACCESS_KEY_ID=AAAAAAAAAAA ## use your AWS access key ID
export AWS_SECRET_ACCESS_KEY=bbbbbbbbbbbbbbbbbb ## use your AWS secret access key id
aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 485790562880.dkr.ecr.us-east-2.amazonaws.com

```

**Extract the database passwords**

```bash
docker compose config | python3 -c 'import sys, yaml; s=yaml.safe_load(sys.stdin)["services"]; [print(k+"_DB_PASSWORD="+s[v]["environment"]["POSTGRES_PASSWORD"]) or print(k+"_DB_USER="+s[v]["environment"].get("POSTGRES_USER", "postgres")) for k, v in {"AKAMAS": "database", "AIRFLOW": "airflow-db"}.items()]' > pass.env
```

**Dump the database**

```bash
export PG_IMAGE='485790562880.dkr.ecr.us-east-2.amazonaws.com/akamas/master-db:1.13.0'

mkdir -p backup
source pass.env

echo " *** Dumping akamas services"
docker run --rm \
    --network akamas \
    -u "$(id -u):$(id -g)" \
    -v $(pwd)/backup:/backup \
    -e PGHOST=database \
    -e PGUSER=${AKAMAS_DB_USER} \
    -e PGPASSWORD=${AKAMAS_DB_PASSWORD} \
    "$PG_IMAGE" \
    pg_dumpall --clean --if-exists --exclude-database=postgres -f /backup/akamas_dump.sql

echo " *** Dumping airflow"
docker run --rm \
    --network akamas \
    -u "$(id -u):$(id -g)" \
    -v $(pwd)/backup:/backup \
    -e PGHOST=airflow-db \
    -e PGUSER=${AIRFLOW_DB_USER} \
    -e PGPASSWORD=${AIRFLOW_DB_PASSWORD} \
    "$PG_IMAGE" \
    pg_dump --clean --if-exists -d airflow -f /backup/airflow_dump.sql
```

**Clean up old database**

```bash
echo " *** Removing old containers"
docker rm -fv database airflow-db kong-db
echo " *** Removing old volumes"
docker volume rm -f akamas_airflow-db-data akamas_database-data akamas_kong-data
```

**Start the updated database**

Update `docker-compose.yml` to use the new database image. Back up the old file and replace the image version using `sed`:

```bash
OLD_VERSION=$(python3 -c 'import sys, yaml; print(yaml.safe_load(sys.stdin)["services"]["database"]["image"].split(":")[1])' < docker-compose.yml)
echo "Current database image version: $OLD_VERSION"
cp docker-compose.yml docker-compose.yml.bak$(date +%s)
sed -i "s|master-db:${OLD_VERSION}|master-db:${PG_IMAGE##*:}|g" docker-compose.yml
diff -u docker-compose.yml.bak* docker-compose.yml
```

Verify the update and start the new database:

```bash
docker compose pull
docker compose up -d database
docker logs -f database
```

Then wait for the message `database system is ready to accept connections` to appear then press CTRL+C

**Restore the data**

```bash
docker run --rm \
    --network akamas \
    -v $(pwd)/backup:/backup \
    -e PGHOST=database \
    -e PGUSER=${AKAMAS_DB_USER} \
    -e PGPASSWORD=${AKAMAS_DB_PASSWORD} \
    "$PG_IMAGE" \
    psql -X -f /backup/akamas_dump.sql

docker run --rm \
    --network akamas \
    -v $(pwd)/backup:/backup \
    -e PGHOST=database \
    -e PGUSER=${AKAMAS_DB_USER} \
    -e PGPASSWORD=${AKAMAS_DB_PASSWORD} \
    "$PG_IMAGE" \
    psql -X -f /backup/airflow_dump.sql
```

**Restart Akamas**

Replace `docker-compose.yml` with the latest Akamas 3.7.x version, as described in [Install the Akamas Server](/akamas-docs/installing/docker/install-the-akamas-server.md), and restart the remaining services:

```bash
docker compose pull
docker compose up -d
```

</details>

<details>

<summary>Kubernetes version</summary>

{% hint style="info" %}
Make sure you are using the correct namespace for the Akamas installation. To switch namespace, run `kubectl config set-context --current --namespace <akamas>`, replacing `<akamas>` with your namespace name.
{% endhint %}

**Check `preStop` hooks**

Ensure the current chart already supports `preStop` hooks for Postgres. Run:

```bash
kubectl get statefulset/database -o jsonpath='{.spec.template.spec.containers[].lifecycle}'
```

If the output contains `preStop`, like in the example below, your chart already supports graceful shutdown and no patching is needed.

```json
{"preStop":{"exec":{"command":["/bin/sh","-c","PGUSER=postgres pg_ctl stop -m fast"]}}}
```

If the output is empty or missing the `preStop` entry, patch the statefulset with:

```bash
kubectl patch statefulset database -p '{"spec":{"template":{"spec":{"terminationGracePeriodSeconds":60,"containers":[{"name":"postgresql","lifecycle":{"preStop":{"exec":{"command":["/bin/sh","-c","PGUSER=postgres pg_ctl stop -m fast"]}}}}]}}}}'
kubectl wait statefulset/database --for=jsonpath='{.status.availableReplicas}=1'
```

**Stop the services**

Stop the Akamas services:

```bash
kubectl scale deployment --all --replicas 0
kubectl scale statefulsets -l 'app.kubernetes.io/name notin (postgresql)' --replicas 0
```

Charts `1.6.4` and later stop all pods except the database automatically. For older charts, scale the database back up manually:

```bash
kubectl scale statefulset database --replicas 1
kubectl wait statefulset/database --for=jsonpath='{.status.availableReplicas}=1'
```

**Dump the database**

Dump the database into a dedicated volume using the job defined in the attached `pg16_dump.yaml` file. It creates a 10Gi PersistentVolumeClaim to store the backup. If your database needs larger storage, update the PVC definition in the YAML file.

{% file src="/files/M0UYk14n7BBfYz5lB9PE" %}

```bash
kubectl apply -f pg16_dump.yaml
kubectl wait job/pg-dump --for=jsonpath='{.status.ready}=1' -o template='{{"initContainer complete\n"}}'
kubectl logs job/pg-dump -f --all-containers=true
```

Once the dump completes, scale down the database:

```bash
kubectl scale statefulset database --replicas 0
```

**Clean up the datadir**

Verify the database is scaled down before proceeding:

```bash
kubectl wait statefulset/database --for=jsonpath='{.status.replicas}=0' --timeout=5s && echo Ok || echo 'ERROR: database is still running'
```

Then clean up the datadir using the attached `pg16_cleanup.yaml` file:

{% file src="/files/ttqmK8SfAhXWiOm58Td6" %}

```bash
kubectl apply -f pg16_cleanup.yaml
kubectl wait job/pg-cleanup --for=jsonpath='{.status.ready}=1' -o template='{{"initContainer complete\n"}}'
kubectl logs job/pg-cleanup -f --all-containers=true
```

{% hint style="info" %}
You can inspect the content of the backup volume using the `pg-debug` pod

```bash
kubectl apply -f pg16_debug.yaml
```

{% endhint %}

{% file src="/files/1iEUbc7jBOFRLwz07G1j" %}

**Upgrade the database**

Once the cleanup completes, patch the statefulset with the new image:

```bash
export PG16_IMAGE='16.6.0-debian-12-r2'
kubectl patch statefulset database -p '{"spec":{"template":{"spec":{"containers":[{"name":"postgresql","image":"485790562880.dkr.ecr.us-east-2.amazonaws.com/akamas/bitnami/postgresql:'${PG16_IMAGE}'"}]}}}}'
kubectl scale statefulset/database --replicas=1
kubectl wait statefulset/database --for=jsonpath='{.status.availableReplicas}=1'
kubectl logs statefulset/database --all-containers=true
```

**Restore the database**

Restore the database using the attached `pg16_restore.yaml` file:

{% file src="/files/bISUYZqyqAAuES0bZzRo" %}

```bash
kubectl apply -f pg16_restore.yaml
kubectl wait job/pg-restore --for=jsonpath='{.status.ready}=1' -o template='{{"initContainer complete\n"}}'
kubectl logs job/pg-restore -f --all-containers=true
```

**Restart the Akamas services**

To complete the upgrade, restart the Akamas services:

```bash
kubectl scale deployment --all --replicas 1
kubectl scale statefulsets --all --replicas 1
```

**Upgrade the Akamas release**

Once verified that the new database is working correctly, upgrade Akamas using the chart associated with the latest 3.7.x release, as described in [Install Akamas](/akamas-docs/installing/kubernetes/install-akamas.md):

```bash
helm upgrade --install \
    --create-namespace --namespace akamas \
    --repo http://helm.akamas.io/charts \
    --version '<1.7.x>' \
    -f akamas.yaml \
    akamas akamas
```

**Final cleanup**

Once you have verified that the instance was upgraded successfully and works correctly, you can delete the backup volume.

```bash
kubectl delete pvc pg-dump
```

</details>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.akamas.io/akamas-docs/managing-akamas/upgrade/migration-procedures.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.