Akamas Docs
3.4.0
Ask or search…
K

Setup Instana integration

Introduction

Akamas leverages the CSV telemetry provider to integrate a variety of data sources such as Instana.
All integrations based on this provider consist of two phases:
  1. 1.
    Metric extraction from Instana
  2. 2.
    Metric import via CSV provider
The first phase is composed of a set of scripts launched by a workflow task that interacts with the Instana API and saves the metrics of interest for the experiment in a CSV file with a proper format.
The second phase is executed by the CSV telemetry provider that imports the metrics from the CSV file.

Prerequisites

To set up the integration you need:
  • A host (or a container) that can be accessed via SSH from Akamas to run the extraction scripts and host the generated CSV file.
  • The host must have the following packages installed:
  • The host must be able to connect to Instana APIs
  • A token to authenticate to your Instana account and extract the metrics

Script setup

The script required to set up this integration is not currently publicly available. To obtain them please contact [email protected].
You can deploy the scripts once and then re-use them for multiple studies as all the required configurations can be provided as arguments which can be changed directly in the akamas workflow yaml or from the UI.
To deploy the scripts, extract the archive to a location of your choice on the host. You can verify that the script can be executed correctly by running the following command, substituting these placeholders:
  • <my-environment> with your environment id.
  • <my-token> with the token you generated from Instana. You can read more on this on the Instana Official documentation.
  • <my-service-id> with the id of one of your services on instana.
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f application_metrics -c my_service -tp service -id <my-service-id>
The script will extract the application metrics and save them to /tmp/instana/metrics.
These are the main parameters that can be used with the script along with their description.
  • endpoint: URL of the Instana environment (ex: https://moviri-moviri.instana.io)
  • token: environment token
  • window_size: defined in ms, is the size of the window for which the metrics are collected. The script collects metrics from now-window_size to now
  • rollup: Depending on the selected timeFrame it's possible to select the rollup. The limitation is that we only return 600 Data points per call, thus if you select a windowSize of 1 hour the most accurate rollup you can query for would be 5s. Valid rollups are:
Rollup
Value
1 second
1
5 seconds
5
1 minute
60
5 minutes
300
1 hour
3600
  • granularity: granularity of the application metrics (services and endpoints)
  • max_attempts: maximum number of attempts before considering the API call failed
  • timeshift: fixed time to add to metrics' timestamp
  • timezone: timezone to use in timestamp
  • output_dir: directory in which save the output files
  • timestamp_format: format in which the epoch timestamp is converted into the final CSV file
  • filename: Output file name
  • component: Akamas component name
  • type: the available types are infrastructure, service and endpoint
  • plugin: plugin type, the available plugins are: kubernetesPod, containerd, process and jvmRuntimePlatform
  • query: query to select the correct entity. Used for infrastructure entities
  • id: entity ID of the selected service or endpoint
Name
Argument
Required
Default
Endpoint
-e, --endpoint
True
-
Token
-t, --token
True
-
Output Directory
-o, --output_directory
True
-
Window Size
-w, --window_size
False
3600000
Rollup
-r, --rollup
False
60
Granularity
-g, --granularity
False
60
Filename
-f, --filename
True
-
Component
-c, --component
True
-
Type
-tp, --type
True
-
Plugin
-p, --plugin
False
-
Query
-q, --query
False
-
Id
-id, --id
False
-
Max Attempts
-ma, --max_attempts
False
5
Timezone
-tz, --timezone
False
UTC
Timeshift
-ts, --timeshift
False
0
Timestamp Format
-tf, --timestamp_format
False
%Y-%m-%d %H:%M:00
Start Timestamp
-st, --start_timestamp
False
-

Akamas setup

Once the scripts have been deployed you can use them across multiple studies.

Workflow

To generate the CSV with the required metric add the following task to the workflow of your study taking care of substituting the following variables. Please note that all these variables can also be updated via UI once the workflow has been created.
  • <my-host> with the hostname or IP of the instance hosting the scripts.
  • <my-user> with the username used to access the instance via SSH.
  • <my-key> with an SSH key to access the instance.
  • <my-environment> with your environment ID.
  • <my-token> with the token you generated from Instana. You can read more on this on the Instana Official documentation.
  • <my-application-component> with the name of the component of type Web Application in your system.
  • <my-jvm-component> with the name of the component of type open-jdk in your system.
  • <my-container-component> with the name of the component of type container in your system.
  • <my-instana-process> with the ID of the Instana process you want to extract.
  • <my-instana-jvm> with the id of the Instana jvm you want to extract.
  • <my-insana-pod-name> with the name of the pod on Instana you want to extract.
  • <my-instana-container-name> with the name of the container on Instana you want to extract.
  • <my-instana-endpoint-id> with the id of the endpoint on Instana you want to extract.
  • <my-instana-service-id> with the id of the service on Instana you want to extract.
Note that if your system does not include all these components you can just omit some commands as described in the yaml file.
Please note that the script will produce results in the /tmp/instana/metrics folder. If you wish to run more studies in parallel you might need to change this folder as well.
workflow.yaml
name: Get Metrics
operator: Executor
arguments:
retries: 2
retry_delay: 10s
command:
# Endpoint - Omit if you system does not contain any Web Application component
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f endpoint_metrics -c <my-application-component> -tp endpoint -id <my-instana-endpoint-id>;
# Service - Omit if you system does not contain any Web Application component
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f service_metrics service_metrics -c <my-application-component> -tp service -id <my-instana-service-id>;
# JVM - Omit if you system does not contain any openjdk comopnent
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f jvm_metrics -c <my-jvm-component> -tp infrastructure -p process --query entity.process.args:*<my-instana-process>*;
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f jvm_metrics -c <my-jvm-component> -tp infrastructure -p jvmRuntimePlatform --query entity.jvm.app.name:*<my-instana-jvm>*;
# Cointainer - Omit if you system does not contain any Kubernetes Container component
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f pod_metrics -c <my-container-component> -tp infrastructure -p kubernetesPod --query entity.kubernetes.pod.name:<my-insana-pod-name>*;
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f pod_metrics -c <my-container-component> -tp infrastructure -p containerd --query entity.kubernetes.container.name:<my-instana-container-name>* AND entity.kubernetes.pod.name:<my-instana-pod-name>*
host:
hostname: <my-host>
username: <my-username>
key: <my-key>.key

Telemetry Instances

To set up the CSV telemetry provider create a new telemetry instance for each of your system components.
Here you can find the configuration for each supported component type.
Take care of substituting the following variables.
  • <my-host> with the hostname or IP of the instance hosting the scripts.
  • <my-user> with the username used to access the instance via ssh.
  • <my-key> with an ssh key to access the instance.

Container

ContainerTelemetryInstanceCSV.yaml
provider: CSV File
config:
protocol: scp
address: <my-host>
username: <mu-user>
authType: key
auth: <my-key>.key
remoteFilePattern: /tmp/instana/metrics/pod_metrics_*.csv
componentColumn: Component
timestampColumn: TS
timestampFormat: yyyy-MM-dd HH:mm:ss zzz
metrics:
- metric: container_cpu_request
datasourceMetric: container_cpu_request
- metric: container_cpu_limit
datasourceMetric: container_cpu_limit
- metric: container_memory_request
datasourceMetric: container_memory_request
- metric: container_memory_limit
datasourceMetric: container_memory_limit
- metric: container_cpu_util
datasourceMetric: container_cpu_util
- metric: container_memory_util
datasourceMetric: container_memory_util
- metric: container_memory_working_set
datasourceMetric: container_memory_working_set
- metric: container_cpu_throttle_time
datasourceMetric: container_cpu_throttle_time

JVM

JVMTelemetryInstanceCSV.yaml
provider: CSV File
config:
protocol: scp
address: <my-host>
username: <mu-user>
authType: key
auth: <my-key>.key
remoteFilePattern: /tmp/instana/metrics/jvm_metrics_*.csv
componentColumn: Component
timestampColumn: TS
timestampFormat: yyyy-MM-dd HH:mm:ss zzz
metrics:
- metric: jvm_threads_deadlocked
datasourceMetric: jvm_threads_deadlocked
- metric: jvm_heap_size
datasourceMetric: jvm_heap_size
- metric: jvm_memory_used
datasourceMetric: jvm_memory_used
- metric: jvm_gc_duration
datasourceMetric: jvm_gc_duration

Web Application

WebAppTelemetryInstanceCSV.yaml
provider: CSV File
config:
protocol: scp
address: <my-host>
username: <mu-user>
authType: key
auth: <my-key>.key
remoteFilePattern: /tmp/instana/metrics/service_metrics_*.csv
componentColumn: Component
timestampColumn: TS
timestampFormat: yyyy-MM-dd HH:mm:ss zzz
metrics:
- metric: transactions_throughput
datasourceMetric: transactions_throughput
- metric: transactions_error_throughput
datasourceMetric: transactions_error_throughput
- metric: transactions_response_time
datasourceMetric: transactions_response_time
- metric: transactions_response_time_p90
datasourceMetric: transactions_response_time_p90
- metric: transactions_response_time_p99
datasourceMetric: transactions_response_time_p99
WebAppTelemetryInstanceCSV.yaml
provider: CSV File
config:
protocol: scp
address: <my-host>
username: <mu-user>
authType: key
auth: <my-key>.key
remoteFilePattern: /tmp/instana/metrics/endpoint_metrics_*.csv
componentColumn: Component
timestampColumn: TS
timestampFormat: yyyy-MM-dd HH:mm:ss zzz
metrics:
- metric: requests_throughput
datasourceMetric: requests_throughput
- metric: requests_error_throughput
datasourceMetric: requests_error_throughput
- metric: requests_response_time
datasourceMetric: requests_response_time
- metric: requests_response_time_p90
datasourceMetric: requests_response_time_p90
- metric: requests_response_time_p99
datasourceMetric: requests_response_time_p99

Supported Metrics

Here you can find the list of supported metrics. Metrics from Instana are mapped to metrics from the Akamas optimization pack. As an example the memoryRequests metric on the kubernetesPod entity in Instana is mapped to the container_memory_request metric of component type Kubernetes Container.

Infrastructure and runtime

kubernetesPod
Instana Metric
Akamas Component Type
Akamas Metric
cpuRequests * 1000
Kubernetes Container
container_cpu_request
cpuLimits * 1000
Kubernetes Container
container_cpu_limit
memoryRequests
Kubernetes Container
container_memory_request
memoryLimits
Kubernetes Container
container_memory_limit
containerd
Instana Metric
Akamas Component Type
Akamas Metric
cpu.total_usage
Kubernetes Container
container_cpu_util
memory.usage
Kubernetes Container
container_memory_util
memory.total_rss
Kubernetes Container
container_memory_working_set
cpu.throttling_time
Kubernetes Container
container_cpu_throttle_time
jvmRuntimePlatform
Instana Metric
Akamas Component Type
Akamas Metric
threads.blocked
java-openjdk-XX
jvm_threads_deadlocked
jvm.heap.maxSize
java-openjdk-XX
jvm_heap_size
memory.used
java-openjdk-XX
jvm_memory_used
suspension.time
java-openjdk-XX
jvm_gc_duration

Service

Instana Metric
Akamas Component Type
Akamas Metric
calls
Web Application
transactions_throughput
erroneousCalls
Web Application
transactions_error_throughput
latency - MEAN
Web Application
transactions_response_time
latency - P90
Web Application
transactions_response_time_p90
latency - P99
Web Application
transactions_response_time_p99

Endpoint

Instana Metric
Akamas Component Type
Akamas Metric
calls
Web Application
requests_throughput
erroneousCalls
Web Application
requests_error_throughput
latency - MEAN
Web Application
requests_response_time
latency - P90
Web Application
requests_response_time_p90
latency - P99
Web Application
requests_response_time_p99
# Setup Instana integration

Introduction

Akamas leverages the CSV telemetry provider to integrate a variety of data sources such as Instana.
All integrations based on this provider consist of two phases:
  1. 1.
    Metric extraction from Instana
  2. 2.
    Metric import via CSV provider
The first phase is composed of a set of scripts launched by a workflow task that interacts with the Instana API and saves the metrics of interest for the experiment in a CSV file with a proper format.
The second phase is executed by the CSV telemetry provider that imports the metrics from the CSV file.

Prerequisites

In order to set up the integration you need:
  • A host (or a container) that can be accessed via SSH from Akamas to run the extraction scripts and host the generated CSV file.
  • The host must have the following packages installed:
  • The host must be able to connect to Instana APIs
  • A token to authenticate to your Instana account and extract the metrics

Script setup

The script required to set up this integration is not currently publicly available. To obtain them please contact [email protected].
You can deploy the scripts once and then re-use them for multiple studies as all the required configurations can be provided as arguments which can be changed directly in the akamas workflow yaml or from the UI.
To deploy the scripts, extract the archive to a location of your choice on the host. You can verify that the script can be executed correctly by running the following command, substituting these placeholders:
  • <my-environment> with your environment ID.
  • <my-token> with the token you generated from Instana. You can read more on this on the Instana Official documentation.
  • <my-service-id> with the ID of one of your services on instana.
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f application_metrics -c my_service-tp service -id <my-service-id>
The script will extract the application metrics and save them to /tmp/instana/metrics.
These are the main parameters that can be used with the script along with their description.
  • endpoint: URL of the Instana environment (ex: https://moviri-moviri.instana.io)
  • token: environment token
  • window_size: defined in ms, is the size of the window for which the metrics are collected. The script collects metrics from now-window_size to now
  • rollup: Depending on the selected timeFrame it's possible to select the rollup. The limitation is that we only return 600 Data points per call, thus if you select a windowSize of 1 hour the most accurate rollup you can query for would be 5s. Valid rollups are:
Rollup
Value
1 second
1
5 seconds
5
1 minute
60
5 minutes
300
1 hour
3600
  • granularity: granularity of the application metrics (services and endpoints)
  • max_attempts: maximum number of attempts before considering the API call failed
  • timeshift: fixed time to add to metrics' timestamp
  • timezone: timezone to use in timestamp
  • output_dir: directory in which save the output files
  • timestamp_format: format in which the epoch timestamp is converted into the final CSV file
  • filename: Output file name
  • component: Akamas component name
  • type: the available types are infrastructure, service and endpoint
  • plugin: plugin type, the available plugins are: kubernetesPod, containerd, process and jvmRuntimePlatform
  • query: query to select the correct entity. Used for infrastructure entities
  • id: entity ID of the selected service or endpoint
Name
Argument
Required
Default
Endpoint
-e, --endpoint
True
-
Token
-t, --token
True
-
Output Directory
-o, --output_directory
True
-
Window Size
-w, --window_size
False
3600000
Rollup
-r, --rollup
False
60
Granularity
-g, --granularity
False
60
Filename
-f, --filename
True
-
Component
-c, --component
True
-
Type
-tp, --type
True
-
Plugin
-p, --plugin
False
-
Query
-q, --query
False
-
Id
-id, --id
False
-
Max Attempts
-ma, --max_attempts
False
5
Timezone
-tz, --timezone
False
UTC
Timeshift
-ts, --timeshift
False
0
Timestamp Format
-tf, --timestamp_format
False
%Y-%m-%d %H:%M:00
Start Timestamp
-st, --start_timestamp
False
-

Akamas setup

Once the scripts have been deployed you can use them across multiple studies.

Workflow

To generate the CSV with the required metric add the following task to the workflow of your study taking care of substituting the following variables. Please note that all these variables can also be updated via UI once the workflow has been created.
  • <my-host> with the hostname or IP of the instance hosting the scripts.
  • <my-user> with the username used to access the instance via SSH.
  • <my-key> with an SSH key to access the instance.
  • <my-environment> with your environment ID.
  • <my-token> with the token you generated from Instana. You can read more on this on the Instana Official documentation.
  • <my-application-component> with the name of the component of type Web Application in your system.
  • <my-jvm-component> with the name of the component of type open-jdk in your system.
  • <my-container-component> with the name of the component of type container in your system.
  • <my-instana-process> with the ID of the Instana process you want to extract.
  • <my-instana-jvm> with the id of the Instana jvm you want to extract.
  • <my-insana-pod-name> with the name of the pod on Instana you want to extract.
  • <my-instana-container-name> with the name of the container on Instana you want to extract.
  • <my-instana-endpoint-id> with the id of the endpoint on Instana you want to extract.
  • <my-instana-service-id> with the id of the service on Instana you want to extract.
Note that if your system does not include all these components you can just omit some commands as described in the yaml file.
Please note that the script will produce results in the /tmp/instana/metrics folder. If you wish to run more studies in parallel you might need to change this folder as well.
workflow.yaml
name: Get Metrics
operator: Executor
arguments:
retries: 2
retry_delay: 10s
command:
# Endpoint - Omit if you system does not contain any Web Application component
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f endpoint_metrics -c <my-application-component> -tp endpoint -id <my-instana-endpoint-id>;
# Service - Omit if you system does not contain any Web Application component
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f service_metrics service_metrics -c <my-application-component> -tp service -id <my-instana-service-id>;
# JVM - Omit if you system does not contain any openjdk comopnent
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f jvm_metrics -c <my-jvm-component> -tp infrastructure -p process --query entity.process.args:*<my-instana-process>*;
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f jvm_metrics -c <my-jvm-component> -tp infrastructure -p jvmRuntimePlatform --query entity.jvm.app.name:*<my-instana-jvm>*;
# Cointainer - Omit if you system does not contain any Kubernetes Container component
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f pod_metrics -c <my-container-component> -tp infrastructure -p kubernetesPod --query entity.kubernetes.pod.name:<my-insana-pod-name>*;
python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f pod_metrics -c <my-container-component> -tp infrastructure -p containerd --query entity.kubernetes.container.name:<my-instana-container-name>* AND entity.kubernetes.pod.name:<my-instana-pod-name>*
host:
hostname: <my-host>
username: <my-username>
key: <my-key>.key

Telemetry Instances

To set up the CSV telemetry provider create a new telemetry instance for each of your system components.
Here you can find the configuration for each supported component type.
Take care of substituting the following variables.
  • <my-host> with the hostname or IP of the instance hosting the scripts.
  • <my-user> with the username used to access the instance via ssh.
  • <my-key> with an ssh key to access the instance.

Container

ContainerTelemetryInstanceCSV.yaml
provider: CSV File
config:
protocol: scp
address: <my-host>
username: <mu-user>
authType: key
auth: <my-key>.key
remoteFilePattern: /tmp/instana/metrics/pod_metrics_*.csv
componentColumn: Component
timestampColumn: TS
timestampFormat: yyyy-MM-dd HH:mm:ss zzz
metrics:
- metric: container_cpu_request
datasourceMetric: container_cpu_request
- metric: container_cpu_limit
datasourceMetric: container_cpu_limit
- metric: container_memory_request
datasourceMetric: container_memory_request
- metric: container_memory_limit
datasourceMetric: container_memory_limit
- metric: container_cpu_util
datasourceMetric: container_cpu_util
- metric: container_memory_util
datasourceMetric: container_memory_util
- metric: container_memory_working_set
datasourceMetric: container_memory_working_set
- metric: container_cpu_throttle_time
datasourceMetric: container_cpu_throttle_time

JVM

JVMTelemetryInstanceCSV.yaml
provider: CSV File
config:
protocol: scp
address: <my-host>
username: <mu-user>
authType: key
auth: <my-key>.key
remoteFilePattern: /tmp/instana/metrics/jvm_metrics_*.csv
componentColumn: Component
timestampColumn: TS
timestampFormat: yyyy-MM-dd HH:mm:ss zzz
metrics:
- metric: jvm_threads_deadlocked
datasourceMetric: jvm_threads_deadlocked
- metric: jvm_heap_size
datasourceMetric: jvm_heap_size
- metric: jvm_memory_used
datasourceMetric: jvm_memory_used
- metric: jvm_gc_duration
datasourceMetric: jvm_gc_duration

Web Application

WebAppTelemetryInstanceCSV.yaml
provider: CSV File
config:
protocol: scp
address: <my-host>
username: <mu-user>
authType: key
auth: <my-key>.key
remoteFilePattern: /tmp/instana/metrics/service_metrics_*.csv
componentColumn: Component
timestampColumn: TS
timestampFormat: yyyy-MM-dd HH:mm:ss zzz
metrics:
- metric: transactions_throughput
datasourceMetric: transactions_throughput
- metric: transactions_error_throughput
datasourceMetric: transactions_error_throughput
- metric: transactions_response_time
datasourceMetric: transactions_response_time
- metric: transactions_response_time_p90
datasourceMetric: transactions_response_time_p90
- metric: transactions_response_time_p99
datasourceMetric: transactions_response_time_p99
WebAppTelemetryInstanceCSV.yaml
provider: CSV File
config:
protocol: scp
address: <my-host>
username: <mu-user>
authType: key
auth: <my-key>.key
remoteFilePattern: /tmp/instana/metrics/endpoint_metrics_*.csv
componentColumn: Component
timestampColumn: TS
timestampFormat: yyyy-MM-dd HH:mm:ss zzz
metrics:
- metric: requests_throughput
datasourceMetric: requests_throughput
- metric: requests_error_throughput
datasourceMetric: requests_error_throughput
- metric: requests_response_time
datasourceMetric: