Setup Instana Integration

Introduction

Akamas leverages the CSV telemetry provider to integrate a variety of data sources such as Instana.

All integrations based on this provider consist of two phases:

  1. Metric extraction from Instana

  2. Metric import via CSV provider

The first phase is composed of a set of scripts launched by a workflow task that interacts with the Instana API and saves the metrics of interest for the experiment in a CSV file with a proper format.

The second phase is executed by the CSV telemetry provider that imports the metrics from the CSV file.

Prerequisites

In order to set up the integration you need:

  • A host (or a container) that can be accessed via SSH from Akamas to run the extraction scripts and host the generated CSV file.

  • The host must have the following packages installed:

  • The host must be able to connect to Instana APIs

  • A token to authenticate to your Instana account and extract the metrics

Script setup

The script required to set up this integration is not currently publicly available. To obtain them please contact support@akamas.io.

You can deploy the scripts once and then re-use them for multiple studies as all the required configurations can be provided as arguments which can be changed directly in the akamas workflow yaml or from the UI.

To deploy the scripts, extract the archive to a location of your choice on the host. You can verify that the script can be executed correctly by running the following command, substituting these placeholders:

  • <my-environment> with your environment id.

  • <my-token> with the token you generated from Instana. You can read more on this on the Instana Official documentation.

  • <my-service-id> with the id of one of your services on instana.

python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f application_metrics -c my_service-tp service -id <my-service-id>

The script will extract the application metrics and save them to /tmp/instana/metrics.

These are the main parameters that can be used with the script along with their description.

  • endpoint: URL of the Instana environment (ex: https://moviri-moviri.instana.io)

  • token: environment token

  • window_size: defined in ms, is the size of the window for which the metrics are collected. The script collects metrics from now-window_size to now

  • rollup: Depending on the selected timeFrame it's possible to select the rollup. The limitation is that we only return 600 Data points per call, thus if you select a windowSize of 1 hour the most accurate rollup you can query for would be 5s. Valid rollups are:

Rollup
Value

1 second

1

5 seconds

5

1 minute

60

5 minutes

300

1 hour

3600

  • granularity: granularity of the application metrics (services and endpoints)

  • max_attempts: maximum number of attempts before considering the API call failed

  • timeshift: fixed time to add to metrics' timestamp

  • timezone: timezone to use in timestamp

  • output_dir: directory in which save the output files

  • timestamp_format: format in which the epoch timestamp is converted into the final CSV file

  • filename: Output file name

  • component: Akamas component name

  • type: the available types are infrastructure, service and endpoint

  • plugin: plugin type, the available plugins are: kubernetesPod, containerd, process and jvmRuntimePlatform

  • query: query to select the correct entity. Used for infrastructure entities

  • id: entity id of the selected service or endpoint

Name
Argument
Required
Default

Endpoint

-e, --endpoint

True

-

Token

-t, --token

True

-

Output Directory

-o, --output_directory

True

-

Window Size

-w, --window_size

False

3600000

Rollup

-r, --rollup

False

60

Granularity

-g, --granularity

False

60

Filename

-f, --filename

True

-

Component

-c, --component

True

-

Type

-tp, --type

True

-

Plugin

-p, --plugin

False

-

Query

-q, --query

False

-

Id

-id, --id

False

-

Max Attempts

-ma, --max_attempts

False

5

Timezone

-tz, --timezone

False

UTC

Timeshift

-ts, --timeshift

False

0

Timestamp Format

-tf, --timestamp_format

False

%Y-%m-%d %H:%M:00

Start Timestamp

-st, --start_timestamp

False

-

Akamas setup

Once the scripts have been deployed you can use them across multiple studies.

Workflow

In order to generate the CSV with the required metric add the following task to the workflow of your study taking care of substituting the following variables. Please note that all these variables can also be updated via UI once the workflow has been created.

  • <my-host> with the hostname or IP of the instance hosting the scripts.

  • <my-user> with the username used to access the instance via ssh.

  • <my-key> with an ssh key to access the instance.

  • <my-environment> with your environment id.

  • <my-token> with the token you generated from Instana. You can read more on this on the Instana Official documentation.

  • <my-application-component> with the name of the component of type Web Application in your system.

  • <my-jvm-component> with the name of the component of type open-jdk in your system.

  • <my-container-component> with the name of the component of type container in your system.

  • <my-instana-process> with the id of the Instana process you want to extract.

  • <my-instana-jvm> with the id of the Instana jvm you want to extract.

  • <my-insana-pod-name> with the name of the pod on Instana you want to extract.

  • <my-instana-container-name> with the name of the container on Instana you want to extract.

  • <my-instana-endpoint-id> with the id of the endpoint on Instana you want to extract.

  • <my-instana-service-id> with the id of the service on Instana you want to extract.

Note that if your system does not include all these components you can just omit some commands as described in the yaml file.

Please note that the script will produce results in the /tmp/instana/metrics folder. If you wish to run more studies in parallel you might need to change this folder as well.

workflow.yaml
name: Get Metrics
operator: Executor
arguments:
  retries: 2
  retry_delay: 10s  
  command:
 # Endpoint - Omit if you system does not contain any Web Application component
  python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f endpoint_metrics -c <my-application-component> -tp endpoint -id <my-instana-endpoint-id>;
  # Service - Omit if you system does not contain any Web Application component
  python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f service_metrics service_metrics -c <my-application-component> -tp service -id <my-instana-service-id>;
  # JVM - Omit if you system does not contain any openjdk comopnent
  python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f jvm_metrics -c <my-jvm-component> -tp infrastructure -p process --query entity.process.args:*<my-instana-process>*;
  python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f jvm_metrics -c <my-jvm-component> -tp infrastructure -p jvmRuntimePlatform --query entity.jvm.app.name:*<my-instana-jvm>*;
  # Cointainer - Omit if you system does not contain any Kubernetes Container component
  python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f pod_metrics -c <my-container-component> -tp infrastructure -p kubernetesPod --query entity.kubernetes.pod.name:<my-insana-pod-name>*;
  python3 get_instana_metrics.py -e https://<my-environment>.instana.io -t <my-token> -o /tmp/instana/metrics -f pod_metrics -c <my-container-component> -tp infrastructure -p containerd --query entity.kubernetes.container.name:<my-instana-container-name>* AND entity.kubernetes.pod.name:<my-instana-pod-name>*
  host:
    hostname: <my-host>
    username: <my-username>
    key: <my-key>.key

Telemetry Instances

To set up the CSV telemetry provider create a new telemetry instance for each of your system components.

Here you can find the configuration for each supported component type.

Take care of substituting the following variables.

  • <my-host> with the hostname or IP of the instance hosting the scripts.

  • <my-user> with the username used to access the instance via ssh.

  • <my-key> with an ssh key to access the instance.

Container

ContainerTelemetryInstanceCSV.yaml
provider: CSV File
config:
  protocol: scp
  address: <my-host>
  username: <mu-user>
  authType: key
  auth: <my-key>.key
  remoteFilePattern: /tmp/instana/metrics/pod_metrics_*.csv
  componentColumn: Component
  timestampColumn: TS
  timestampFormat: yyyy-MM-dd HH:mm:ss zzz

metrics:    
  - metric: container_cpu_request
    datasourceMetric: container_cpu_request
  - metric: container_cpu_limit
    datasourceMetric: container_cpu_limit
  - metric: container_memory_request
    datasourceMetric: container_memory_request
  - metric: container_memory_limit
    datasourceMetric: container_memory_limit
  - metric: container_cpu_util
    datasourceMetric: container_cpu_util
  - metric: container_memory_util
    datasourceMetric: container_memory_util
  - metric: container_memory_working_set
    datasourceMetric: container_memory_working_set
  - metric: container_cpu_throttle_time
    datasourceMetric: container_cpu_throttle_time

JVM

JVMTelemetryInstanceCSV.yaml
provider: CSV File
config:
  protocol: scp
  address: <my-host>
  username: <mu-user>
  authType: key
  auth: <my-key>.key
  remoteFilePattern: /tmp/instana/metrics/jvm_metrics_*.csv
  componentColumn: Component
  timestampColumn: TS
  timestampFormat: yyyy-MM-dd HH:mm:ss zzz

metrics:    
  - metric: jvm_threads_deadlocked
    datasourceMetric: jvm_threads_deadlocked
  - metric: jvm_heap_size
    datasourceMetric: jvm_heap_size
  - metric: jvm_memory_used
    datasourceMetric: jvm_memory_used
  - metric: jvm_gc_duration
    datasourceMetric: jvm_gc_duration

Web Application

WebAppTelemetryInstanceCSV.yaml
provider: CSV File
config:
  protocol: scp
  address: <my-host>
  username: <mu-user>
  authType: key
  auth: <my-key>.key
  remoteFilePattern: /tmp/instana/metrics/service_metrics_*.csv
  componentColumn: Component
  timestampColumn: TS
  timestampFormat: yyyy-MM-dd HH:mm:ss zzz

metrics:    
  - metric: transactions_throughput
    datasourceMetric: transactions_throughput
  - metric: transactions_error_throughput
    datasourceMetric: transactions_error_throughput
  - metric: transactions_response_time
    datasourceMetric: transactions_response_time
  - metric: transactions_response_time_p90
    datasourceMetric: transactions_response_time_p90
  - metric: transactions_response_time_p99
    datasourceMetric: transactions_response_time_p99
WebAppTelemetryInstanceCSV.yaml
provider: CSV File
config:
  protocol: scp  
  address: <my-host>
  username: <mu-user>
  authType: key
  auth: <my-key>.key
  remoteFilePattern: /tmp/instana/metrics/endpoint_metrics_*.csv
  componentColumn: Component
  timestampColumn: TS
  timestampFormat: yyyy-MM-dd HH:mm:ss zzz

metrics:    
  - metric: requests_throughput
    datasourceMetric: requests_throughput
  - metric: requests_error_throughput
    datasourceMetric: requests_error_throughput
  - metric: requests_response_time
    datasourceMetric: requests_response_time
  - metric: requests_response_time_p90
    datasourceMetric: requests_response_time_p90
  - metric: requests_response_time_p99
    datasourceMetric: requests_response_time_p99

Supported Metrics

Here you can find the list of supported metrics. Metrics from instana are mapped to metrics from the Akamas optimization pack. As an example the memoryRequests metric on the kubernetesPod entity in Instana is mapped to the container_memory_request metric of component type Kubernetes Container.

Infrastructure and runtime

kubernetesPod

Instana Metric
Akamas Component Type
Akamas Metric

cpuRequests * 1000

Kubernetes Container

container_cpu_request

cpuLimits * 1000

Kubernetes Container

container_cpu_limit

memoryRequests

Kubernetes Container

container_memory_request

memoryLimits

Kubernetes Container

container_memory_limit

containerd

Instana Metric
Akamas Component Type
Akamas Metric

cpu.total_usage

Kubernetes Container

container_cpu_util

memory.usage

Kubernetes Container

container_memory_util

memory.total_rss

Kubernetes Container

container_memory_working_set

cpu.throttling_time

Kubernetes Container

container_cpu_throttle_time

jvmRuntimePlatform

Instana Metric
Akamas Component Type
Akamas Metric

threads.blocked

java-openjdk-XX

jvm_threads_deadlocked

jvm.heap.maxSize

java-openjdk-XX

jvm_heap_size

memory.used

java-openjdk-XX

jvm_memory_used

suspension.time

java-openjdk-XX

jvm_gc_duration

Service

Instana Metric
Akamas Component Type
Akamas Metric

calls

Web Application

transactions_throughput

erroneousCalls

Web Application

transactions_error_throughput

latency - MEAN

Web Application

transactions_response_time

latency - P90

Web Application

transactions_response_time_p90

latency - P99

Web Application

transactions_response_time_p99

Endpoint

Instana Metric
Akamas Component Type
Akamas Metric

calls

Web Application

requests_throughput

erroneousCalls

Web Application

requests_error_throughput

latency - MEAN

Web Application

requests_response_time

latency - P90

Web Application

requests_response_time_p90

latency - P99

Web Application

requests_response_time_p99

Last updated