The FileConfigurator operator allows configuring systems tuned by Akamas by interpolating configuration parameters into files on remote machines.

The operator performs the following operations:

1. It reads an input file from a remote machine containing templates for interpolating the configuration parameters generated by Akamas

2. It replaces the values of configuration parameters in the input file

3. It writes the file with replaced configuration parameters on a specified path on another remote machine

Access on remote machines is performed using SFTP (SSH).

Templates for configuration parameters

The FileConfigurator allows writing templates for configuration parameters in two ways:

• specify that a parameter should be interpolated directly:

${component_name.parameter_name}

• specify that all parameters of a component should be interpolated:

${component_name.*}

Suffix or prefix for interpolated parameters

It is possible to add a prefix or suffix to interpolated configuration parameters by acting at the component-type level:

Notice that any parameter that does not contain the FileConfigurator element in the operators' attribute is ignored and not written.

name: Component Type 1
description: My Component type
parameters:
- name: x1
  domain:
    type: real
    domain: [-5.0, 10.0]
  defaultValue: -5.0
  # Under this section, the operator to be used to configure the parameters is defined
  operators:
    FileConfigurator:
      # using this OPTIONAL confTemplate property is possible to interpolate the parameter value with a prefix and a suffix
      confTemplate: "PREFIX${value}SUFFIX"

In the example above, the parameter x1 will be interpolated with the prefix PREFIX and the suffix SUFFIX, ${value} will be replaced with the actual value of the parameter at each experiment.

Example

Let's assume we want to apply the following configuration:

component1.param1: 1024
component1.param2: Category1
component2.param3: 7
component2.param4: 35.4

where component1 is of type MyComponentType and MyComponentType is defined as follows:

name: MyComponentType
description: "MyComponentType
parameters:
- name: param1
  domain:
    type: real
    domain: [-5.0, 10.0]
  defaultValue: -5.0
  # Under this section, the operator to be used to configure the parameters is defined
  operators:
    FileConfigurator:
      # using this OPTIONAL confTemplate property is possible to interpolate the parameter value with a prefix and a suffix
      confTemplate: "X1:${value}MB"
...

A template file to interpolate only parameter component1.param1 and all parameters from component2 would look like this:

myexecutable.sh -PARAM ${component1.param1} -PARAMS ${component2.*}

The file after the configuration parameters are interpolated would look like this:

myexecutable.sh -PARAM X1:1024MB -PARAMS 7 35.4

Note that the file in this example contains a bash command whose arguments are constructed by interpolating configuration parameters. This represents a typical use case for the File Configurator: to construct the right bash commands that will configure a system with the new configuration parameters computed by Akamas.

Operator arguments

source and target structures and arguments

Here follows the structure of either the source or target operator argument

Get operator arguments from component

The component argument can be used to refer to a component by name and use its properties as the arguments of the operator. In case the mapped arguments are already provided to the operator, there is no override.

In this case, the operator replaces in the template file only tokens referring to the specified component. A parameter bound to any component will cause the substitution to fail.

Component property to operator argument mapping

Examples

Configure parameters for an Apache server with explicit source and target machine information

name: RemoteConfOperatorTestStandalone
operator: FileConfigurator
arguments:
  source:
    hostname: template-server
    username: akamas-user1
    password: akamas-password1
    path: /templates/frontend-httpd.conf
  target:
    hostname: frontend-server
    username: akamas-user2
    password: akamas-password22
    path: /etc/httpd/httpd.conf

Configure parameters for an Apache server with information taken from a Component

name: RemoteConfOperatorTestStandalone
operator: FileConfigurator
arguments:
  component: apache-server-1

where the apache-server-1 component is defined as:

name: apache-server-1
description: The Apache server instance
componentType: Apache Server 2.4

properties:
  hostname: apache.akamas.io
  username: ubuntu
  key: key.pem
  sourcePath: templates/httpd.conf.template
  targetPath: /etc/httpd/httpd.conf

This page introduces the OracleConfigurator operator, a workflow operator that allows configuring the optimized parameters of an Oracle instance.

Prerequisites

This section provides the minimum requirements that you should meet in order to use the OracleConfigurator operator.

Supported versions

• Oracle 12c or later

Network requirements

• The Oracle operator must be able to connect to the Oracle URL or IP address and port (default port: 1521).

Permissions

• The user used to log into the database must have ALTER SYSTEM privileges.

Supported component types

In order to configure the tuned parameters the Oracle Configurator operator requires to be bound to a component with one of the following types:

• Oracle Database 12c

• Oracle Database 18c

• Oracle Database 19c

Databases hosted on Amazon RDS are not supported.

Configuration options

When you define an OracleExecutor task in the workflow you should specify some configuration information to allow the operator to connect to the Oracle instance.

You can specify configuration information within the config part of the YAML of the instance definition. The operator can also inherit some specific arguments from the properties of a bound component when not specified in the task.

Operator arguments

The following table describes all the properties for the definition of a task using the OracleConfigurator operator.

Examples

Update database entries

In the following example, the workflow leverages the OracleConfigurator operator to update the database parameters before triggering the execution of the load test for a component oracledb:

This page introduces the OracleExecutor operator, a workflow operator that allows executing custom queries on an Oracle instance.

Prerequisites

This section provides the minimum requirements that you should meet in order to use the Oracle Executor operator.

Supported versions

• Oracle 12c or later

Network requirements

• The OracleExecutor operator must be able to connect to the Oracle URL or IP address and port (default port is 1521)

Permissions

• The user used to log into the database must have enough privilege to perform the required queries

Operator arguments

When you define a task that uses the Oracle Executor operator you should specify some configuration information to allow the operator to connect to the Oracle instance and execute queries.

The operator inherits the connection arguments from the properties of the component when referenced in the task definition. The Akamas user can also override the properties of the component or not reference it at all defining the connection fields directly in the configuration of the task.

The following table provides the list of all properties required to define a task that uses the OracleExecutor operator.

Notice: it is a good practice to define only queries that update the state of the database. Is not possible to use SELECT queries to extract data from the database.

Examples

Truncate tables after a load test

In the following example, the operator performs a cleanup action on a table of the database:

Update database entries

In the following example, the operator leverages its templating features to update a table:

The referenced oracledb component contains properties that specify how to connect to the Oracle database instance:

The WindowsExecutor operator executes a command on a target Windows machine using WinRM.

The command can be anything that runs on a Windows Command Prompt.

Operator arguments

host structure and arguments

Here follows the structure of the host argument

host:
  protocol: [https|http]
  hostname: this_is_a_hostname
  port: 5863
  path: /wsman
  username: this_is_a_username
  password: this_is_a_password
  validateCertificate: false

with its arguments:

Get operator arguments from component

The component argument can refer to a Component by name and use its properties as the arguments of the operator. In case the mapped arguments are already provided to the operator, there is no override.

Here is an example of a component that overrides the host and the command arguments:

name: LoadRunnerMachine
componentType: WebApplication
properties:
  command: "dir c:\"
  host:
    hostname: lr.mydomain.com
    username: MyLoadRunnerUser
    password: MyPassword

Examples

Execute a dir command with explicit host information

name: TestConnectivity
operator: WindowsExecutor
arguments:
  command: "dir c:\"
  host:
    hostname: frontend.akamas.io
    username: administrator
    password: MyPassword

Execute a dir command with host information taken from a Component

name: TestConnectivity
operator: WindowsExecutor
arguments:
  command: "dir c:\"
  component: frontend1

This section documents the out-of-the-box workflow operators.

All operators accept some common, optional, arguments that allow you to control how the operator is executed within your workflow.

The following table reports all the arguments that can be used with any operator.

The LinuxConfigurator operator allows configuring systems tuned by Akamas by applying parameters related to the Linux kernel using different strategies.

The operator can configure provided Components or can configure every Component which has parameters related to the Linux kernel.

The parameters are applied via SSH protocol.

Using

In the most basic use of the Operator, it is sufficient to add a task of type LinuxConfigurator in the workflow.

- name: LinuxConf
  operator: LinuxConfigurator
  arguments:
    component: ComponentName

The operator makes use of properties specified in the component to identify which instance should be configured, how to access it, and any other information required to apply the configuration.

Operator arguments

If no component is provided, this operator will try to configure every parameter defined for the Components of the System under test

Supported Component Properties

The following table highlights the properties that can be specified on components and are used by this operator.

Filter parameters and block/network devices

The properties blockDevices and networkDevices allow specifying which parameters to apply to each block/network-device associated with the Component, as well as which block/network-device should be left untouched by the LinuxConfigurator.

If the properties are omitted, then all block/network-devices associated with the Component will be configured will all the available related parameters.

All block-devices called loopN (where N is an integer number greater or equal to 0) are automatically excluded from the Component’s block-devices

The properties blockDevices and networkDevices are lists of objects with the following structure:

Examples

blockDevices:
- name: "xvd[a-z]"
  parameters:
    - os_StorageReadAhead
    - os_StorageQueueScheduler

In this example, only the parameters os_StorageReadAhead and os_StorageQeueuScheduler are applied to all the devices that match the regex "xvd[a-z]" (i.e. xvda, xvdb, …, xvdc).

blockDevices:
- name: "xvdb|loop0"
  parameters:
    - os_StorageMaxSectorsKb

In these examples, only the parameter os_StorageMaxSectorKb is applied to block device xvdb and loop0.

networkDevices:
  - name: wlp4s0
    parameters: []

In this example, no parameters are applied to the wlp4s0 network device, which is therefore excluded from the optimization.

How are parameters applied to the system?

To support the scenario in which some configuration parameters related to the Linux kernel may be applied using the strategies supported by this operator, while others with other strategies (e.g, using a file to be written on a remote machine), it is necessary to specify which parameters should be applied with the LinuxConfigurator, and this is done at the ComponentType level; moreover, still at the ComponentType level, it is necessary to specify which strategy should be used to configure each parameter. This information is already embedded in the Linux Optimization pack and, usually, no customization is required.

Sysctl strategy

With this strategy, a parameter is configured by leveraging the sysctl utility. The sysctl variable to map to the parameter that needs to be configured is specified using the key argument.

name: Component Type 1
description: My Component type
parameters:
  - name: net_forwarding
    domain:
      type: integer
      domain: [0, 1]
    defaultValue: 1
    operators:
      # the parameter is configured using LinuxConfigurator
      LinuxConfigurator:
        sysctl:
          key: net.ipv4.forwarding

Echo strategy

With this strategy, a parameter is configured by echoing and piping its value into a provided file. The path of the file is specified using the file argument.

name: Component Type 1
description: My Component type
parameters:
  - name: os_MemoryTransparentHugepageEnabled
    domain:
      type: categorical
      categories: [always, never]
    defaultValue: always
    operators:
      LinuxConfigurator:
        echo:
          file: /sys/kernel/mm/transparent_hugepage/enabled

Map strategy

With this strategy, each possible value of a parameter is mapped to a command to be executed on the machine the LinuxConfigurator operates on(this is especially useful for categorical parameters).

name: Component Type 1
description: My Component type
parameters:
  - name: os_MemorySwap
    domain:
      type: categorical
      categories: [swapon, swapoff]
    defaultValue: swapon
    operators:
      LinuxConfigurator:
        map:
          swapon: command1
          swapoff: command2

Command strategy

With this strategy, a parameter is configured by executing a command into which the parameter value is interpolated.

name: Component Type 1
description: My Component type
parameters:
  - name: os_MemorySwap
    domain:
      type: categorical
      categories: [swapon, swapoff]
    defaultValue: swapon
    operators:
      LinuxConfigurator:
        command:
          cmd: sudo ${value} -a

The Executor Operator can be used to execute a shell command on a target machine using SSH.

Operator arguments

Host structure and arguments

Here follows the structure of the host argument:

host:
  hostname: this_is_a_hostname
  username: this_is_a_username
  password: this_is_a_password
  sshPort: 22
  key: this_is_a_key

with its arguments:

Get operator arguments from component

The component argument can refer to a component by name and use its properties as the arguments of the operator (see mapping here below). In case the mapped arguments are already provided to the operator, there is no override.

Component property to operator argument mapping

Examples

Let's assume you want to run a script on a remote host and expect the script to be executed successfully within 30 seconds but might fail occasionally.

Launch a script, wait for its completion, and in case of failures or timeout retry 3 times by waiting 10 seconds between retries:

name: Run Script
operator: Executor
arguments:
  timeout: 30s
  retries: 3
  retry_delay: 10s
  command: bash /tmp/myscript.sh
  host:
    hostname: frontend.akamas.io
    username: akamas
    key: secret.key

Execute a uname command with explicit host information (explicit SSH key)

name: TestConnectivity
operator: Executor
arguments:
  command: bash uname -a
  host:
    hostname: frontend.akamas.io
    username: akamas
    key: |-
      -----BEGIN RSA PRIVATE KEY-----
      RSA KEY HERE
      -----END RSA PRIVATE KEY-----

Execute a uname command with explicit host information (imported SSH key)

name: TestConnectivity
operator: Executor
arguments:
  command: bash uname -a
  host:
    hostname: frontend.akamas.io
    username: akamas
    key: path/to/key

Execute a uname command with host information taken from a Component

name: TestConnectivity
operator: Executor
arguments:
  command: bash uname -a
  component: frontend1

Start a load-testing script and keep it running in the background during the workflow

name: TestConnectivity
operator: Executor
arguments:
  command: bash start_load.sh
  component: tester
  detach: true

Troubleshooting

Troubles in running sh scripts remotely

Due to the stderr configuration, it could happen that invoking a bash script on a server has a different result than running the same script from Akamas Executor Operator. This is quite common with Tomcat startup scripts like $HOME/tomcat/apache-tomcat_1299/bin/startup.sh.

To avoid this issue simply create a wrapper bash file on the target server adding the set -m instruction before the sh command, eg:

#!/bin/bash
set -m;
$HOME/tomcat/apache-tomcat_1299/bin/startup.sh

and then configure the Executor Operator to run the wrapper script like:

command: "bash $HOME/akamasScript/tomcatStart.sh

You can run the following to emulate the same behavior of Akamas running scripts over SSH:

ssh -t <user>@<server> <your command here>

Troubles in keeping a script running in the background

There are cases in which you would like to keep a script running for the whole duration of the test. Some examples could be:

• A script applying load to your system for the duration of the workflow

• The manual start of an application to be tested

• The setup of a listener that gathers logs, metrics, or data

In all the instances where you need to keep a task running beyond the task that started it, you must use the detach: true property. Note that a detached executor task returns immediately, so you should run only the background task in detached mode.

Remember to keep all tasks requiring synchronous (standard) behavior out of the detached task.

Example:

switch on machine and wait for SSH
run application test in background → detached mode
execute test run

Library references

The library used to execute scripts remotely is Fabric, a high-level Python library designed to execute shell commands remotely over SSH, yielding useful Python objects in return.

The Fabric library uses a connection object to execute scripts remotely (see connection — Fabric documentation). The option of a dedicated detach mode comes from implementing the more robust disown property from the Invoke Runner underlying the Connection (see runners — Invoke documentation). This is the reason you should rely on detach whenever possible instead of running the background processes straight into the script.

In the Frequently Asked/Answered Questions (FAQ) — Fabric documentation you may find some further information about the typical problems and solutions due to hanging problems for background processes.

The Sleep operator pauses a workflow for a given amount of time.

Operator arguments

Examples

Pause a workflow for 30 seconds

name: Pause
operator: Sleep
arguments:
  seconds: 30

The WindowsFileConfigurator operator allows configuring systems tuned by Akamas by interpolating configuration parameters into files on remote Windows machines.

The operator performs the following operations:

1. It reads an input file from a remote machine containing templates for interpolating the configuration parameters generated by Akamas

2. It replaces the values of configuration parameters in the input file

3. It writes the file with replaced configuration parameters on a specified path on another remote machine

Access on remote machines is performed using WinRM

Templates for configuration parameters

The Windows File Configurator allows writing templates for configuration parameters in two ways:

• a single parameter is specified to be interpolated:

• all parameters of a component to be interpolated:

Adding a suffix or prefix for interpolated parameters

It is possible to add a prefix or suffix to interpolated configuration parameters by acting at the component-type level:

In the example above, the parameter x1 will be interpolated with the prefix PREFIX and the suffix SUFFIX, ${value} will be replaced with the actual value of the parameter at each experiment.

Example

Suppose we have the configuration of the following parameters for experiment 1 of a study:

where component1 is of type MyComponentType defined as follows:

A template file to interpolate only parameter component1.param1 and all parameters from component2 would look like this:

The file after the configuration parameters are interpolated would look like this:

Note that the file in this example contains a bash command whose arguments are constructed by interpolating configuration parameters. This represents a typical use case for the WindowsFileConfigurator: to construct the right bash commands that configure a system with the new configuration parameters computed by Akamas.

Operator arguments

source and target structure and arguments

Here follows the structure of either the source or target operator argument

Get operator arguments fromcomponent

The component argument can be used to refer to a Component by name and use its properties as the arguments of the operator. In case the mapped arguments are already provided to the operator, there is no override.

Notice that in this case, the operator replaces in the template file only tokens referring to the specified component. A parameter bound to any component causes the substitution to fail.

Component property to operator argument mapping

Examples

Configure parameters for an Apache server with explicit source and target information

Configure parameters for an Apache server with information taken from a Component

Where the apache-server-1 component is defined as:

The SparkLivy operator uses Livy to run Spark applications on a Spark instance.

Operator arguments

Parameters applied from Experiments

The operator fetches the following parameters from the current Experiment to apply them to the System under test.

Examples

Execute with Livy

The SSHSparkSubmit operator connects to a Spark instance invoking a spark-submit on a machine reachable via SSH.

Operator arguments

Get operator arguments from component

This operator automatically maps some properties of its component to some arguments. In case the mapped arguments are already provided to the operator, the is no override.

Component property to operator argument mapping

The NeoLoadWeb operator allows piloting performance tests on a target system by leveraging the Tricentis NeoLoad Web solution.

Once triggered, this operator will configure and start the execution of a NeoLoad test run on the remote endpoint. When the test is unable to run then the operator blocks the Akamas workflow issuing an error.

Operator arguments

This operator requires five pieces of information to pilot successfully performance tests within Akamas:

1. The location of a .zip archive(project file) containing the definition of the performance test. This location can be a URL accessible via HTTP/HTTPS or a file path accessible via SFTP. Otherwise, the unique identifier of a previously uploaded project must be provided.

2. The name of the scenario to be used for the test

3. The URL of the NeoLoad Web API (either on-premise or SaaS)

4. The URL of the NeoLoad Web API for uploading project files

5. The account token used to access the NeoLoad Web APIs

When a projectFile is specified the Operator uploads the provided project to NeoLoad and launches the specified scenario. After the execution of the scenario, the project is deleted from NeoLoad. When a projectId is specified the Operator expects the project to be already available on NeoLoad. Please refer to on how to upload a project and obtain a project ID.

ProjectFile structure and arguments

The projectFile argument needs to be specified differently depending on the protocol used to get the specification of the performance test:

• HTTP/HTTPS

• SSH (SFTP)

HTTP/HTTPS

Here follows the structure of the projectFile argument in the case in which HTTP/HTTPS is used to get the specification of the performance test:

with its arguments:

SSH (SFTP)

Here follows the structure of the projectFile argument in the case in which SFTP is used to get the specification of the performance test.

with its arguments

component structure and arguments

The component argument can be used to refer to a component by name and use its properties as the arguments of the operator.

Component property to Operator argument mapping

Examples

Without component argument

With component argument

The SparkSubmit operator connects to a Spark instance and invokes a local spark-submit to schedule a job.

Operator arguments