A system represents of the entire system which is the target of optimization.

A system is a single object irrespective of the number or type of entities or layers that are in the scope of the optimization. It can be used to model and describe a wide set of entities like:

• An N-layers application

• A single micro-service

• A single (or a collection of) batch job(s)

A System is made of one or more components. Each component represents one of the elements in the system, whose parameters are involved in the optimization or whose metrics are collected to evaluate the results of such optimization.

Construct

A system is described by the following properties:

• The full micro-services stack of an application

• a name that uniquely identifies the system

• a description that clarifies what the system refers to

The construct to be used to define a system is described on the System template page.

Commands

A system is an Akamas resource that can be managed via CLI using the resource management commands.

User Interface

The Akamas UI displays systems (depending on the user privileges on the defined workspaces) in a specific top-level menu.

A workflow is a set of tasks that run in sequence to evaluate a configuration as part of an optimization study. A task is a single action performed within a workflow.

Workflows allow you to automate Akamas optimization studies, by automatically executing a sequence of tasks such as initializing an environment, triggering load testing, restoring a database, applying configurations, and much more.

These are examples of common tasks:

• Launch remote commands via SSH

• Apply parameter values in configuration files

• Execute Spark jobs via spark-submit API

• Start performance tests by integrating with external tools such as Neoload

Workflows are first-class entities that can be defined globally and then used in multiple optimization studies.

Akamas provides several workflow operators that can be used to perform tasks in a workflow. Some operators are general-purpose, such as those executing a command or script on a specific host, while others provide native integrations with specific technologies and tools, such as Spark History Server or load testing tools.

Construct

The construct to be used to define a workflow is described on the Workflow template page.

Commands

A telemetry provider is an Akamas resource that can be managed via CLI using the resource management commands.

User Interface

The Akamas UI shows systems in a specific top-level menu.

The list of tasks is displayed when drilling down to each specific workflow.

A component represents an element of a system. Typically, systems are made up of different entities and layers which can be modeled by components. In other words, a system can be considered a collection of related components.

Notice that a component is a black-box definition of each entity involved in an optimization study, so detailed modeling of the entities being involved in the optimization is not required. The only relevant elements are the parameters that are involved in the optimization and the metrics that are collected to evaluate the results of such an optimization.

Notice that only the entities that are directly involved in the optimization need to be modeled and defined within Akamas. An entity is involved in an optimization study if it is optimized or monitored by Akamas, where "optimized" means that Akamas is optimizing at least one of its parameters, and "monitored" means that Akamas is monitoring at least one of its metrics.

Construct

A component is described by the following mandatory properties (other properties can be defined but are not mandatory):

• a name that uniquely identifies the component within the system

• a description that clarifies what the component refers to

• a component type that identifies the technology of the component (see component type)

In general, a component contains a set of each of the following:

• parameter(s) in the scope of the optimization

• metric(s) needed to define the optimization goal

• metric(s) needed to define the optimization constraints

• metric(s) that are not needed to either define the optimization goal or constraints, and hence not used by Akamas to perform the optimization, but are collected in order to support the analysis (and which can be possibly added at a later time as part of optimization goal or constraint when refining the optimization).

The construct to be used to define a component is described on the Component template page.

Commands

A component is an Akamas resource that can be managed via CLI using the resource management commands.

User Interface

The Akamas UI shows more details about components by drilling down their respective system.

A metric is a measured property of a system.

Examples of a metric include:

• the response time of an application

• the utilization of a CPU

• the amount of time spent in garbage collection

• the cost of a cloud service

Metrics are used to both specify the optimization goal and constraints (e.g. minimize the heap size while keeping response time < 1000 and error rate <= 10% of a baseline value), and to assess the behavior of the system with respect to each specific configuration applied.

Construct

A metric is described by the following properties:

• a name that uniquely identifies the metric

• a description that clarifies the semantics of the metric

• a unit that defines the unit of measurement used by the metric

The construct to be used to define a metric is described on the Metric template page.

User Interface

Metrics are displayed in the Akamas UI when drilling down to each system component.

and are represented in metric charts for each specific optimization study

Please notice that in order for a metric to be displayed in the Akamas UI, it has to be collected from a Telemetry Provider by means of a specific Telemetry Instance defined for each specific target system.

A component type is a blueprint for a component that describes the type of entity the component refers to. In Akamas, a component needs to be associated with a component type, from which the component inherits its metrics and parameters.

Component types are platform entities (i.e.: shared among all the users) usually provided off the shelf and shipped within the Optimization Packs. Typically, different component types within the same optimization pack are used to model different versions/releases of the same technology.

Akamas' users with appropriate privileges can create custom component types and optimization packs, as described on the Creating custom optimization pack page.

Construct

A component type is described by the following mandatory properties (other properties can be defined but are not mandatory):

• a name that uniquely identifies the component type within the system

• a description that clarifies what the component type refers to

• a parameter definitions array (more on Parameters later)

• a metrics array (more on Metrics later)

The construct to be used to define a component type is described on the Component type template page.

Commands

A component type is an Akamas resource that can be managed via CLI using the resource management commands.

User Interface

When visualizing system components the component type is displayed.

The following figure shows the out-of-the-box JVM component types related to the JVM optimization pack.

A parameter is a property of the system that can be applied and tuned to change the system's behavior. Akamas optimizes systems by changing parameters to achieve the stated goal while respecting the defined constraints.

Examples of a parameter include:

• Configuration knobs (e.g. JVM garbage collection type)

• Resource settings (e.g. amount of memory allocated to a Spark job)

• Algorithms settings (e.g. learning rate of a neural network)

• Architectural properties (e.g. how many caching layers in an enterprise application)

• Type of resources (e.g. AWS EC2 instance or EBS volume type)

• Any other thing (e.g. amount of sugar in your cookies)

The following table describes the parameter types:

Construct

A parameter is described by the following properties:

• a name that uniquely identifies the parameter

• a description that clarifies the semantics of the parameter

• a unit that defines the unit of measurement used by the parameter

Although users can create parameters with any name, we suggest using the naming convention context_parameter where

• context refers to the technology or more general environment in which that metric is defined (e.g. elasticsearch, jvm, mysql, spark)

• parameter is the parameter name in the original context (e.g. gcType, numberOfExecutors)

This makes it possible to identify parameters more easily and avoid any potential name clash.

The construct to be used to define a parameter is described on the Parameter template page.

User Interface

Parameters are displayed in the Akamas UI when drilling down to each system component.

For each optimization study, the optimization scope is the set of parameters that Akamas can change to achieve the defined optimization goal.

The optimization goal defines the objective of an to be achieved by changing the system parameters to modify the system behavior while also satisfying any defined optimization constraints on the system metrics, possibly representing SLOs.

Construct

A goal is defined by:

• an optimization objective: either maximize or minimize

• a scoring function (scalar): either a single metric or a formula defined by one or more metrics

One or more constraints can be associated with a goal

• a formula defined on one or more metrics, referring to either absolute values (absolute constraints) or relative to a baseline value (relative constraints)

Notice that relative constraints are only supported by while absolute constraints are supported by both offline and .

Goals and constraints are not an as they are defined as part of an optimization study. The construct to be used to define a goal and its constraints are described in the page of the section.

Commands

Goals and constraints are not an and are always defined as part of an optimization study.

User Interface

Goals and constraints are displayed in the Akamas UI when drilling down each optimization study.

The detail of the formula used to define the goal may also be displayed:

A telemetry instance is an instance of a that collects data from a specific instance of the data source.

A telemetry instance is an instance of a telemetry provider, providing the required information on how to connect and collect a given set of metrics from a specific data source.

While telemetry providers are platform-wide entities, telemetry instances are defined at each system level.

Construct

The construct to be used to define a telemetry instance is described on the page.

Commands

A telemetry provider is an that can be managed via CLI using the

User Interface

Telemetry instances are displayed in the Akamas UI when drilling down each system component.

An optimization pack is a software object that provides a convenient facility for encapsulating all the knowledge (e.g. metrics, parameters with their default values and domain ranges) required to apply Akamas optimizations to a set of entities associated with the same technology.

Notice that while optimization packs are very convenient for modeling systems and creating studies, it is not required for these entities to be covered by an optimization pack.

Akamas provides a library of out-the-box optimization packs and new custom optimization packs can be easily added (no coding is required).

Construct

An optimization pack needs to include the entities that encapsulate technology-specific information related to the supported component types:

• supported component types

• parameters and metrics for each component type

• supported telemetry providers (optional)

Commands

An optimization pack is an that can be managed via CLI using the

User Interface

The Akamas UI shows systems in a specific top-level menu.

An optimization pack encapsulates one or more of the following technology-specific elements:

• Component Types: these represent the type of the component(s) included, each with its associated parameters and metrics

• Telemetry Providers: that define where to collect metrics

An optimization pack enables Akamas users to optimize a technology without necessarily being an expert in that technology and to code their knowledge about a technology or a specific application to be reused in multiple optimization studies to ease the modeling process.

A KPI is a metric that is worth considering when analyzing the result of an offline optimization study, looking for (sub)optimal configurations generated by Akamas AI to be applied.

Akamas automatically considers any metric referred to in the defined optimization goal and constraints for an offline optimization study as a KPI. Moreover, any other metrics of the system component can be specified as a KPI for an offline optimization study.

Construct

A KPI is defined as follows (from the UI or the CLI):

KPIs are not an as they are defined as part of an optimization study. The construct to define KPIs is described on the page of the section.

Commands

KPIs are not an and are always defined as part of an optimization study.

User Interface

The number and first KPIs are displayed in the Akamas UI in the header of each offline optimization study.

The full list of KPIs is displayed by drilling down to the KPIs section.

From this section, it is possible to modify the list of KPIs and change their names and other attributes.

This guide provides a describing Akamas key concepts with their associated construct templates, command line commands, and user interfaces.

This guide also provides references to:

• structure and operators

• metric mapping

• metrics and parameters

• to administer Akamas, manage users, authenticate and manage its resources

A telemetry provider is a software object that represents a data source of metrics. A is a specific instance of a telemetry provider that refers to a specific data source.

Examples of telemetry providers are:

• monitoring tools (e.g. Prometheus or Dynatrace)

• load testing tools (e.g. LoadRunner or Neoload)

• CSV files

A telemetry provider is a platform-wide entity that can be reused across systems to ease the integration with metrics sources.

Akamas provides a number of out-of-the-box . Custom telemetry providers can also be created.

Construct

The construct to be used to define a telemetry provider is described on the page.

Commands

A telemetry provider is an that can be managed via CLI using the

User Interface

The Akamas UI shows systems in a specific top-level menu.

While Akamas leverages similar AI methods for both live optimizations and optimization studies, the way these methods are applied is radically different. Indeed, for optimization studies running in pre-production environments, the approach is to explore the configuration space by also accepting potential failed experiments, to identify regions that do not correspond to viable configurations. Of course, this approach cannot be accepted for live optimization running in production environments. For this purpose, Akamas live optimization uses observations of configuration changes combined with the automatic detection of workload contexts and provides several customizable safety policies when recommending configurations to be approved, revisited, and applied.

Akamas provides a few customizable optimizer options (refer to the options described on the Optimize step page of the reference guide) that should be configured so as to make configurations recommended in live optimization and applied to production environments as safe as possible.

Exploration factor

Akamas provides an optimizer option known as the exploration factor that only allows gradual changes to the parameters. This gradual optimization allows Akamas to observe how these changes impact the system behavior before applying the following gradual changes.

By properly configuring the optimizer, Akamas can gradually explore regions of the configuration space and slowly approach any potentially risky regions, thus avoiding recommending any configurations that may negatively impact the system. Gradual optimization takes into account the maximum recommended change for each parameter. This is defined as a percentage (default is 5%) with respect to the baseline value. For example, in the case of a container whose CPU limit is 1000 millicores, the corresponding maximum allowed change is 50 millicores. It is important to notice that this does not represent an absolute cap, as Akamas also takes into account any good configurations observed. For example, in the event of a traffic peak, Akamas would recommend a good configuration that was observed working fine for a similar workload in the past, even if the change is higher than 5% of the current configuration value.

Notice that this feature would not work for categorical parameters (e.g. JVM GC Type) as their values do not change incrementally. Therefore, when it comes to these parameters, Akamas by default takes a conservative approach of only recommending configurations with categorical parameters taking already observed before values. This still allows some never-observed values to be recommended as users are allowed to modify values also for categorical parameters when operating in human-in-the-loop mode. Once Akamas has observed that that specific configuration is working fine, the corresponding value can then be recommended. For example, a user might modify the recommended configuration for GC Type from Serial to Parallel. Once Parallel has been observed as working fine, Akamas would consider it for future recommendations of GC Type, while other values (e.g. G1) would not be considered until verified as safe recommendations.

The exploration factor can be customized for each live optimization individually and changed while live optimizations are running.

Safety factor

Akamas provides an optimizer option known as the safety factor designed to prevent Akamas from selecting configurations (even if slowly approaching them) that may impact the ability to match defined SLOs. For example, when optimizing container CPU limits, lower and lower CPU limits might be recommended, up to the point that the limit becomes too low that the application performance degrades.

Akamas takes into account the magnitude of constraint breaches: a severe breach is considered more negative than a minor breach. For example, in the case of an SLO of 200 ms on response time, a configuration causing a 1 sec response time is assigned a very different penalty than a configuration causing a 210 ms response time. Moreover, Akamas leverages the smart constraint evaluation feature that takes into account if a configuration is causing constraints to approach their corresponding thresholds. For example, in the case of an SLO of 200 ms on response time, a configuration changing response time from 170 ms to 190 ms is considered more problematic than one causing a change from 100 ms to 120 ms. The first one is considered by Akamas as corresponding to a gray area that should not be explored.

The safety factor is also used when starting the study in order to validate the behavior of the baseline to identify the safety of exploring configurations close to the baseline. If the baseline presents some constraint violations, then even exploring configurations close to the baseline might cause a risk. If Akamas identifies that, in the baseline configuration, more than (safety_factor*number_of_trials) manifest constraint violations then the optimization is stopped.

If your baseline has some trials failing constraint validation we suggest you analyze them before proceeding with the optimization

The safety factor is set by default to 0.5 and can be customized for each live optimization individually and changed while live optimizations are running.

Outlier detection

It is also worth mentioning that Akamas also features an outlier detection capability to compensate for production environments typically being noisy and much less stable than staging environments, thus displaying highly fluctuating performance metrics. As a consequence, constraints may fail from time to time, even for perfectly good configurations. This may be due to a variety of causes, such as shared infrastructure on the cloud, slowness of external systems, etc.

An optimization study (or study for short) represents an optimization initiative aimed at optimizing a goal on a target system. A study instructs Akamas about the space to explore and the KPIs used to evaluate whether a con configuration is good or bad

Akamas supports two types of optimizations:

• Offline Optimization Studies are optimization studies where the workload is simulated by leveraging a load-testing tool.

• Live Optimization Studies are applied to systems that need to be optimized in production with respect to varying workloads observed while running live. For example, a microservices application can be optimized live by having Kubernetes and JVM parameters dynamically tuned for multiple microservices so as to minimize costs while matching response time objectives.

Construct

A study is described by the following properties

• system: the system under optimization

• parameters: the set of parameters being optimized

• metrics: the set of metrics to be collected

• workflow: the workflow describing tasks to perform experiments/trials

• goal: the desired optimization goal to be achieved

• constraints: the optimization constraints that any configuration needs to satisfy

• steps: the steps that are executed to run specific configurations (e.g. the baseline) and run the optimization

The construct to be used to define an optimization is described on the Study template page.

Commands

An optimization study is an Akamas resource that can be managed via CLI using the resource management commands.

User Interface

The Akamas UI shows optimization studies in 2 specific top-level menus: one for offline optimization studies and another for live optimization studies.

A workspace is a virtual environment that groups systems, workflows, and studies to restrict user access to them: a user can access these resources only when granted the required permissions to that workspace.

Akamas defines two user roles according to the assigned permission on the workspace:

• Contributors (write permission) can create and manage workspace resources (studies, telemetry instances, systems, and workflows) and can also do exports/imports, view all global resources (Optimization Packs, and Telemetry Providers), and see remaining credits;

• Viewers (read permission) can only access optimization results but cannot create or modify workspace resources.

Workspaces and accesses are managed by users with administrative privileges. A user with administrator privileges can manage licenses, users, workspaces, and install/deinstall Optimization Packs, and Telemetry Providers.

Workspaces can be defined according to different criteria, such as:

• By department (e.g. Performance, Development)

• By initiative (e.g. Poc, Training)

• By application (e.g. Registry, Banking..)

A workspace is described by the following property:

• a name that uniquely identifies the workspace

Commands

A workspace is an Akamas resource that can be managed via CLI using the resource management commands. See also this page devoted to commands on how to define users and workspaces.

User Interface

The workspace a study belongs to is always displayed. Filters can be used to select only studies belonging to specific workspaces

Component types are defined using a YAML manifest with the following structure:

# General section
name: function_branin
description: A component type for the branin analytical function

# Parameters section
parameters:
  - name: x1
    domain:
      type: real
      domain: [-5.0, 10.0]
    defaultValue: -5.0
    decimals: 3
    operators:
    FileConfigurator:
      confTemplate: "${value}"

  - name: x2
    domain:
      type: real
      domain: [0.0, 15.0]
    defaultValue: 0.0

  - name: x3
    domain:
      type: categorical
      categories: [cat1,cat2,cat3]
    operators:
    LinuxConfigurator:
      echo:
        file: /sys/class/block/nvme0n1/queue/scheduler

# Metrics section
metrics:
  - name: function_value

and properties for the general section:

The parameter section describes the relationship between the component type and already defined parameters with the following properties:

The metric section describes the relationship between the component type and already defined metrics with the following properties:

Notice that component type definitions are shared across all the workspaces on the same Akamas installation, and require an account with administrative privileges to manage them.

Examples

Example of a component for the Cassandra component type:

name: Cassandra
description: The Cassandra NoSQL database version 3
parameters:
  - name: cassandra_compactionStrategy
    domain:
      type: categorical
      categories: [A, B]
    defaultValue: A

metrics:
  - name: total_rate
  - name: read_rate
  - name: write_rate
  - name: read_response_time_avg
  - name: read_response_time_p90
  - name: read_response_time_p99
  - name: read_response_time_max
  - name: write_response_time_avg
  - name: write_response_time_p90
  - name: write_response_time_p99
  - name: write_response_time_max

Example of a component for the Linux operating component type:

name: Linux OS
description: A component type for the Linux Operating System
parameters:
  #CPU Related
  - name: os_cpuSchedMinGranularity
    domain:
      type: integer
      domain: [300000, 30000000]
    defaultValue: 3000000
  - name: os_cpuSchedWakeupGranularity
    domain:
      type: integer
      domain: [400000, 40000000]
    defaultValue: 4000000
  - name: osCpu.schedMigrationCost
    domain:
      type: integer
      domain: [100000, 5000000]
    defaultValue: 500000
  - name: os_CPUSchedChildRunsFirst
    domain:
      type: integer
      domain: [0, 1]
    defaultValue: 0
  - name: os_CPUSchedLatency
    domain:
      type: integer
      domain: [2400000, 240000000]
    defaultValue: 24000000
  - name: os_CPUSchedAutogroupEnabled
    domain:
      type: integer
      domain: [0, 1]
    defaultValue: 1
  - name: os_CPUSchedNrMigrate
    domain:
      type: integer
      domain: [3, 320]
    defaultValue: 32
  #Memory Related
  - name: os_MemorySwappiness
    domain:
      type: integer
      domain: [0, 100]
    defaultValue: 60
  - name: os_MemoryVmVfsCachePressure
    domain:
      type: integer
      domain: [10, 100]
    defaultValue: 100
  - name: os_MemoryVmMinFree
    domain:
      type: integer
      domain: [10240, 1024000]
    defaultValue: 67584
  - name: os_MemoryVmDirtyRatio
    domain:
      type: integer
      domain: [1, 99]
    defaultValue: 10
  - name: os_MemoryTransparentHugepageEnabled
    domain:
      type: categorical
      categories: ['True', 'False']
    defaultValue: 'True'
  - name: os_MemoryTransparentHugepageDefrag
    domain:
      type: categorical
      categories: ['True', 'False']
    defaultValue: 'True'
  - name: os_MemorySwap
    domain:
      type: categorical
      categories: ['True', 'False']
    defaultValue: 'True'
  - name: os_MemoryVmDirtyExpire
    domain:
      type: integer
      domain: [300, 30000]
    defaultValue: 3000
  - name: os_MemoryVmDirtyWriteback
    domain:
      type: integer
      domain: [50, 5000]
    defaultValue: 500

metrics:
  - name: cpu_num
  - name: cpu_util
  - name: mem_util
  - name: load_avg
  - name: swapins
  - name: swapouts
  - name: disk_iops_writes
  - name: disk_iops_reads
  - name: disk_iops_total
  - name: disk_await_worst
  - name: proc_blocked
  - name: context_switch
  - name: tcp_retrans
  - name: tcp_tozerowin
  - name: net_band_rx_bits
  - name: net_band_tx_bits
  - name: network_in_byte_rate
  - name: network_out_byte_rate
  - name: mem_fault_minor
  - name: mem_fault_major
  - name: mem_active_file
  - name: mem_active_anon
  - name: mem_inactive_file
  - name: mem_inactive_anon

Systems are defined using a YAML manifest with the following structure:

# General section
name: Analytical functions
description: A collection of analytical functions

with the following properties:

Example

The following represents a system (for Cassandra related system)

name: system1
description: my system with 3 nodes of cassandra

This section describes all the structures that can be used to define resources and objects in Akamas.

Components are defined using a YAML manifest with the following structure:

and properties:

Examples

Example of a component for OpenJDK11:

Example of a component for the Linux operating system:

Parameters are defined using a YAML manifest with the following structure:

with the following properties:

Notice that parameter definitions are shared across all the workspaces on the same Akamas installation, and require an account with administrative privileges to manage them.

Example

The following represents a set of parameters for a JVM component

The following represents a set of CPU-related parameters for the Linux operating system

Telemetry Providers are defined using a YAML manifest with the following structure:

with the following properties:

Refer to the page which describes the out-of-the-box Telemetry Providers that are created automatically at Akamas install time.

Telemetry instances are defined using a YAML manifest with the following structure:

with the following properties for the global section

and the metrics section

This section provides a definition of Akamas' key concepts and terms and also provides references to the related construct properties, commands, and user interfaces.

Optimization studies are defined using a YAML manifest with the following structure:

with the following mandatory properties:

Some of these optional properties depend on whether the study is an offline or live optimization study.

Number of trials

It is possible to perform more than one trial per experiment to validate the score of a configuration under test, e.g., to consider noisy environments.

The following fragment of the YAML definition of a study sets the number of trials to 3:

Notice: This is a global property of the study which can be overwritten for each step.

Trial aggregation

The trial aggregation policy defines how trial scores are aggregated to form experiment scores.

There are three different types of strategies to aggregate trial scores:

• AVG: the score of an experiment is the average of the scores of its trials - this is the default

• MIN: the score of an experiment is the minimum among the scores of its trials

• MAX: the score of an experiment is the maximum among the scores of its trial

The following fragment of the YAML definition of a study sets the trial aggregation to MAX:

Examples

The following system refers to an offline optimization study for a system modeling an e-commerce service, where a windowing strategy is specified:

The following offline study refers to a tuning initiative for a Cassandra-based system (ID 2)

The following offline study is for tuning another Cassandra-based system (ID 3) by acting only on JVM and Linux parameters

Workflow are defined using a YAML manifest with the following structure:

with the following properties:

The full list of Operators and related options is provided on the pages.

Example

A workflow for the java-based renaissance benchmark application

A windowing policy of type stability discards temporal intervals in which a given metric is not stable, and selects, among the remaining intervals, the ones in which another target metric is maximized or minimized. Stability windowing can be sample-based or time-frame based.

The stability windowing has the following structure:

and for the comparison metric section

Example

The following fragment is an example of stability windowing (time-frame based):

A windowing policy of type trim trims the temporal interval of a trial, both from the start and from the end of a specified temporal amount (e.g., 3 seconds).

The trim windowing has the following structure:

In case a windowing policy is not specified, the default windowing corresponding to trim[0s,0s] is considered.

Example

The following fragment shows a windowing strategy of type "trim" where the time window is specified to start 10s after the beginning of the trial and to end immediately before the end of the trial:

Optimization goals and constraints are defined using a YAML manifest with the following structure:

where:

Function

The function field of the Goal of a Study details the characteristics of the function Akamas should minimize or maximize to reach the desired performance objective.

The function field has the following structure:

Where:

Formula

The formula field represents the mathematical expression of the performance objective for the Study and contains variables and operators with the following characteristics:

• Valid operators are: +, -, *, /, ^, sqrt(variable), log(variable), max(variable1, variable2), and min(variable1, variable2)

• Valid variables are in the form:

  • <component_name>.<metric_name>, which correspond directly to metrics of Components of the System under test

  • <variable_name>, which should match variables specified in the variables field

Variables

The variables field contains the specification of additional variables present in the formula, variables that can offer more flexibility compared to directly specifying each metric of each Component in the formula.

Notice: each subfield of variables specifies a variable with its characteristics, the name of the subfield is the name of the variable.

The variable subfield has the following structure:

It is possible to use the notation <component_name>.<metric_name> in the metric field to automatically filter the metric’s data point by that component name is applied.

Constraints

The constraints field specifies constraints on the metrics of the system under test. For a configuration to be valid for the defined goal, such constraints must be satisfied. Constraints can be defined as absolute or relativeToBaseline.

Each constraint has the form of:

mathematical_operationcomparison_operatorvalue_to_compare

where valid mathematical operations include:

• + - * / ^

• min max

• sqrt log (log is a natural logarithm)

valid comparison operators include:

• > < <= >=

• == != (equality, inequality)

and valid values to compare include:

• absolute values (e.g, 104343)

• percentage values relative to the baseline (e.g, 20%)

As an example, you could define an absolute constraint with the following snippet:

Relative constraints can be defined by adding other constraints under the relativeToBaseline section. In the example below, for the configuration to be considered valid, it's required that the metric jvm.memory_used does not exceed by 80% the value measured in the baseline.

Aggregations

Variables used in the study formula specification and in the constraints definition can include an aggregation. The following aggregations are available: avg, min, max, sum, p90, p95, p99.

Examples

The following example refers to a study whose goal is to optimize the throughput of a Java service (jpetstore), that is to maximize the throughput (measured as elements_per_second) while keeping errors (error_rate) and latency (avg_duration, max_duration) under control (absolute values):

The following example refers to a study whose goal is to optimize the memory consumption of Docker containers in a microservices application, that is to minimize the average memory consumption of Docker containers within the application of appId="app1" by observing memory limits, also normalizing by the maximum duration of a benchmark (containers_benchmark_duration).

The Windowing field in a study specifies the windowing policy to be adopted to score the experiments of an optimization study.

The two available windowing strategies have different structures:

• : trim the temporal interval of a trial, both from the start and the end of a specified temporal amount - this is the default strategy

• : discard temporal intervals in which a given metric is not stable and selects the temporal interval in which a metric is maximized or minimized.

In case the windowing strategy is not specified, the entire time window is considered.

Metrics are defined using a YAML manifest with the following structure:

and properties:

Supported units of measure

The supported units of measure for metrics are:

Notice that supported units of measure are automatically scaled for visualization purposes. In particular, for units of information, Akamas uses a base 2 scaling for bytes, i.e., 1 kilobyte = 1024 bytes, 1 megabyte = 1024 kilobytes, and so on. Other units of measure are only scaled up using millions or billions (e.g., 124000000 custom units become 124 Mln custom units).

The FileConfigurator operator allows configuring systems tuned by Akamas by interpolating configuration parameters into files on remote machines.

The operator performs the following operations:

1. It reads an input file from a remote machine containing templates for interpolating the configuration parameters generated by Akamas

2. It replaces the values of configuration parameters in the input file

3. It writes the file with replaced configuration parameters on a specified path on another remote machine

Access on remote machines is performed using SFTP (SSH).

Templates for configuration parameters

The FileConfigurator allows writing templates for configuration parameters in two ways:

• specify that a parameter should be interpolated directly:

• specify that all parameters of a component should be interpolated:

Suffix or prefix for interpolated parameters

It is possible to add a prefix or suffix to interpolated configuration parameters by acting at the component-type level:

Notice that any parameter that does not contain the FileConfigurator element in the operators' attribute is ignored and not written.

In the example above, the parameter x1 will be interpolated with the prefix PREFIX and the suffix SUFFIX, ${value} will be replaced with the actual value of the parameter at each experiment.

Example

Let's assume we want to apply the following configuration:

where component1 is of type MyComponentType and MyComponentType is defined as follows:

A template file to interpolate only parameter component1.param1 and all parameters from component2 would look like this:

The file after the configuration parameters are interpolated would look like this:

Note that the file in this example contains a bash command whose arguments are constructed by interpolating configuration parameters. This represents a typical use case for the File Configurator: to construct the right bash commands that will configure a system with the new configuration parameters computed by Akamas.

Operator arguments

source and target structures and arguments

Here follows the structure of either the source or target operator argument

Get operator arguments from component

The component argument can be used to refer to a component by name and use its properties as the arguments of the operator. In case the mapped arguments are already provided to the operator, there is no override.

In this case, the operator replaces in the template file only tokens referring to the specified component. A parameter bound to any component will cause the substitution to fail.

Component property to operator argument mapping

Examples

Configure parameters for an Apache server with explicit source and target machine information

Configure parameters for an Apache server with information taken from a Component

where the apache-server-1 component is defined as:

The Steps field in a study specifies the sequence of steps executed while running the study. These steps are run in exactly the same order in which they will be executed.

The following types of steps are available:

• Baseline: performs an experiment and sets it as a baseline for all the other ones

• Bootstrap: imports experiments from other studies

• Preset: performs an experiment with a specific configuration

• Optimize: performs experiments and generates optimized configurations

Notice that the structure corresponding to the steps is different for the different types of steps.

The MetricSelection field in a study specifies the metrics of the system. Such selection has only the purpose to specify which metrics need to be tracked while running the study and does not affect the optimization.

In case this selection is not specified, all metrics are considered.

A metrics selection can either assume the value of all to indicate that all the available metrics of the system of the study should be tracked, or can assume the value of a list of the names of metrics of the system that should be tracked prepended with the name of the component.

Example

The following fragment is an example:

metricsSelection:
  - application.response_time
  - application.error_rate
  - jvm1.gc_time