Study template

Optimization studies are defined using a YAML manifest with the following structure:

system: 1
name: Optimizing the e-shop application
goal:
  objective: maximize
  function:
    formula: payments_per_sec
    variables:
      payments_per_sec:
        metric: eshop_payments
        labels:
          componentName: eshop

workflow: eshop_jmeter_test
steps:
  - name: baseline
    type: baseline
    values:
      tomcat.maxThreads: 1024
      jvm.maxHeap: 2048
      jvm.garbageCollectorType: G1GC
      postgres.shared_buffers: 4096

with the following mandatory properties:

Field
Type
Value restrictions
Is required
Default Value
Description

system

object reference

TRUE

The system the study refers to

name

string

TRUE

The name of the study

goal

object

TRUE

The goal and constraint description - see Goal & Constraints

kpis

list

FALSE

The KPIs description - see KPI

numberOfTrials

integer

FALSE

1

The number of trials for each experiment - see below

trialAggregation

string

MAX, MIN, AVG

FALSE

AVG

The aggregation used to calculate the score across multiple trials - see below

parametersSelection

list

FALSE

all

The list of parameters to be tuned - see Parameter selection

metricsSelection

list

FALSE

all

The list of metrics - see Metric selection

workloadsSelection

object array

FALSE

The list of defined workloads - this only applies to live optimization studies - see Workload Selection

windowing

string

FALSE

trim

The windowing strategy - this only applies to offline optimization studies - see Windowing strategy

workflow

object reference

TRUE

The workflow the study refers to

steps

list

TRUE

The description of the steps - see Steps

Some of these optional properties depend on whether the study is an offline or live optimization study.

Number of trials

It is possible to perform more than one trial per experiment in order to validate the score of a configuration under test, e.g. to take into account noisy environments.

The following fragment of the YAML definition of a study sets the number of trials to 3:

Notice: This is a global property of the study which can be overwritten for each step.

Trial aggregation

The trial aggregation policy defines how trial scores are aggregated to form experiment scores.

There are three different types of strategies to aggregate trial scores:

  • AVG: the score of an experiment is the average of the scores of its trials - this is the default

  • MIN: the score of an experiment is the minimum among the scores of its trials

  • MAX: the score of an experiment is the maximum among the scores of its trial

The following fragment of the YAML definition of a study sets the trial aggregation to MAX:

Examples

The following system refers to an offline optimization study for a system modeling an e-commerce service, where a windowing strategy is specified:

The following offline study refers to a tuning initiative for a Cassandra-based system (ID 2)

The following offline study is for tuning another Cassandra-based system (ID 3) by acting only on JVM and Linux parameters

Last updated

Was this helpful?