Akamas Docs
3.2.2
Search
K

Study template

Optimization studies are defined using a YAML manifest with the following structure:
1
system: 1
2
name: Optimizing the e-shop application
3
goal:
4
objective: maximize
5
function:
6
formula: payments_per_sec
7
variables:
8
payments_per_sec:
9
metric: eshop_payments
10
labels:
11
componentName: eshop
12
13
workflow: eshop_jmeter_test
14
steps:
15
- name: baseline
16
type: baseline
17
values:
18
tomcat.maxThreads: 1024
19
jvm.maxHeap: 2048
20
jvm.garbageCollectorType: G1GC
21
postgres.shared_buffers: 4096
with the following mandatory properties:
Field
Type
Value restrictions
Is required
Default Value
Description
system
object reference
TRUE
The system the study refers to
name
string
TRUE
The name of the study
goal
object
TRUE
The goal and constraint description - see Goal & Constraints
kpis
list
FALSE
The KPIs description - see KPI
numberOfTrials
integer
FALSE
1
The number of trials for each experiment - see below
trialAggregation
string
MAX, MIN, AVG
FALSE
AVG
The aggregation used to calculate the score across multiple trials - see below
parametersSelection
list
FALSE
all
The list of parameters to be tuned - see Parameter selection
metricsSelection
list
FALSE
all
The list of metrics - see Metric selection
workloadsSelection
object array
FALSE
The list of defined workloads - this only applies to live optimization studies - see Workload Selection
windowing
string
FALSE
trim
The windowing strategy - this only applies to offline optimization studies - see Windowing strategy
workflow
object reference
TRUE
The workflow the study refers to
steps
list
TRUE
The description of the steps - see Steps
Some of these optional properties depend on whether the study is an offline or live optimization study.

Number of trials

It is possible to perform more than one trial per experiment in order to validate the score of a configuration under test, e.g. to take into account noisy environments.
The following fragment of the YAML definition of a study sets the number of trials to 3:
numberOfTrials: 3
Notice: This is a global property of the study which can be overwritten for each step.

Trial aggregation

The trial aggregation policy defines how trial scores are aggregated to form experiment scores.
There are three different types of strategies to aggregate trial scores:
  • AVG: the score of an experiment is the average of the scores of its trials - this is the default
  • MIN: the score of an experiment is the minimum among the scores of its trials
  • MAX: the score of an experiment is the maximum among the scores of its trial
The following fragment of the YAML definition of a study sets the trial aggregation to MAX:
trialAggregation: MAX # other possible values are AVG, MIN

Examples

The following system refers to an offline optimization study for a system modeling an e-commerce service, where a windowing strategy is specified:
1
system: "bde4f259-9a51-4c67-87aa-3c5bc599c6b9" # id of the system to optimize with the actions defined in this study
2
workflow: "eshop_jmeter_test" # name of the workflow to use to perform trials
3
name: Optimizing the e-shop application # name of the study
4
goal: # the performance goal to achieve
5
objective: "maximize"
6
function:
7
formula: "eshop.payments_per_second"
8
windowing: # the temporal window in which to compute the score of a trial
9
type: "trim"
10
trim: ["10s", "0s"] # use the duration of the trial minus 0s from start and end to compute the score of the trial
11
parametersSelection: "all" # use all available configuration parameters
12
metricsSelection: "all" # gather all metrics
13
steps: # the steps to conduct to perform experiments and trials
14
- name: "my_baseline" # do first a baseline with the provided configuration
15
type: "baseline"
16
values:
17
jvm.maxHeap: 2048
18
jvm.gcType: "-XX:+UseParallelGC"
19
- name: my_optimization # then do 20 optimization experiments of 2 trials each
20
type: optimize
21
numberOfExperiments: 200
22
numberOfTrials: 2
The following offline study refers to a tuning initiative for a Cassandra-based system (ID 2)
1
system: 2
2
name: Optimizing the cassandra - team 2
3
goal:
4
objective: minimize
5
function:
6
formula: read_response_time_p90
7
variables:
8
read_response_time_p90:
9
metric: read_response_time_p90
10
labels:
11
componentName: cassandra
12
13
windowing:
14
type: trim
15
trim: [5m, 1m]
16
17
workflow: cassandra_workflow
18
parametersSelection:
19
- name: cassandra_jvm.jvm_maxHeapSize
20
- name: cassandra.cassandra_concurrentReads
21
- name: cassandra.cassandra_concurrentWrites
22
- name: cassandra.cassandra_fileCacheSizeInMb
23
- name: cassandra.cassandra_memtableCleanupThreshold
24
- name: cassandra.cassandra_concurrentCompactors
25
26
steps:
27
- name: baseline_step
28
type: baseline
29
values:
30
cassandra_jvm.jvm_maxHeapSize: 1024
31
cassandra.cassandra_concurrentReads: 32
32
cassandra.cassandra_concurrentWrites: 32
33
cassandra.cassandra_fileCacheSizeInMb: 512
34
cassandra.cassandra_memtableCleanupThreshold: 0.11
35
cassandra.cassandra_concurrentCompactors: 2
36
37
- name: optimization_step
38
type: optimize
39
optimizer: CALABI
40
numberOfExperiments: 50
The following offline study is for tuning another Cassandra-based system (ID 3) by acting only on JVM and Linux parameters
1
system: 3
2
name: Optimizing a Cassandra NoSQL database version 3 (jvm + os parameters)
3
goal:
4
objective: minimize
5
function:
6
formula: (x1+x2)/2
7
variables:
8
x1:
9
metric: write_response_time_p90
10
labels:
11
componentName: cassandra_team1
12
x2:
13
metric: read_response_time_p90
14
labels:
15
componentName: cassandra_team1
16
17
windowing:
18
type: trim
19
trim: [8m,2m]
20
21
numberOfTrials: 2
22
workflow: cassandra_workflow_jvm_os
23
24
parametersSelection:
25
- name: JVM1.jvm_maxHeapSize
26
- name: JVM1.jvm_newRatio
27
- name: JVM1.jvm_survivorRatio
28
- name: JVM1.jvm_maxTenuringThreshold
29
- name: JVM1.jvm_gcType
30
- name: JVM1.jvm_concurrentGCThreads
31
- name: os1.os_cpuSchedMinGranularity
32
- name: os1.os_cpuSchedWakeupGranularity
33
- name: os1.os_CPUSchedMigrationCost
34
- name: os1.os_CPUSchedChildRunsFirst
35
- name: os1.os_CPUSchedLatency
36
37
steps:
38
- name: baseline_step
39
type: baseline
40
values:
41
JVM_team1.jvm_maxHeapSize: 1024
42
JVM_team1.jvm_newRatio: 2
43
JVM_team1.jvm_survivorRatio: 8
44
JVM_team1.jvm_maxTenuringThreshold: 15
45
JVM_team1.jvm_gcType: UseConcMarkSweepGC
46
JVM_team1.jvm_concurrentGCThreads: 8
47
os_team1.os_cpuSchedMinGranularity: 3000000
48
os_team1.os_cpuSchedWakeupGranularity: 4000000
49
os_team1.os_CPUSchedMigrationCost: 500000
50
os_team1.os_CPUSchedChildRunsFirst: 0
51
os_team1.os_CPUSchedLatency: 24000000
52
53
- name: optimization_sobol
54
type: optimize
55
optimizer: SOBOL
56
numberOfExperiments: 3
57
58
- name: optimization_calabi
59
type: optimize
60
optimizer: CALABI
61
numberOfExperiments: 50