Study template
Optimization studies are defined using a YAML manifest with the following structure:
system: 1
name: Optimizing the e-shop application
goal:
objective: maximize
function:
formula: payments_per_sec
variables:
payments_per_sec:
metric: eshop_payments
labels:
componentName: eshop
workflow: eshop_jmeter_test
steps:
- name: baseline
type: baseline
values:
tomcat.maxThreads: 1024
jvm.maxHeap: 2048
jvm.garbageCollectorType: G1GC
postgres.shared_buffers: 4096with the following mandatory properties:
system
object reference
TRUE
The system the study refers to
name
string
TRUE
The name of the study
goal
object
TRUE
kpis
list
FALSE
numberOfTrials
integer
FALSE
1
The number of trials for each experiment - see below
trialAggregation
string
MAX, MIN, AVG
FALSE
AVG
The aggregation used to calculate the score across multiple trials - see below
parametersSelection
list
FALSE
all
metricsSelection
list
FALSE
all
workloadsSelection
object array
FALSE
windowing
string
FALSE
trim
workflow
object reference
TRUE
The workflow the study refers to
steps
list
TRUE
Some of these optional properties depend on whether the study is an offline or live optimization study.
Number of trials
It is possible to perform more than one trial per experiment in order to validate the score of a configuration under test, e.g. to take into account noisy environments.
The following fragment of the YAML definition of a study sets the number of trials to 3:
numberOfTrials: 3Notice: This is a global property of the study which can be overwritten for each step.
Trial aggregation
The trial aggregation policy defines how trial scores are aggregated to form experiment scores.
There are three different types of strategies to aggregate trial scores:
AVG: the score of an experiment is the average of the scores of its trials - this is the default
MIN: the score of an experiment is the minimum among the scores of its trials
MAX: the score of an experiment is the maximum among the scores of its trial
The following fragment of the YAML definition of a study sets the trial aggregation to MAX:
trialAggregation: MAX # other possible values are AVG, MINExamples
The following system refers to an offline optimization study for a system modeling an e-commerce service, where a windowing strategy is specified:
system: "bde4f259-9a51-4c67-87aa-3c5bc599c6b9" # id of the system to optimize with the actions defined in this study
workflow: "eshop_jmeter_test" # name of the workflow to use to perform trials
name: Optimizing the e-shop application # name of the study
goal: # the performance goal to achieve
objective: "maximize"
function:
formula: "eshop.payments_per_second"
windowing: # the temporal window in which to compute the score of a trial
type: "trim"
trim: ["10s", "0s"] # use the duration of the trial minus 0s from start and end to compute the score of the trial
parametersSelection: "all" # use all available configuration parameters
metricsSelection: "all" # gather all metrics
steps: # the steps to conduct to perform experiments and trials
- name: "my_baseline" # do first a baseline with the provided configuration
type: "baseline"
values:
jvm.maxHeap: 2048
jvm.gcType: "-XX:+UseParallelGC"
- name: my_optimization # then do 20 optimization experiments of 2 trials each
type: optimize
numberOfExperiments: 200
numberOfTrials: 2The following offline study refers to a tuning initiative for a Cassandra-based system (ID 2)
system: 2
name: Optimizing the cassandra - team 2
goal:
objective: minimize
function:
formula: read_response_time_p90
variables:
read_response_time_p90:
metric: read_response_time_p90
labels:
componentName: cassandra
windowing:
type: trim
trim: [5m, 1m]
workflow: cassandra_workflow
parametersSelection:
- name: cassandra_jvm.jvm_maxHeapSize
- name: cassandra.cassandra_concurrentReads
- name: cassandra.cassandra_concurrentWrites
- name: cassandra.cassandra_fileCacheSizeInMb
- name: cassandra.cassandra_memtableCleanupThreshold
- name: cassandra.cassandra_concurrentCompactors
steps:
- name: baseline_step
type: baseline
values:
cassandra_jvm.jvm_maxHeapSize: 1024
cassandra.cassandra_concurrentReads: 32
cassandra.cassandra_concurrentWrites: 32
cassandra.cassandra_fileCacheSizeInMb: 512
cassandra.cassandra_memtableCleanupThreshold: 0.11
cassandra.cassandra_concurrentCompactors: 2
- name: optimization_step
type: optimize
optimizer: CALABI
numberOfExperiments: 50The following offline study is for tuning another Cassandra-based system (ID 3) by acting only on JVM and Linux parameters
system: 3
name: Optimizing a Cassandra NoSQL database version 3 (jvm + os parameters)
goal:
objective: minimize
function:
formula: (x1+x2)/2
variables:
x1:
metric: write_response_time_p90
labels:
componentName: cassandra_team1
x2:
metric: read_response_time_p90
labels:
componentName: cassandra_team1
windowing:
type: trim
trim: [8m,2m]
numberOfTrials: 2
workflow: cassandra_workflow_jvm_os
parametersSelection:
- name: JVM1.jvm_maxHeapSize
- name: JVM1.jvm_newRatio
- name: JVM1.jvm_survivorRatio
- name: JVM1.jvm_maxTenuringThreshold
- name: JVM1.jvm_gcType
- name: JVM1.jvm_concurrentGCThreads
- name: os1.os_cpuSchedMinGranularity
- name: os1.os_cpuSchedWakeupGranularity
- name: os1.os_CPUSchedMigrationCost
- name: os1.os_CPUSchedChildRunsFirst
- name: os1.os_CPUSchedLatency
steps:
- name: baseline_step
type: baseline
values:
JVM_team1.jvm_maxHeapSize: 1024
JVM_team1.jvm_newRatio: 2
JVM_team1.jvm_survivorRatio: 8
JVM_team1.jvm_maxTenuringThreshold: 15
JVM_team1.jvm_gcType: UseConcMarkSweepGC
JVM_team1.jvm_concurrentGCThreads: 8
os_team1.os_cpuSchedMinGranularity: 3000000
os_team1.os_cpuSchedWakeupGranularity: 4000000
os_team1.os_CPUSchedMigrationCost: 500000
os_team1.os_CPUSchedChildRunsFirst: 0
os_team1.os_CPUSchedLatency: 24000000
- name: optimization_sobol
type: optimize
optimizer: SOBOL
numberOfExperiments: 3
- name: optimization_calabi
type: optimize
optimizer: CALABI
numberOfExperiments: 50Last updated
Was this helpful?