# Study template Optimization studies are defined using a YAML manifest with the following structure: ```yaml system: 1 name: Optimizing the e-shop application goal: objective: maximize function: formula: payments_per_sec variables: payments_per_sec: metric: eshop_payments labels: componentName: eshop workflow: eshop_jmeter_test steps: - name: baseline type: baseline values: tomcat.maxThreads: 1024 jvm.maxHeap: 2048 jvm.garbageCollectorType: G1GC postgres.shared_buffers: 4096 ``` with the following mandatory properties: | Field | Type | Value restrictions | Is required | Default Value | Description | | --------------------- | ---------------- | ------------------- | ----------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `system` | object reference | | TRUE | | The system the study refers to | | `name` | string | | TRUE | | The name of the study | | `goal` | object | | TRUE | | The goal and constraint description - see [Goal & Constraints](https://docs.akamas.io/akamas-docs/3.6/reference/construct-templates/study-template/goal-and-constraints) | | `kpis` | list | | FALSE | | The KPIs description - see [KPI](https://docs.akamas.io/akamas-docs/3.6/reference/construct-templates/study-template/kpis) | | `numberOfTrials` | integer | | FALSE | 1 | The number of trials for each experiment - see below | | `trialAggregation` | string | `MAX`, `MIN`, `AVG` | FALSE | `AVG` | The aggregation used to calculate the score across multiple trials - see below | | `parametersSelection` | list | | FALSE | `all` | The list of parameters to be tuned - see [Parameter selection](https://docs.akamas.io/akamas-docs/3.6/reference/construct-templates/study-template/parameter-selection) | | `metricsSelection` | list | | FALSE | `all` | The list of metrics - see [Metric selection](https://docs.akamas.io/akamas-docs/3.6/reference/construct-templates/study-template/metric-selection) | | `requireApproval` | boolean | | FALSE | `FALSE` unless a `workloadsSelection` is provided | When enabled, new experiments will require the user to approve the configuration before executing the trials | | `workloadsSelection` | object array | | FALSE | | The list of defined workloads - this only applies to live optimization studies - see [Workload Selection](https://docs.akamas.io/akamas-docs/3.6/reference/construct-templates/study-template/workload-selection) | | `windowing` | string | | FALSE | `trim` | The windowing strategy - this only applies to offline optimization studies - see [Windowing strategy](https://docs.akamas.io/akamas-docs/3.6/reference/construct-templates/study-template/windowing-strategy) | | `workflow` | object reference | | TRUE | | The workflow the study refers to | | `steps` | list | | TRUE | | The description of the steps - see [Steps](https://docs.akamas.io/akamas-docs/3.6/reference/construct-templates/study-template/steps) | Some of these optional properties depend on whether the study is an offline or live optimization study. #### Number of trials It is possible to perform more than one trial per experiment to validate the score of a configuration under test, e.g., to consider noisy environments. The following fragment of the YAML definition of a study sets the number of trials to 3: ```yaml numberOfTrials: 3 ``` Notice: This is a global property of the study which can be overwritten for each step. #### Trial aggregation The trial aggregation policy defines how trial scores are aggregated to form experiment scores. There are three different types of strategies to aggregate trial scores: * **AVG:** the score of an experiment is the average of the scores of its trials - this is the default * **MIN:** the score of an experiment is the minimum among the scores of its trials * **MAX:** the score of an experiment is the maximum among the scores of its trial The following fragment of the YAML definition of a study sets the trial aggregation to MAX: ```yaml trialAggregation: MAX # Other possible values are AVG, MIN ``` ### Examples The following system refers to an offline optimization study for a system modeling an e-commerce service, where a windowing strategy is specified: ```yaml system: "bde4f259-9a51-4c67-87aa-3c5bc599c6b9" # id of the system to optimize with the actions defined in this study workflow: "eshop_jmeter_test" # name of the workflow to use to perform trials name: Optimizing the e-shop application # name of the study goal: # the performance goal to achieve objective: "maximize" function: formula: "eshop.payments_per_second" windowing: # the temporal window in which to compute the score of a trial type: "trim" trim: ["10s", "0s"] # use the duration of the trial minus 0s from start and end to compute the score of the trial parametersSelection: "all" # use all available configuration parameters metricsSelection: "all" # gather all metrics steps: # the steps to conduct to perform experiments and trials - name: "my_baseline" # do first a baseline with the provided configuration type: "baseline" values: jvm.maxHeap: 2048 jvm.gcType: "-XX:+UseParallelGC" - name: my_optimization # then do 20 optimization experiments of 2 trials each type: optimize numberOfExperiments: 200 numberOfTrials: 2 ``` The following offline study refers to a tuning initiative for a Cassandra-based system (ID 2) ```yaml system: 2 name: Optimizing the cassandra - team 2 goal: objective: minimize function: formula: read_response_time_p90 variables: read_response_time_p90: metric: read_response_time_p90 labels: componentName: cassandra windowing: type: trim trim: [5m, 1m] workflow: cassandra_workflow parametersSelection: - name: cassandra_jvm.jvm_maxHeapSize - name: cassandra.cassandra_concurrentReads - name: cassandra.cassandra_concurrentWrites - name: cassandra.cassandra_fileCacheSizeInMb - name: cassandra.cassandra_memtableCleanupThreshold - name: cassandra.cassandra_concurrentCompactors steps: - name: baseline_step type: baseline values: cassandra_jvm.jvm_maxHeapSize: 1024 cassandra.cassandra_concurrentReads: 32 cassandra.cassandra_concurrentWrites: 32 cassandra.cassandra_fileCacheSizeInMb: 512 cassandra.cassandra_memtableCleanupThreshold: 0.11 cassandra.cassandra_concurrentCompactors: 2 - name: optimization_step type: optimize numberOfExperiments: 50 ``` The following offline study is for tuning another Cassandra-based system (ID 3) by acting only on JVM and Linux parameters ```yaml system: 3 name: Optimizing a Cassandra NoSQL database version 3 (jvm + os parameters) goal: objective: minimize function: formula: (x1+x2)/2 variables: x1: metric: write_response_time_p90 labels: componentName: cassandra_team1 x2: metric: read_response_time_p90 labels: componentName: cassandra_team1 windowing: type: trim trim: [8m,2m] numberOfTrials: 2 workflow: cassandra_workflow_jvm_os parametersSelection: - name: JVM1.jvm_maxHeapSize - name: JVM1.jvm_newRatio - name: JVM1.jvm_survivorRatio - name: JVM1.jvm_maxTenuringThreshold - name: JVM1.jvm_gcType - name: JVM1.jvm_concurrentGCThreads - name: os1.os_cpuSchedMinGranularity - name: os1.os_cpuSchedWakeupGranularity - name: os1.os_CPUSchedMigrationCost - name: os1.os_CPUSchedChildRunsFirst - name: os1.os_CPUSchedLatency steps: - name: baseline_step type: baseline values: JVM_team1.jvm_maxHeapSize: 1024 JVM_team1.jvm_newRatio: 2 JVM_team1.jvm_survivorRatio: 8 JVM_team1.jvm_maxTenuringThreshold: 15 JVM_team1.jvm_gcType: UseConcMarkSweepGC JVM_team1.jvm_concurrentGCThreads: 8 os_team1.os_cpuSchedMinGranularity: 3000000 os_team1.os_cpuSchedWakeupGranularity: 4000000 os_team1.os_CPUSchedMigrationCost: 500000 os_team1.os_CPUSchedChildRunsFirst: 0 os_team1.os_CPUSchedLatency: 24000000 - name: optimization_sobol type: optimize numberOfExperiments: 50 ```