Offline Study
Last updated
Last updated
Offline optimization studies are optimization studies where the workload is simulated by leveraging a load-testing tool.
Offline optimization studies are typically used to optimize systems in pre-production environments, with respect to planned and what-if scenarios that cannot be directly run in production. Scenarios include new application releases, planned technology changes (e.g. new JVM or DB), cloud migration or new provider, expected workload growth, and resilience under failure scenarios (from chaos engineering).
The following figure represents the iterative process associated with offline optimizations:
The following 5 phases can be identified for each iteration (also known as experiment):
Apply configuration: Akamas applies the parameter configuration (one or more parameters) to the target system by leveraging a set of workflow operators
Apply workload: Akamas triggers a workload on the target system by also leveraging a set of workflow operators
Collect KPIs: Akamas collects the metrics related to the target system - only those metrics that are specified by each telemetry instance defined in the system
Score vs goal: Akamas scores the applied parameter configuration against the defined goal and constraints - the score is the value of the goal function
Recommend Conf: Akamas AI engine identifies the configuration for the next iteration until a termination condition for the study is met (e.g. number of experiments).
Thanks to its patented AI (reinforcement learning) algorithms, Akamas can find the optimal configuration without having to explore all the possible configurations.
For each experiment, Akamas allows multiple trials to be executed. A trial is a repetition of the same experiment to reduce the impact of noise on the result of an experiment.
Environments can be noisy for several reasons such as:
External conditions (e.g. background jobs, "noisy neighbors" in the cloud)
Measurement errors (e.g. monitoring tools not always 100% accurate)
This approach is consistent with scientific and engineering practices, where the strategy to minimize the impact of noise is to repeat the same experiment multiple times.
An offline optimization study can include multiple steps.
Typically there are at least two steps:
Baseline step: a single experiment that is run by applying the already deployed configuration before the Akamas optimization is applied - the results of this experiment are used as a reference (baseline) for assessing the optimization and as such is a mandatory step for each study
Optimize step: a defined number of experiments used to identify the optimal configuration by leveraging Akamas AI.
Other steps are:
Bootstrap step: imported experiments from other optimization studies
Preset step: a single experiment with a defined configuration
The steps to be executed can be specified when defining an offline optimization study.
An offline optimization study is an Akamas resource that can be managed via CLI using the resource management commands.
The Akamas UI shows offline optimization studies in a specific top-level menu.
The details and results of an offline optimization study are displayed when drilling down (there are multiple tabs and sections).
