Comment on page
Spark optimization pack
The Spark optimization pack allows tuning applications running on the Apache Spark framework. Through this optimization pack, Akamas is able to explore the space of the Spark parameters in order to find the configurations that best optimize the allocated resources or the execution time.
To achieve these goals the optimization pack provides parameters that focus on the following areas:
- Driver and executors' resources allocation
- Parallelism
- Shuffling
- Spark SQL
Similarly, the bundled metrics provide visibility on the following statistics from the Spark History Server:
- Execution time
- Executors' resource usage
- Garbage collection time
Component Type | Description |
|---|---|
Spark Application 2.2.0 | |
Spark Application 2.3.0 | |
Spark Application 2.4.0 |
Here’s the command to install the Spark optimization pack using the Akamas CLI:
akamas install optimization-pack Spark
Last modified 4mo ago