Spark optimization pack
The Spark optimization pack allows tuning applications running on the Apache Spark framework. Through this optimization pack, Akamas is able to explore the space of the Spark parameters in order to find the configurations that best optimize the allocated resources or the execution time.
To achieve these goals the optimization pack provides parameters that focus on the following areas:
Driver and executors' resources allocation
Parallelism
Shuffling
Spark SQL
Similarly, the bundled metrics provide visibility on the following statistics from the Spark History Server:
Execution time
Executors' resource usage
Garbage collection time
Component Types
| Component Type | Description |
|---|---|
Spark Application 2.2.0 | |
Spark Application 2.3.0 | |
Spark Application 2.4.0 |
Installing
Here’s the command to install the Spark optimization pack using the Akamas CLI:
Last updated