Comment on page
Spark History Server provider
The Spark History Server provider collects metrics from a Spark History Server instance and makes them available to Akamas.
Prerequisites
This section provides the minimum requirements that you should match before using the Spark History Server telemetry provider.
- Apache Spark 2.3
- Spark History Server API must be reachable at the provided address and port (the default port is
18080).
- spark-application
You can check Spark History Server provider metrics mapping to see how component-types metrics are extracted by this provider.
- Versions < 2.0.0 are compatible with Akamas until version 1.8.0
- Versions >= 2.0.0 are compatible with Akamas from version 1.9.0
This section lists the workflow operators this provider depends on:
Akamas uses components to identify specific elements of the system to be monitored and optimized. Your system might contain multiple components to model, for example, a Spark application and each host of the cluster. To point Akamas to the right component when extracting metrics you need to add a property called
sparkApplication to your Spark Application component. The provider will only extract metrics for components for which this property has been specified.name: My Application
properties:
sparkApplication: "true"
Last modified 4mo ago