Spark History Server provider

The Spark History Server provider collects metrics from a Spark History Server instance and makes them available to Akamas.


This section provides the minimum requirements that you should match before using the Spark History Server telemetry provider.

Supported versions

  • Apache Spark 2.3

Network requirements

  • Spark History Server API must be reachable at the provided address and port (the default port is 18080).

Supported component types

  • spark-application

You can check Spark History Server provider metrics mapping to see how component-type metrics are extracted by this provider.

Akamas supported version

  • Versions < 2.0.0 are compatible with Akamas until version 1.8.0

  • Versions >= 2.0.0 are compatible with Akamas from version 1.9.0

Workflow requirements

This section lists the workflow operators this provider depends on:

Components configuration

Akamas uses components to identify specific elements of the system to be monitored and optimized. Your system might contain multiple components to model, for example, a Spark application and each host of the cluster. To point Akamas to the right component when extracting metrics you need to add a property called sparkApplication to your Spark Application component. The provider will only extract metrics for components for which this property has been specified.

name: My Application
  sparkApplication: "true"

Last updated