Skip to content

Enable metrics for Apache Spark OLAC

Prerequisite

Make sure Privacera Monitoring is setup and OpenTelemetry Collector is configured before enabling metrics.

Refer Privacera Monitoring for more details.

  1. Navigate to the ~/privacera/privacera-manager/config/custom-vars folder and edit the vars.spark-standalone.yml file.

    Bash
    1
2
    cd ~/privacera/privacera-manager/config/custom-vars
vi vars.spark-standalone.yml

  2. Add below content and save the vars.spark-standalone.yml file to enable metrics.

    Bash
    1
    STANDALONE_SPARK_PLUGIN_METRICS_ENABLE: "true"

    JVM Metrics

    The JVM Metrics will be enabled once the Spark Plugin Metrics are enabled by-default. To disable the metrics add the following property

    Bash
    STANDALONE_SPARK_PLUGIN_METRICS_JVM_ENABLE: "false"

  3. Run the following commands to update your Privacera Manager Platform instance with the update property

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  4. Once the post-install process is complete, proceed with the steps given in Setup for Access Management for Apache Spark

Add the keys under Custom Properties while configuring the connector. See Configure the Apache Spark OLAC connector.

Key Value
privacera.spark.plugin.metrics.enable true
privacera.spark.plugin.metrics.jvm.enable true
privacera.spark.plugin.metrics.otel.exporter.enable true
otel.exporter.otlp.endpoint OpenTelemetry Collector endpoint, for example http://otel.monitoring.svc.cluster.local:4318