Azure Databricks Job parameters

Azure Databricks is a cloud-based data analytics platform that enables you to process large workloads of data.

The following table describes Azure Databricks job parameters:

Parameter

Description

Connection profile

Determines which connection profile is used to connect to the Azure Databricks workspace.

Databricks Job ID

Determines the ID of the job created in your Databricks workspace.

Parameters

Defines task parameters to override when the job runs, according to the Databricks convention. The list of parameters must begin with the name of the parameter type. For example:

  • "notebook_params":{"param1":"val1", "param2":"val2"}
  • "jar_params": ["param1", "param2"]

For more information about the parameter types, review the properties of RunParameters in the OpenAPI specification provided through the Azure Databricks documentation.

For no parameters, specify the following value:

  • "params": {}

Idempotency Token

(Optional) Defines a token to use to rerun job runs that timed out in Databricks.

Values:

  • Control-M-Idem_%%ORDERID — With this token, upon rerun, Control-M invokes the monitoring of the existing job run in Databricks. Default.
  • Any other value — Replaces the Control-M idempotency token. When you rerun a job using a different token, Databricks creates a new job run with a new unique run ID.

Status Polling Frequency

(Optional) Determines the number of seconds to wait before checking the status of the job between intervals.

Default: 30

Parent Topic

Job types