Application Workflow Jobs
The following topics describe job attributes that work with application workflow platforms and services:
Airflow Job
Airflow enables you to monitor and manage DAG workflows in Control-M. You can monitor DAG executions in the Airflow tab in the Monitoring domain. You can also view the specific details of each task, open the DAG in the Airflow web server user interface, and view XCom variables from the Airflow tab.
To create an Airflow job, see Creating a Job. For more information about this plug-in, see Control-M for Airflow.
The following table describes the Airflow job type attributes.
Attribute |
Description |
---|---|
Connection Profile |
Determines the authorization credentials that are used to connect Control-M to Airflow, as described in Airflow Connection Profile Parameters . Rules:
|
DAG ID |
Defines the unique identifier of a DAG.
|
Configuration JSON |
(Optional) Defines the JSON object, which describes additional configuration parameters. |
Output Details |
Determines whether to include Airflow DAG task logs in the Control-M job output, as follows:
You can also view all task logs from the Airflow tab in an Airflow job in the Monitoring domain. In addition, you can view the log of each Airflow task execution represented by the Try Number field. If a task executed 3 times during the DAG run, the Try Number shows three options. |
Apache NiFi Job
Apache NiFi is an open-source tool that automates data flow across systems in real time.
To create an Apache NiFi job, see Creating a Job. For more information about this plug-in, see Control-M for Apache NiFi.
The following table describes the Apache NiFi job attributes.
Attribute |
Description |
---|---|
Connection Profile |
Determines the authorization credentials that are used to connect Control-M to Apache NiFi, as described in Apache NiFi Connection Profile Parameters. Rules:
|
Processor Group ID | Defines the ID number of a specific processor group. |
Processor ID | Defines the ID number of a specific processor. |
Action |
Determines one of the following actions to perform on Apache NiFi:
|
Disconnected Node Ack | Determines whether to disconnect the node to allow mutable requests to proceed. |
Status Polling Frequency |
Determines the number of seconds to wait before checking the job status. Default: 5 |
Failure Tolerance |
Determines the number of times the job tries to run before ending Not OK. Default: 0 |
Astronomer Job
Astronomer is a workload automation service based on Apache Airflow that enables you to create, schedule, and manage your workflows.
To create an Astronomer job, see Creating a Job. For more information about this plug-in, see Control-M for Astronomer.
The following table describes the Astronomer job attributes.
Attribute |
Description |
---|---|
Connection Profile |
Determines the authorization credentials that are used to connect Control-M to Astronomer, as described in Astronomer Connection Profile Parameters. Rules:
|
DAG Name |
Defines the logical name of the Directed Acyclic Graph (DAG), as defined in the Airflow interface. |
DAG Run ID |
(Optional) Defines the specific DAG run (execution) ID in Airflow. |
Parameters |
Defines the JSON-based body parameters to pass when the DAG executes, in the following format: Copy
|
Status Polling Frequency |
Determines the number of seconds to wait before checking the job status. Default: 60 |
Failure Tolerance |
Determines the number of times the job tries to run before ending Not OK. Default: 3 |
AWS Step Functions Job
AWS Step Functions enables you to create visual workflows that can integrate other AWS services.
To create an AWS Step Functions job, see Creating a Job. For more information about this plug-in, see Control-M for AWS Step Functions.
The following table describes the AWS Step Functions job attributes.
Attribute |
Description |
---|---|
Connection Profile |
Determines the authorization credentials that are used to connect Control-M to AWS Step Functions, as described in AWS Step Functions Connection Profile Parameters. Rules:
|
Execution Name |
Defines the name of the Step Function execution. |
State Machine ARN |
Determines the Step Function state machine to use. A state machine is a workflow, and an Amazon Resource Name (ARN) is a standardized AWS resource address. arn:aws:states:us-east-1:155535555553:stateMachine:MyStateMachine |
Parameters |
Defines the parameters for the Step Function job, in JSON format, which enables you to control how the job executes. Use backslashes to escape quotes. Copy
If you are not adding parameters, type {}. |
Show Execution Logs |
Determines whether to append the log to the output A tab in the job properties pane of the Monitoring domain where the job output appears that indicates whether a job ended OK, and is used, for example, with jobs that check file location.. |
Status Polling Frequency |
Determines the number of seconds to wait before checking the job status. Default: 20 |
Failure Tolerance |
Determines the number of times to check the job status before ending Not OK. Default: 2 |
Azure Logic Apps Job
Azure Logic Apps enables you to design and automate cloud-based workflows and integrations.
To create an Azure Logic Apps job, see Creating a Job. For more information about this plug-in, see
The following table describes the Azure Logic Apps job attributes.
Attribute |
Description |
---|---|
Connection Profile |
Determines the authorization credentials that are used to connect Control-M to Azure Logic Apps, as described in Azure Logic Apps Connection Profile Parameters. Rules:
|
Workflow |
Determines which of the Consumption logic app workflows executes from your predefined set of workflows. This job does not execute Standard logic app workflows. |
Parameters |
Defines parameters, in JSON format, that enable you to control the presentation of data. Copy
Rules:
|
Get Logs |
Determines whether to display the job output when the job ends. |
Failure Tolerance |
Determines the number of times to check the job status before ending Not OK. Default: 2 |
Status Polling Frequency |
Determines the number of seconds to wait before checking the job status. Default: 20 |
GCP Composer Job
Google Cloud (GCP) Composer is a managed workflow orchestration service built on Apache Airflow that enables you to automate workflow tasks.
To create a GCP Composer job, see Creating a Job. For more information about this plug-in, see Control-M for GCP Composer.
The following table describes GCP Composer job attributes.
Attribute |
Description |
---|---|
Connection Profile |
Determines the authorization credentials that are used to connect Control-M to GCP Composer, as described in . Rules:
|
DAG Name |
Defines the DAG logical name as defined in the GCP interface. |
DAG Run ID |
(Optional) Defines the specific DAG run (execution) ID in GCP Composer. |
Parameters JSON Input |
Defines the JSON-based body parameters to pass when the DAG executes, in the following format: Copy
|
Status Polling Frequency |
Determines the number of seconds to wait before checking the job status. Default: 60 |
Failure Tolerance |
Determines the number of times to check the job status before ending Not OK. Default: 2 |
GCP Workflows Job
GCP Workflows enables you to design and automate cloud-based workflows and integrations.
To create a GCP Workflows job, see Creating a Job. For more information about this plug-in, see
The following table describes GCP Workflows job attributes.
Attribute |
Description |
---|---|
Connection Profile |
Determines the authorization credentials that are used to connect Control-M to GCP Workflows, as described in GCP Workflows Connection Profile Parameters. Rules:
|
Project ID |
Defines the GCP project ID where the batch job executes. A project is a set of configuration settings that define the resources your GCP Workflows jobs use and how they interact with GCP. |
Location |
Defines the region where the GCP Workflow job executes. us-central1 |
Workflow Name |
Determines the predefined GCP Workflow that executes. |
Parameters |
Defines the JSON-based body parameters that are passed to the function, in the following format: Copy
|
Execution Label |
Defines a job execution label, which enables you to group similar executions in the GCP Workflows log. {"labelName": "name"} |
Show Workflow Results |
Determines whether the GCP Workflow results appear in the job output. |
Status Polling Frequency |
Determines the number of seconds to wait before checking the job status. Default: 20 |
Failure Tolerance |
Determines the number of times to check the job status before ending Not OK. Default: 3 |