Data Integration Jobs
The following topics describe job attributes that work with Data Integration platforms and services.
AWS Glue Job
The following table describes AWS Glue job attributes.
Attribute |
Description |
---|---|
Connection profile |
Determines the authorization credentials that are used to connect Control-M to AWS Glue. Rules:
|
Glue Job Name |
Defines the AWS Glue job name that you want to run. After you create an AWS Glue pipeline, it is saved as a jobname and you can run it. |
Glue Job Arguments |
Determines whether to add arguments to the AWS Glue job. |
Arguments |
Defines the AWS Glue job runtime parameters Format: {"--myArg1": "myVal1", "--myArg2": "myVal2"} |
Status Polling Frequency |
(Optional) Determines the number of seconds to wait before checking the status of the job between intervals. Default: 15 |
Azure Data Factory Job
Azure Data Factory is a cloud-based ETL and data integration service that allows you to create data-driven workflows to automate the movement and transformation of data.
The following table describes the Azure Data Factory job type attributes.
Attribute |
Description |
---|---|
Connection profile |
Determines the authorization credentials that are used to connect Control-M to Azure Data Factory. Rules:
|
Resource Group Name |
Determines the Azure Resource Group that is associated with a specific data factory. A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group. |
Data Factory Name |
Determines the name of the Azure Data Factory that contains the pipeline you want to run. |
Pipeline Name |
Determines which data pipeline runs when the Control-M job is executed. |
Parameters |
Defines specific parameters in JSON format that are passed when the Data Pipeline runs. Format: {"var1":"value1", "var2":"value2"} |
Status Polling Frequency |
(Optional) Determines the number of seconds to wait before checking the status of the job between intervals. Default: 15 |
Azure Synapse Job
Azure Synapse Analytics enables you to perform data integration and big data analytics.
The following table describes Azure Synapse job parameters:
Attribute |
Description |
---|---|
Connection profile |
Determines the authorization credentials that are used to connect Control-M to Azure Synapse. |
Pipeline Name |
Defines the name of a pipeline that you defined in your Azure Synapse workspace. |
Parameters |
Defines pipeline parameters to override when the job runs, defined in JSON format as pairs of name and value, as follows:: Copy
For no parameters, specify {}. |
Status Polling Interval |
(Optional) Defines the number of seconds to wait before checking the status of the job. Default: 20 seconds |
Boomi AtomSphere Job
The Boomi AtomSphere job enables you to integrate Boomi processes with your existing Control-M workflows.
The following table describes Boomi job parameters:
Parameter |
Description |
---|---|
Connection profile |
Determines the authorization credentials that are used to connect Control-M to Boomi AtomSphere. |
Atom Name |
Defines the name of a Boomi Atom associated with the Boomi process. |
Process Name |
Defines the name of a Boomi process associated with the Boomi Atom. |
Status Polling Frequency |
Determines the number of seconds to wait before checking the status of the job between intervals. Default: 20 seconds |
Tolerance |
Defines the number of API call retries during the status check phase. If the API call that checks the status fails due to the Boomi limitation of a maximum of 5 calls per second, it will retry again according to the number in the Tolerance field. Default: 3 times |
Informatica Job
The Informatica job enables you to automate an Informatica workflow or tasks within the workflow, and define the parameters to pass to the workflow.
The following table describes the Informatica job type attributes.
Attribute |
Description |
---|---|
Connection Profile |
Determines the authorization credentials that are used to connect Control-M to Informatica. Rules:
Variable Name: %%INF-ACCOUNT |
Repository Folder |
Defines the Repository Folder containing the workflow that you want to run. Variable Name: %%INF-REP_FOLDER |
Workflow |
Defines the workflow that you want to run in Control-M for Informatica. Variable Name: %%INF-WORKFLOW |
Instance Name |
Defines the specific instance of the workflow that you want to run. Variable Name: %%INF-INSTANCE_NAME |
OS profile |
Enables you to specify an OS profile when running or rerunning an Informatica job. |
Run Options |
Defines options regarding the workflow task hierarchy. |
Depth |
Determines the number of levels within the workflow task hierarchy that is used to select workflow tasks. Default: 10 Variable Name: %%INF- DEPTH |
Run |
Determines whether to run the whole workflow, start from a specific task, or run a single task as follows:
|
Parameters |
Determines an array of parameters that is passed to the workflow. Each parameter is comprised of the following:
|
Include workflow events log in Job output |
Determines whether to include the workflow event log in the job output |
Include detailed error log for failed sessions |
Determines whether to include a detailed error log for a workflow that failed. |
Get session statistics and log |
Determines whether to retrieve session statistics and log messages. |
Action on rerun |
Determines which operation is executed when the workflow is in a suspended status, as follows:
|
Workflow parameters file |
Defines the path and name of the workflow parameters file. Variable Name: %%INF-WORKFLOW_PARAMETERS_FILE |
Informatica Cloud Services Job
Informatica Cloud Services jobs enable you to automate your Informatica workflows for multi-cloud and on-premises data integration through the Control-M environment.
The following table describes Informatica Cloud Services Job attributes.
Attribute |
Description |
---|---|
Connection Profile |
Determines the authorization credentials that are used to connect Control-M to Informatica Cloud Services. Rules:
Variable Name: %%INF-ACCOUNT |
Task Type |
Determines one of the following task types to run on Informatica Cloud:
|
Task Name |
Defines the name of the task to execute on Informatica Cloud. This parameter is not relevant for a taskflow. |
TaskFlow URL |
Defines the Service URL of the taskflow to execute on Informatica Cloud. In Informatica Data Integration, you can obtain this Service URL through the Properties Detail option of the taskflow. |
Input Fields |
Defines input fields for a taskflow. Format: input1=value1&input2=value2&input3=value3 |
Call Back URL |
(Optional) Defines a publicly available URL to which to post the job status. |
Verification Poll Interval |
Defines the number of seconds between polls for job status verification. |
Talend Data Management Job
The Talend Data Management Job enables the integration of data management and data integration tasks or plans from Talend with your existing Control-M workflows.
The following table describes Talend Data Management job attributes.
Attribute |
Description |
---|---|
Connection profile |
Determines the authorization credentials that are used to connect Control-M to Talend Data Management. Rules:
|
Task/Plan Execution |
Determines one of the following options for execution in Talend:
|
Task Name /
|
Defines the name of the Talend task or plan to execute, as defined in the Tasks and Plans page in the Talend Management Console. |
Parameters |
(For a task) Defines specific parameters to pass when the Talend job runs, defined as JSON pairs of parameter name and value. All parameter names must contain the parameter_ prefix. Format: {"parameter_param1":"value1", "parameter_param2":"value2"} For no parameters, specify {}. |
Log Level |
(For a task) Determines one of the following levels of detail in log messages for the triggered task in the Talend Management Console:
|
Bring logs to output |
(For a task) Determines whether to show Talend log messages in the job output. Values: checked|unchecked Default: unchecked |
Task Polling Intervals / Plan Polling Intervals |
Determines the number of seconds to wait before checking the status of the triggered task or plan. Default: 10 second |