Data Integration Jobs

The following topics describe job attributes that work with Data Integration platforms and services.

AWS Glue Job

The following table describes AWS Glue job attributes.

Attribute

Description

Connection profile

Determines the authorization credentials that are used to connect Control-M to AWS Glue.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Spaces

Glue Job Name

Defines the AWS Glue job name that you want to run. After you create an AWS Glue pipeline, it is saved as a jobname and you can run it.

Glue Job Arguments

Determines whether to add arguments to the AWS Glue job.

Arguments

Defines the AWS Glue job runtime parameters

Format: {"--myArg1": "myVal1", "--myArg2": "myVal2"}

Status Polling Frequency

(Optional) Determines the number of seconds to wait before checking the status of the job between intervals.

Default: 15

Azure Data Factory Job

Azure Data Factory is a cloud-based ETL and data integration service that allows you to create data-driven workflows to automate the movement and transformation of data.

The following table describes the Azure Data Factory job type attributes.

Attribute

Description

Connection profile

Determines the authorization credentials that are used to connect Control-M to Azure Data Factory.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Spaces

  • Variable Name: %%AZURE-ACCOUNT

Resource Group Name

Determines the Azure Resource Group that is associated with a specific data factory. A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group.

Data Factory Name

Determines the name of the Azure Data Factory that contains the pipeline you want to run.

Pipeline Name

Determines which data pipeline runs when the Control-M job is executed.

Parameters

Defines specific parameters in JSON format that are passed when the Data Pipeline runs.

Format: {"var1":"value1", "var2":"value2"}

Status Polling Frequency

(Optional) Determines the number of seconds to wait before checking the status of the job between intervals.

Default: 15

Azure Synapse Job

Azure Synapse Analytics enables you to perform data integration and big data analytics.

The following table describes Azure Synapse job parameters:

Attribute

Description

Connection profile

Determines the authorization credentials that are used to connect Control-M to Azure Synapse.

Pipeline Name

Defines the name of a pipeline that you defined in your Azure Synapse workspace.

Parameters

Defines pipeline parameters to override when the job runs, defined in JSON format as pairs of name and value, as follows::

Copy
 {"param1":"value1", "param2":"value2"}

For no parameters, specify {}.

Status Polling Interval

(Optional) Defines the number of seconds to wait before checking the status of the job.

Default: 20 seconds

Boomi AtomSphere Job

The Boomi AtomSphere job enables you to integrate Boomi processes with your existing Control-M workflows.

The following table describes Boomi job parameters:

Parameter

Description

Connection profile

Determines the authorization credentials that are used to connect Control-M to Boomi AtomSphere.

Atom Name

Defines the name of a Boomi Atom associated with the Boomi process.

Process Name

Defines the name of a Boomi process associated with the Boomi Atom.

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the job between intervals.

Default: 20 seconds

Tolerance

Defines the number of API call retries during the status check phase. If the API call that checks the status fails due to the Boomi limitation of a maximum of 5 calls per second, it will retry again according to the number in the Tolerance field.

Default: 3 times

Informatica Job

The Informatica job enables you to automate an Informatica workflow or tasks within the workflow, and define the parameters to pass to the workflow.

The following table describes the Informatica job type attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to Informatica.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Spaces

Variable Name: %%INF-ACCOUNT

Repository Folder

Defines the Repository Folder containing the workflow that you want to run.

Variable Name: %%INF-REP_FOLDER

Workflow

Defines the workflow that you want to run in Control-M for Informatica.

Variable Name: %%INF-WORKFLOW

Instance Name

Defines the specific instance of the workflow that you want to run.

Variable Name: %%INF-INSTANCE_NAME

OS profile

Enables you to specify an OS profile when running or rerunning an Informatica job.

Run Options

Defines options regarding the workflow task hierarchy.

Depth

Determines the number of levels within the workflow task hierarchy that is used to select workflow tasks.

Default: 10

Variable Name: %%INF- DEPTH

Run

Determines whether to run the whole workflow, start from a specific task, or run a single task as follows:

  • Run the whole workflow: Runs the whole workflow.

  • Start from task: Starts the workflow from the task that you specify.

    Variable Name: %%INF- START_FROM_TASK

  • Run single task: Runs the task that you specify.

    Variable Name: %%INF-RUN_SINGLE_TASK

Parameters

Determines an array of parameters that is passed to the workflow.

Each parameter is comprised of the following:

  • Scope: Defines the scope of the parameter in an array definition.

  • Name: Defines the parameter name in an array definition.

  • Value: Defines the parameter value in an array definition.

Include workflow events log in Job output

Determines whether to include the workflow event log in the job outputClosedA tab in the job properties pane in the Monitoring domain that shows the output of a job, which indicates whether a job ended OK, and used, for example, with jobs that check file location.

Include detailed error log for failed sessions

Determines whether to include a detailed error log for a workflow that failed.

Get session statistics and log

Determines whether to retrieve session statistics and log messages.

Action on rerun

Determines which operation is executed when the workflow is in a suspended status, as follows:

  • Recover: Restarts suspended workflow from point of failure

  • Force restart: Restarts a suspended workflow from the beginning.

  • Force restart from a specific task: Restarts the suspended workflow that you define

    Variable Name: %%INF- RESTART_FROM_TASK

Workflow parameters file

Defines the path and name of the workflow parameters file.

Variable Name: %%INF-WORKFLOW_PARAMETERS_FILE

Informatica Cloud Services Job

Informatica Cloud Services jobs enable you to automate your Informatica workflows for multi-cloud and on-premises data integration through the Control-M environment.

The following table describes Informatica Cloud Services Job attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to Informatica Cloud Services.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Spaces

Variable Name: %%INF-ACCOUNT

Task Type

Determines one of the following task types to run on Informatica Cloud:

  • Mapping Task

  • Masking Task

  • PowerCenter Task

  • Replication Task

  • Synchronization Task

  • Linear Taskflow

  • Taskflow

Task Name

Defines the name of the task to execute on Informatica Cloud.

This parameter is not relevant for a taskflow.

TaskFlow URL

Defines the Service URL of the taskflow to execute on Informatica Cloud.

In Informatica Data Integration, you can obtain this Service URL through the Properties Detail option of the taskflow.

Input Fields

Defines input fields for a taskflow.

Format: input1=value1&input2=value2&input3=value3

Call Back URL

(Optional) Defines a publicly available URL to which to post the job status.

Verification Poll Interval

Defines the number of seconds between polls for job status verification.

Talend Data Management Job

The Talend Data Management Job enables the integration of data management and data integration tasks or plans from Talend with your existing Control-M workflows.

The following table describes Talend Data Management job attributes.

Attribute

Description

Connection profile

Determines the authorization credentials that are used to connect Control-M to Talend Data Management.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Spaces

Task/Plan Execution

Determines one of the following options for execution in Talend:

  • Execute Task

  • Execute Plan

Task Name /
Plan Name

Defines the name of the Talend task or plan to execute, as defined in the Tasks and Plans page in the Talend Management Console.

Parameters

(For a task) Defines specific parameters to pass when the Talend job runs, defined as JSON pairs of parameter name and value. All parameter names must contain the parameter_ prefix.

Format: {"parameter_param1":"value1", "parameter_param2":"value2"}

For no parameters, specify {}.

Log Level

(For a task) Determines one of the following levels of detail in log messages for the triggered task in the Talend Management Console:

  • Information — All logs available

  • Warning — Only warning logs

  • Error — Only Error logs

  • Off — No logs

Bring logs to output

(For a task) Determines whether to show Talend log messages in the job output.

Values: checked|unchecked

Default: unchecked

Task Polling Intervals /

Plan Polling Intervals

Determines the number of seconds to wait before checking the status of the triggered task or plan.

Default: 10 second