Cloud Computing Jobs

The following topics describe job attributes that work with cloud computing platforms and services:

AWS Batch Job

AWS Batch enables you to manage and run batch computing workloads in AWS.

The following table describes the AWS Batch job attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to AWS Batch.

Rules:

  • Characters: 1−30

  • Case Sensitive: Yes

  • Invalid Characters: Blank spaces.

Use Advanced JSON Format

Determines whether you supply your own JSON parameters.

JSON Format

Defines the parameters for the batch job, in JSON format, that enable you to control how the job runs.

For a description of this JSON syntax, see the description of SubmitJob in the AWS Batch API Reference.

Copy
{
   "containerOverrides":
   {
      "command":["echo","Hello, from Control-M"],
         "resourceRequirements":[
         {
            "type":"VCPU",
            "value":"2"
         }
      ]
   }
   "jobDefinition": "ctm-batch-job-definition",
   "jobName": "ctm-batch-job-single",
   "jobQueue": "ctm-batch-job-queue",
   "timeout"
   {
      "attemptDurationSeconds": 70
   }
}

Job Name

Defines the name of the batch job.

Job Definition and Revision

Determines which predefined job definition and version number (revision) is applied to the job, depending upon how you complete the field, as follows:

  • Job Definition Name: When you type only the job definition name in the field, this job definition and its most recent version is applied to the job.

  • Job Definition Name:Version Number: When you type the job definition name—followed by a colon (:), followed by a version number—in the field, this job definition and version number is applied to the job.

ctm-batch-job-definition:3

Job Queue

Determines the job queue, which stores your batch job.

Container Overrides Command

(Optional) Defines a command, in JSON format, that overrides the specified command in the job definition.

Job Attempts

(Optional) Determines the number of times to retry a job run, which overrides the retry attempts determined in the job definition.

Valid Values: 1–10

Execution Timeout

(Optional) Determines the number of seconds to wait before a timeout occurs in a batch job, which overrides the specified timeouts in the job definition.

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the AWS Batch job.

Set to 120 seconds or longer for jobs that run for more than an hour.

Default: 20

AWS EC2 Job

AWS Elastic Compute Cloud (EC2) is a cloud computing platform that enables you to create virtual machines.

The following table describes the AWS EC2 job type attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to AWS EC2.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Operations

Determines one of the following operations to perform on the AWS EC2 Virtual Machine:

  • Create: Creates a new virtual machine.

  • Create from Instance Template: Creates a new virtual machine based on a template.

  • Start: Starts an existing virtual machine.

  • Stop: Stops a running virtual machine.

  • Reboot: Resets a virtual machine.

  • Terminate: Deletes an existing virtual machine.

Launch Template ID

Defines the template to use to create a VM from a template.

Instance ID

Defines the name of the VM instance where you want to run the operation.

This parameter is available for all operations except for the Create operations.

Instance Name

Defines the name of a new VM instance for Create operations.

Placement Availability Zone

Determines which AWS EC2 zone to use for a Create operation.

Instance Type

Determines the software requirements of the host computer when you create a new AWS EC2 Virtual Machine.

Subnet ID

Defines the Subnet ID that is required to launch the instance in a Create operation.

Key Name

Defines the security credential key set for a Create operation.

Image ID

Defines the ID of the Amazon Machine Image (AMI) that is required to launch the instance in a Create operation.

Number of copies

Determines the number of copies of the VM to create in a Create operation.

Default: 1

Get Instance logs

Determines whether to display logs from the AWS EC2 instance at the end of the job output.

This parameter is available for all operations except for the Terminate operation.

Verification Poll Interval

Determines the number of seconds to wait before job status verification.

Set to 120 seconds or longer for jobs that run for more than an hour.

Default: 15 seconds

Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2 times

AWS ECS Job

AWS Elastic Container Service (ECS) is a container management service that enables you to run, stop, manage, and monitor containerized applications in a cluster.

The following table describes the AWS ECS job type attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to AWS ECS.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Action

Determines one of the following actions to perform on AWS ECS:

  • Preset JSON: Runs the task according to the attributes that you define in this job.

  • Manual JSON: Runs the task according to the JSON parameters that you supply in the Parameters attribute.

Parameters

Defines the parameters, in JSON format, that enable you to control how the AWS ECS task runs.

For a description of this JSON syntax, see RunTask in the AWS ECS API Reference.

Copy
"Parameters"
{
   "cluster":"ECSIntegrationCluster",
   "launchType":"FARGATE",
   "networkConfiguration":
   {
      "awsvpcConfiguration":
      {
         "assignPublicIp":"ENABLED",
         "securityGroups":["sg-01e4a5bfac4189d10"],
         "subnets":["subnet-045ddaf41d4852fd7"]
      }
   },
   "overrides":
   {
         "containerOverrides":[
         {
            "command":["/bin/sh -c 'whoami'"],
            "environment":[
            {
               "name":"var1",
               "value":"hello"
            }],
            "name":"IntegrationURI"
            }]
   },
   "taskDefinition":"ECSIntegrationTask"
}

ECS Cluster Name

Defines the ECS cluster on the AWS ECS platform where the job runs.

An ECS cluster is a logical group of tasks and services.

ECS Task Definition

Defines the task definition on the AWS ECS platform.

The task definition describes the container image, command, environment variables, and other parameters that run your application.

Launch Type

Determines the type infrastructure where your tasks and services run:

  • Fargate: Runs your application on a virtual machine that is automatically provisioned and managed.

  • EC2: Runs your application on a predefined AWS EC2 virtual machine that you provision and manage.

  • External: Runs your application on a different virtual machine that you define in the AWS ECS platform.

Assign Public IP

Determines whether the job has a public internet protocol (IP).

  • Yes

  • No

Network Security Groups

Defines which network security group your task is connected to, through the elastic network interface, which is a virtual network card that controls inbound and outbound traffic.

Network Subnets

Defines the virtual subnet, which determines the IP addresses for the task.

Override Container

Defines which override container to use that overrides the default container image, command, or other settings specified in the task definition.

Override Command

Defines the command to run in the container that overrides any command specified in the task definition.

Environment Variables

Defines the environment variables for the container, which are used to manage the container and pass information to the application that runs inside it.

Logs

Determines whether the logs from the AWS ECS platform appear at the end of the Control-M job output.

  • Get Logs

  • Don't Get Logs

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the job.

Set to 120 seconds or longer for jobs that run for more than an hour.

Default: 10

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 5

Azure Batch Accounts Job

Azure Batch Accounts jobs enable you to run large-scale, computer-intensive tasks.

The following table describes the Azure Batch Accounts job attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to Azure Batch Accounts.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Batch Job ID

Defines the name of the job that is created in Azure Portal.

Task ID Prefix

Defines a prefix that is added to the task ID.

Task Command Line

Defines the commands that the job performs.

Max Wall Clock Time

Determines the maximum amount of time the job runs, as follows:

  • Unlimited

  • Custom

Variable name: %%AWS-BATCH_ WALLCLOCK

Max Wall Time Digits

Defines the Custom maximum number of seconds, minutes, hours, or days the job runs.

Default: 1

Max Wall Time Unit

Determines the Custom units used to define the maximum amount of time the job runs.

  • Seconds

  • Minutes

  • Hours

  • Days

Default: Minutes

Max Task Retry Count

Determines the maximum number of times to rerun a failed job, as follows:

  • None

  • Unlimited

  • Custom

Default: None

Retry Number

Determines the Custom maximum number of times to rerun a failed job.

Default: 1

Retention Time

Determines the file retention period, as follows:

  • Unlimited: Retains the job directory and its files indefinitely.
  • Custom: Determines the number of days, hours, minutes, or seconds to retain the job directory and its files.

Default: Unlimited

Retention Time Digits

Determines the number of seconds, minutes, hours, or days to set the Custom file retention period for.

Default: 1

Retention Time Unit

Determines the unit of time used to set the Custom file retention period for.

  • Seconds

  • Minutes

  • Hours

  • Days

Default: Hours

Append Log to Output

Determines whether to add the job log to the outputClosedA tab in the job properties pane in the Monitoring domain that shows the output of a job, which indicates whether a job ended OK, and used, for example, with jobs that check file location.

Status Polling Interval

Determines the number of seconds to wait before checking the status of the Azure Batch Accounts job.

Set to 120 seconds or longer for jobs that run for more than an hour.

Default: 20

Azure Functions Job

Azure Functions is a cloud service for application development.

The following table describes Azure Functions job attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to Azure Functions.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Variable name: %%AZURE-ACCOUNT

Function App

Defines the name of the Azure function application that you want to run.

Function Name

Defines the name of the function that you want to run.

Optional Input Parameters

Defines the function parameters, in JSON format, that enable you to control the presentation of data.

Copy

  "param1":"value1",
  "param2":"value2"
}

For no parameters, type {}.

Function Type

Determines which of the following types of Azure functions to run:

  • Activity: The basic unit of work in a durable function and that can be called by an orchestrator function.

  • HTTP: An HTTP-triggered function that can be used to build serverless APIs and respond to webhooks. You cannot call an orchestrator function from an HTTP function.

  • Orchestrator: A type of durable function that enables you to determine how and in what order actions are executed.

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the Azure Functions job.

Set to 120 seconds or longer for jobs that run for more than an hour.

Default: 20

Azure VM Job

Azure Virtual Machine (VM) is a cloud computing platform.

The following table describes the Azure VM job type attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to Azure VM.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

VM Name

Defines the name of the Azure Virtual Machine to run the operation.

Operation

Determines one of the following operations to perform on the Azure Virtual Machine

  • Create\Update: Creates a new virtual machine.

  • Delete: Deletes an existing virtual machine.

  • Deallocate: Empties a virtual machine.

  • Reset: Resets a virtual machine.

  • Start: Starts a virtual machine.

  • Stop: Stops a running virtual machine.

Input Parameters

Defines the input parameters in JSON format for a Create operation.

Copy
{"param1":"value1", "param2", "value2", …}

Get Logs

Determines whether to display logs from Azure VM at the end of the job output.

This parameter is available for all operations except for the Delete operation.

Delete VM disk

Determines whether to delete the Azure Virtual Machine disk when you delete an Azure Virtual Machine.

Verification Poll Interval

Determines the number of seconds to wait before job status verification.

Set to 120 seconds or longer for jobs that run for more than an hour.

Default: 15 seconds

Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2 times

GCP Batch Job

Google Cloud Platform (GCP) Batch enables you to manage, schedule, and run batch computing workloads on a virtual machine that is provisioned to accommodate your resource and capacity needs.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to GCP Batch.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Project ID

Defines the GCP project ID where the batch job runs.

A project is a set of configuration settings that define the resources your GCP Batch jobs use and how they interact with GCP.

Region

Defines the region that is predefined in the GCP Batch platform where the virtual machine resources are located.

us-east1

Override Region

Determines whether to override the predefined region in the GCP Batch platform, as follows:

  • Yes

  • No

Default: No

Allowed Locations

Defines the new region, or zones in a region, where the virtual machine resources are located.

This limits the selection of virtual machine resources to two zones in the US Central region:

["zones/us-central1-a", "zones/us-central1-c"]

Job Name

Defines a unique name for the batch job.

Priority

Determines the run priority of the batch job.

Larger numbers indicate higher priority.

If Job A has a priority of 20, and Job B has a priority of 1, Job A runs first.

Valid Values: Any number between 1 and 99.

Default: 99

Runnable Type

Determines one of the following types of batch jobs:

  • Script: Runs a UNIX shell scripting language.

  • Container: Creates a Docker container from an image.

Task Script Text

Defines the shell script that the batch job runs.

Container Image URI

Defines the Uniform Resource Identifier (URI) that points to the container image.

Entry Point

Defines the entry point (ENTRYPOINT) for the Docker container, which overrides the entry point defined in the original image.

An entry point is the location in the container where the program begins its execution.

Copy
"/bin/project"

Override Commands

Determines whether to override the Docker command (CMD), which is the executable container application code that is defined in the original image, as follows.

  • Yes

  • No

Default: No

Commands

Defines the Docker command (CMD) that executes when a Docker container runs.

If the container image contains an entry point or if an entry point is defined in the Entry Point attribute above, the command is appended as an argument.

Container Volumes

Defines the file or directory to copy (mount) onto a container volume, which is a virtual hard drive, inside the Docker container.

This copies the /home/user directory on the host machine and pastes it into the /app directory on a volume in the Docker container:

Copy
/home/usr:/app/

CPU

Determines the number of millicores of virtual CPU resources that are reserved for the batch job.

Virtual machines measure CPU resources in millicores (m), which are thousandths of a CPU core. For example, 2000m equals 2 cores.

Memory

Determines the number of mebibytes (MiB; mega binary bytes) of virtual memory resources that are reserved for the batch job.

Maximum Retry Count

Determines the number of times to retry a job run when it fails.

Valid Values: Any number between 0 and 10.

Default: 0

Instance Policy

Determines which kind of virtual machine (instance) runs the job.

  • Machine Type: Runs the job on one of the virtual machine types available in the Google Compute Engine (GCE), which is the infrastructure-as-a-service component of GCP.

  • Machine Template: Runs the job on a custom-made virtual machine, based on a predefined template in the GCE.

Machine Type

Defines the virtual machine type that runs the job.

e2-micro

Machine Template

Defines the virtual machine template that runs the job.

Provisioning Model

Determines the price and availability of virtual machine resources, as follows:

  • Standard: Allocates virtual machine resources to the user and is used for production-level jobs.
  • Spot: Allocates virtual machine resources only when there is excess capacity in the Google Compute Engine (GCE). These resources disappear when they are needed by other users, which might prevent a job from running or cause it to fail.

Default: Standard

Logs Policy

Determines whether save the batch job logs and where they appear.

  • No Logs: The batch job logs are not saved in the GCP Batch platform or in the Control-M Log in the Monitoring domain.

  • Cloud Logging: The batch job logs are saved in both the GCP Batch platform and the Control-M Log in the Monitoring domain.

Use Advanced JSON Format

Determines whether you supply your own JSON parameters.

JSON Format

Defines the parameters for the batch job, in JSON format, that enable you to control how the job runs.

For a description of this JSON syntax, see the description of Resource:Job in the GCP Batch Job API Reference.

Copy
{
"priority": "99",
"taskGroups": [
{
"taskSpec": {
"computeResource": {
"cpuMilli": "1000",
"memoryMib": "1000"
},
"runnables": [
{
"script": {
"text": "Total tasks: ${BATCH_TASK_COUNT}"
}
}
]
},
"taskCount": "1",
"parallelism": "1"
}
],
"allocationPolicy": {
"location": {
"allowedLocations": [
"regions/us-east1",
"zones/us-east1-b",
"zones/us-east1-c",
"zones/us-east1-d"
]
},
"labels": {
"batch-job-id": "guy3"
},
"instances": [
{
"policy": {
"machineType": "e2-micro",
"provisioningModel": "STANDARD"
}
}
]
},
"logsPolicy": {
"destination": "CLOUD_LOGGING"
}
}

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the GCP Batch job.

Set to 120 seconds or longer for jobs that run for more than an hour.

Default: 10

GCP VM Job

The Google Cloud Platform (GCP) Virtual Machine (VM) job enables you to create, manage, and delete virtual machines on the Google Compute Engine (GCE).

The following table describes the GCP VM job type attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to GCP VM.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Project ID

Defines the project ID of the Google Cloud Project Virtual Machine.

Zone

Defines the name of the zone for the request.

Example: us-central1-f

Operation

Determines one of the following operations to perform on the Google Virtual Machine:

  • Create: Creates a virtual machine

  • Create from template: Creates a new virtual machine based on a template.

  • Start: Starts an existing virtual machine.

  • Stop: Stops a running virtual machine.

  • Reset: Resets a virtual machine.

  • Delete: Deletes an existing virtual machine.

Template Name

Defines the name of a template for creation of a new Google Virtual Machine from a template.

Instance Name

Defines the name of the VM instance where you want to run the operation.

This parameter is available for all operations except for the Create operations.

Parameters

Defines the input parameters in JSON format for a Create operation.

Copy
{"param1":"value1", "param2", "value2", …}

Get Logs

Determines whether to display logs from Google VM at the end of the job output.

This parameter is available for all operations except for the Delete operation.

Verification Poll Interval

Determines the number of seconds to wait before job status verification.

Set to 120 seconds or longer for jobs that run for more than an hour.

Default: 15

Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2