Cloud Computing Jobs

The following topics describe job attributes that work with cloud computing platforms and services:

AWS Batch Job

AWS Batch enables you to manage and execute batch computing workloads in AWS.

The following table describes the AWS Batch job attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to AWS Batch.

Rules:

  • Characters: 1−30

  • Case Sensitive: Yes

  • Invalid Characters: Blank spaces.

Use Advanced JSON Format

Determines whether you supply your own JSON parameters.

JSON Format

Defines the parameters for the batch job, in JSON format, that enable you to control how the job executes.

For a description of this JSON syntax, see the description of SubmitJob in the AWS Batch API Reference.

Copy
{
   "containerOverrides":
   {
      "command":["echo","Hello, from Control-M"],
         "resourceRequirements":[
         {
            "type":"VCPU",
            "value":"2"
         }
      ]
   }
   "jobDefinition": "ctm-batch-job-definition",
   "jobName": "ctm-batch-job-single",
   "jobQueue": "ctm-batch-job-queue",
   "timeout"
   {
      "attemptDurationSeconds": 70
   }
}

Job Name

Defines the name of the batch job.

Job Definition and Revision

Determines which predefined job definition and version number (revision) is applied to the job, depending upon how you complete the field, as follows:

  • Job Definition Name: When you type only the job definition name in the field, this job definition and its most recent version is applied to the job.

  • Job Definition Name:Version Number: When you type the job definition name—followed by a colon (:), followed by a version number—in the field, this job definition and version number is applied to the job.

ctm-batch-job-definition:3

Job Queue

Determines the job queue, which stores your batch job.

Container Overrides Command

(Optional) Defines a command, in JSON format, that overrides the specified command in the job definition.

Job Attempts

(Optional) Determines the number of times to retry a job execution, which overrides the retry attempts determined in the job definition.

Valid Values: 1–10

Execution Timeout

(Optional) Determines the number of seconds to wait before a timeout occurs in a batch job, which overrides the specified timeouts in the job definition.

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the AWS Batch job.

Default: 20

AWS EC2 Job

AWS Elastic Compute Cloud (EC2) enables you to create virtual machines in the Amazon cloud-computing platform.

The following table describes the AWS EC2 job type attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to AWS EC2.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Operations

Determines one of the following operations to perform on the AWS EC2 Virtual Machine:

  • Create: Creates a virtual machine.

  • Create from Instance Template: Creates a virtual machine based on a template.

  • Start: Starts an existing virtual machine.

  • Stop: Stops a running virtual machine.

  • Reboot: Reboots a virtual machine.

  • Terminate: Deletes an existing virtual machine.

Launch Template ID

Defines the template to use to create a VM from a template.

Instance ID

Defines the name of the VM instance where you want to execute the operation.

This parameter is available for all operations except for the Create operations.

Instance Name

Defines the name of a new VM instance for Create operations.

Placement Availability Zone

Determines which AWS EC2 zone to use for a Create operation.

Instance Type

Determines the software requirements of the host computer when you create a new AWS EC2 Virtual Machine.

Subnet ID

Defines the Subnet ID that is required to launch the instance in a Create operation.

Key Name

Defines the security credential key set for a Create operation.

Image ID

Defines the ID of the Amazon Machine Image (AMI) that is required to launch the instance in a Create operation.

Number of copies

Determines the number of copies of the VM to create in a Create operation.

Default: 1

Get Instance logs

Determines whether to display logs from the AWS EC2 instance at the end of the job output.

This parameter is available for all operations except for the Terminate operation.

Verification Poll Interval

Determines the number of seconds to wait before job status verification.

Default: 15 seconds

Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2 times

Azure Batch Accounts Job

Azure Batch Accounts enables you to efficiently execute large-scale, parallel, computer-intensive tasks in the cloud.

The following table describes the Azure Batch Accounts job attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to Azure Batch Accounts.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Batch Job ID

Defines the name of the job that is created in Azure Portal.

Task ID Prefix

Defines a prefix that is added to the task ID.

Task Command Line

Defines the commands that the job performs.

Max Wall Clock Time

Determines the maximum amount of time the job executes, as follows:

  • Unlimited

  • Custom

Variable name: %%AWS-BATCH_ WALLCLOCK

Max Wall Time Digits

Defines the Custom maximum number of seconds, minutes, hours, or days the job executes.

Default: 1

Max Wall Time Unit

Determines the Custom units used to define the maximum amount of time the job executes.

  • Seconds

  • Minutes

  • Hours

  • Days

Default: Minutes

Max Task Retry Count

Determines the maximum number of times to re-execute a failed job, as follows:

  • None

  • Unlimited

  • Custom

Default: None

Retry Number

Determines the Custom maximum number of times to re-execute a failed job.

Default: 1

Retention Time

Determines the file retention period, as follows:

  • Unlimited: Retains the job directory and its files indefinitely.
  • Custom: Determines the number of days, hours, minutes, or seconds to retain the job directory and its files.

Default: Unlimited

Retention Time Digits

Determines the number of seconds, minutes, hours, or days to set the Custom file retention period for.

Default: 1

Retention Time Unit

Determines the unit of time used to set the Custom file retention period for.

  • Seconds

  • Minutes

  • Hours

  • Days

Default: Hours

Append Log to Output

Determines whether to add the job log to the outputClosed A tab in the job properties pane of the Monitoring domain where the job output appears that indicates whether a job ended OK, and is used, for example, with jobs that check file location..

Status Polling Interval

Determines the number of seconds to wait before checking the status of the Azure Batch Accounts job.

Default: 20

Azure Functions Job

The Azure Functions job enables you to develop, test, and run applications in the cloud.

The following table describes Azure Functions job attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to Azure Functions.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Variable name: %%AZURE-ACCOUNT

Function App

Defines the name of the Azure function application that you want to execute.

Function Name

Defines the name of the function that you want to execute.

Optional Input Parameters

Defines the function parameters, in JSON format, that enable you to control the presentation of data.

Copy

  "param1":"value1",
  "param2":"value2"
}

For no parameters, type {}.

Function Type

Determines which of the following types of Azure functions to execute:

  • Activity: The basic unit of work in a durable function and that can be called by an orchestrator function.

  • HTTP: An HTTP-triggered function that can be used to build serverless APIs and respond to webhooks. You cannot call an orchestrator function from an HTTP function.

  • Orchestrator: A type of durable function that enables you to determine how and in what order actions are executed.

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the Azure Functions job.

Default: 20

Azure VM Job

Azure Virtual Machine (VM) enables you to create, manage, and delete virtual machines in the Azure cloud.

The following table describes the Azure VM job type attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to Azure VM.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

VM Name

Defines the name of the Azure Virtual Machine to execute the operation.

Operation

Determines one of the following operations to perform on the Azure Virtual Machine

  • Create\Update: Creates a virtual machine.

  • Delete: Deletes an existing virtual machine.

  • Deallocate: Empties a virtual machine.

  • Reset: Reboots a virtual machine.

  • Start: Starts a virtual machine.

  • Stop: Stops a running virtual machine.

Input Parameters

Defines the input parameters in JSON format for a Create operation.

Copy
{"param1":"value1", "param2", "value2", …}

Get Logs

Determines whether to display logs from Azure VM at the end of the job output.

This parameter is available for all operations except for the Delete operation.

Delete VM Disk

Determines whether to delete the Azure Virtual Machine disk when you delete an Azure Virtual Machine.

Verification Poll Interval

Determines the number of seconds to wait before job status verification.

Default: 15 seconds

Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2 times

GCP Batch Job

Google Cloud Platform (GCP) Batch enables you to manage, schedule, and execute batch computing workloads on a virtual machine that is provisioned to accommodate your resource and capacity needs.

The following table describes the GCP Batch job type attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to GCP Batch.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Project ID

Defines the GCP project ID where the batch job executes.

A project is a set of configuration settings that define the resources your GCP Batch jobs use and how they interact with GCP.

Region

Defines the region that is predefined in the GCP Batch platform where the virtual machine resources are located.

us-east1

Override Region

Determines whether to override the predefined region in the GCP Batch platform, as follows:

  • Yes

  • No

Default: No

Allowed Locations

Defines the new region, or zones in a region, where the virtual machine resources are located.

This limits the selection of virtual machine resources to two zones in the US Central region:

["zones/us-central1-a", "zones/us-central1-c"]

Job Name

Defines a unique name for the batch job.

Priority

Determines the execution priority of the batch job.

Larger numbers indicate higher priority.

If Job A has a priority of 20, and Job B has a priority of 1, Job A executes first.

Valid Values: Any number between 1 and 99.

Default: 99

Runnable Type

Determines one of the following types of batch jobs:

  • Script: Executes a UNIX shell scripting language.

  • Container: Creates a Docker container from an image.

Task Script Text

Defines the shell script that the batch job executes.

Container Image URI

Defines the Uniform Resource Identifier (URI) that points to the container image.

Entry Point

Defines the entry point (ENTRYPOINT) for the Docker container, which overrides the entry point defined in the original image.

An entry point is the location in the container where the program begins its execution.

Copy
"/bin/project"

Override Commands

Determines whether to override the Docker command (CMD), which is the executable container application code that is defined in the original image, as follows.

  • Yes

  • No

Default: No

Commands

Defines the Docker command (CMD) that executes when a Docker container executes.

If the container image contains an entry point or if an entry point is defined in the Entry Point attribute above, the command is appended as an argument.

Container Volumes

Defines the file or directory to copy (mount) onto a container volume, which is a virtual hard drive, inside the Docker container.

This copies the /home/user directory on the host machine and pastes it into the /app directory on a volume in the Docker container:

Copy
/home/usr:/app/

CPU

Determines the number of millicores of virtual CPU resources that are reserved for the batch job.

Virtual machines measure CPU resources in millicores (m), which are thousandths of a CPU core. For example, 2000m equals 2 cores.

Memory

Determines the number of mebibytes (MiB; mega binary bytes) of virtual memory resources that are reserved for the batch job.

Maximum Retry Count

Determines the number of times to retry a job execution when it fails.

Valid Values: Any number between 0 and 10.

Default: 0

Instance Policy

Determines which kind of virtual machine (instance) executes the job.

  • Machine Type: Executes the job on one of the virtual machine types available in the Google Compute Engine (GCE), which is the infrastructure-as-a-service component of GCP.

  • Machine Template: Executes the job on a custom-made virtual machine, based on a predefined template in the GCE.

Machine Type

Defines the virtual machine type that executes the job.

e2-micro

Machine Template

Defines the virtual machine template that executes the job.

Provisioning Model

Determines the price and availability of virtual machine resources, as follows:

  • Standard: Allocates virtual machine resources to the user and is used for production-level jobs.
  • Spot: Allocates virtual machine resources only when there is excess capacity in the Google Compute Engine (GCE). These resources disappear when they are needed by other users, which might prevent a job from executing or cause it to fail.

Default: Standard

Logs Policy

Determines whether save the batch job logs and where they appear.

  • No Logs: The batch job logs are not saved in the GCP Batch platform or in the Control-M Log in the Monitoring domain.

  • Cloud Logging: The batch job logs are saved in both the GCP Batch platform and the Control-M Log in the Monitoring domain.

Use Advanced JSON Format

Determines whether you supply your own JSON parameters.

JSON Format

Defines the parameters for the batch job, in JSON format, that enable you to control how the job executes.

For a description of this JSON syntax, see the description of Resource:Job in the GCP Batch Job API Reference.

Copy
{
    "priority": "99",
    "taskGroups": [
        {
            "taskSpec": {
                "computeResource": {
                    "cpuMilli": "1000",
                    "memoryMib": "1000"
                },
                "runnables": [
                    {
                        "script": {
                            "text": "Total tasks: ${BATCH_TASK_COUNT}"
                        }
                    }
                ]
            },
            "taskCount": "1",
            "parallelism": "1"
        }
    ],
    "allocationPolicy": {
        "location": {
            "allowedLocations": [
                "regions/us-east1",
                "zones/us-east1-b",
                "zones/us-east1-c",
                "zones/us-east1-d"
            ]
        },
        "labels": {
            "batch-job-id": "guy3"
        },
        "instances": [
            {
                "policy": {
                    "machineType": "e2-micro",
                    "provisioningModel": "STANDARD"
                }
            }
        ]
    },
    "logsPolicy": {
        "destination": "CLOUD_LOGGING"
    }
}

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the GCP Batch job.

Default: 10

GCP Functions Job

The GCP Functions job enables you to develop, test, and run applications in the cloud.

The following table describes GCP Functions job attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to GCP Functions.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Variable name: %%AZURE-ACCOUNT

Project ID

Defines the GCP project ID where the GCP Functions job executes.

A project is a set of configuration settings that define the resources your GCP Batch jobs use and how they interact with GCP.

Location

Defines where the functions job executes.

us-central1-f

Function Name

Defines the name of the function that you want to execute.

Function Parameters

Determines one of the following types of parameters to pass to the function:

  • URL Parameters

  • Body

URL Parameters

Defines the URL parameters that are passed to the function, in the following format:

<Parameter_Name_1>=<Value_1>

Body

Defines the JSON-based body parameters that are passed to the function, in the following format:

Copy
{\"parameter_1\":\"value_1\", \"parameter_2\":\"value_2\"}

For no parameters, type {}.

Maximum Characters: 4,000

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the GCP Functions job.

Default: 20

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2

Get Logs

Determines whether to append the GCP Functions logs to the outputClosed A tab in the job properties pane of the Monitoring domain where the job output appears that indicates whether a job ended OK, and is used, for example, with jobs that check file location..

GCP VM Job

Google Cloud Platform (GCP) Virtual Machine (VM) enables you to create, manage, and delete virtual machines on the Google Compute Engine (GCE).

The following table describes the GCP VM job type attributes.

Attribute

Description

Connection Profile

Determines the authorization credentials that are used to connect Control-M to GCP VM.

Rules:

  • Characters: 1−30

  • Case sensitive: Yes

  • Invalid characters: Blank spaces.

Project ID

Defines the project ID of the Google Cloud Project Virtual Machine.

Zone

Defines the name of the zone for the request.

us-central1-f

Operation

Determines one of the following operations to perform on the Google Virtual Machine:

  • Create: Creates a virtual machine

  • Create from template: Creates a virtual machine based on a template.

  • Start: Starts an existing virtual machine.

  • Stop: Stops a running virtual machine.

  • Reset: Reboots a virtual machine.

  • Delete: Deletes an existing virtual machine.

Template Name

Defines the name of a template for creation of a new Google Virtual Machine from a template.

Instance Name

Defines the name of the VM instance where you want to execute the operation.

This parameter is available for all operations except for the Create operations.

Parameters

Defines the input parameters in JSON format for a Create operation.

Copy
{"param1":"value1", "param2", "value2", …}

Get Logs

Determines whether to display logs from Google VM at the end of the job output.

This parameter is available for all operations except for the Delete operation.

Verification Poll Interval

Determines the number of seconds to wait before job status verification.

Default: 15

Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2