Container Orchestration Jobs

The following topics describe job types for container orchestration platforms and services:

Job:AWS ECS

AWS Elastic Container Service (ECS) is a container management service that enables you to execute, stop, manage, and monitor containerized applications in a cluster.

To deploy and run an AWS ECS job, ensure that you have installed the AWS ECS plug-in with the provision image command or the provision agent::update command.

The following examples show how to define an AWS ECS job.

  • This JSON defines preset Control-M parameters:

    Copy 
    "AWS ECS_Job_1": 
    {
       "Type": "Job:AWS ECS",
       "ConnectionProfile": "ECS",
       "Action": "Preset Json",
       "ECS Cluster Name": "ECSIntegrationCluster",
       "ECS Task Definition": "ECSIntegrationTask",
       "Launch Type": "FARGATE",
       "Assign Public IP": "True",
       "Network Security Groups": "\"sg-01e4a5bfac4189d10\"",
       "Network Subnets": "\"subnet-045ddaf41d4852fd7\", \"subnet-0b574cca721d462dc\", \"subnet-0e108b6ba4fc0c4d7\"",
       "Override Container": "IntegrationURI",
       "Override Command": "\"/bin/sh -c 'whoami'\"",
       "Environment Variables": "{\"name\": \"var1\", \"value\": \"1\"}",
       "Logs": "Get Logs",
       "Status Polling Frequency":"10",
       "Failure Tolerance":"5"
    }

  • This JSON defines job parameters via a submitted JSON body:

    Copy 
    "AWS ECS_Job_2": 
    {
       "Type": "Job:AWS ECS",
       "ConnectionProfile": "ECS",
       "Action": "Manual Json",
       "Parameters":
       {
          "cluster":"ECSIntegrationCluster",
          "launchType":"FARGATE",
          "networkConfiguration":
          {
             "awsvpcConfiguration":
             {
                "assignPublicIp":"ENABLED",
                "securityGroups":["sg-01e4a5bfac4189d10"],
                "subnets":["subnet-045ddaf41d4852fd7","subnet-0b574cca721d462dc","subnet-0e108b6ba4fc0c4d7"]
             }
          },
          "overrides":
          {
             "containerOverrides":[
             {
                "command":["/bin/sh -c 'whoami'"],
                "environment":[ 
                {   
                   "name":"var1",
                   "value":"hello"
                } ],
                "name":"IntegrationURI"
             } ]
          },
          "taskDefinition":"ECSIntegrationTask"
       },
       "Logs": "Get Logs",
       "Status Polling Frequency":"10",
       "Failure Tolerance":"5"
    }

The following table describes the AWS ECS job parameters.

Parameter

Description

Connection Profile

Defines the ConnectionProfile:AWS ECS name that connects Control-M to AWS ECS.

Action

Determines one of the following actions to perform on AWS ECS:

  • Preset JSON: Runs the task according to the attributes that you define in this job.

  • Manual JSON: Runs the task according to the JSON parameters that you supply in the Parameters attribute to submit as a JSON body.

Parameters

Defines parameters to submit as a JSON body, to control how the AWS ECS task runs.

The JSON body replaces various other parameters described below, provided that you set the Action parameter to Manual Json.

For a description of this JSON syntax, see RunTask in the AWS ECS API Reference.

ECS Cluster Name

Defines the ECS cluster on the AWS ECS platform where the job runs.

An ECS cluster is a logical group of tasks and services.

ECS Task Definition

Defines the task definition on the AWS ECS platform.

The task definition describes the container image, command, environment variables, and other parameters that run your application.

Launch Type

Determines the type infrastructure where your tasks and services run:

  • Fargate: Runs your application on a virtual machine that is automatically provisioned and managed.

  • EC2: Runs your application on a predefined AWS EC2 virtual machine that you provision and manage.

  • External: Runs your application on a different virtual machine that you define in the AWS ECS platform.

Assign Public IP

Determines whether the job has a public internet protocol (IP).

Valid Values:

  • True

  • False

Network Security Groups

Defines which network security group your task is connected to, through the elastic network interface, which is a virtual network card that controls inbound and outbound traffic.

Network Subnets

Defines the virtual subnet, which determines the IP addresses for the task.

Override Container

Defines which override container to use that overrides the default container image, command, or other settings specified in the task definition.

Override Command

Defines the command to run in the container that overrides any command specified in the task definition.

Environment Variables

Defines the environment variables for the container, which are used to manage the container and pass information to the application that runs inside it.

Logs

Determines whether the logs from the AWS ECS platform appear at the end of the Control-M job output.

  • Get Logs

  • Don't Get Logs

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the job.

Default: 10

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 5

Job:AWS App Runner

AWS App Runner is a service to deploy containerized web applications and APIs directly from source code or container images, without the need to manage infrastructure.

To deploy and run an AWS App Runner job, ensure that you have installed the AWS App Runner plug-in with the provision image command or the provision agent::update command.

The following example shows how to define an AWS App Runner job.

Copy 
{
    "AWS App Runner_Job_2": {
        "Type": "Job:AWS App Runner",
        "ConnectionProfile": "AAP",
        "Action": "Deploy",
        "Service Arn": "arn:aws:apprunner:us-east-1:xxxxxxxxxx:service/Hello_World/xxxxxxx",
        "Output Job Logs": "unchecked",
        "Status Polling Frequency":"60",
        "Failure Tolerance":"2"        
    }
}

The following table describes the AWS App Runner job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:AWS App Runner name that connects Control-M to AWS App Runner.

Action

Determines one of the following actions to perform on App Runner:

  • Deploy: Deploys an application to an AWS App Runner service.

  • Pause Service: Pauses the AWS App Runner service.

  • Resume Service: Resumes the AWS App Runner service.

Service Arn

Defines the Amazon Resource Name that is a unique identifier assigned to each App Runner service to manage and reference the service.

Output Job Logs

Determines whether to display logs in the output.

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the job.

Default: 60

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2

Job:Azure Container Instances

Azure Container Instances enables you to run an isolated container in Azure, without having to manage any virtual machines and without having to adopt a higher-level service.

To deploy and run an Azure Container Instances job, ensure that you have installed the Azure Container Instances plug-in with the provision image command or the provision agent::update command.

The following example shows how to define an Azure Container Instances job.

Copy 
"Azure Container Instances_Job_2": 
{
   "Type": "Job:Azure Container Instances",
   "ConnectionProfile": "ACI",
   "Resource Group Name": "Karen_Resource_Group",
   "Container Group Name": "ksu-batch-job-word-counter-in-novel",
   "Append Log to Output": "checked",
   "Container Name": "ksu-batch-job-word-counter-in-novel",
   "Tail": "10",
   "Timestamps": "checked",
   "Status Polling Frequency":"60",
   "Failure Tolerance":"2"
}

The following table describes the Azure Container Instances job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:Azure Container Instances name that connects Control-M to Azure Container Instances.

Resource Group Name

Determines the Azure Resource Group that is associated with a specific container instance. A resource group is a container that holds related resources for an Azure solution.

The resource group can include all the resources for the solution, or only those resources that you want to manage as a group.

Container Group Name

Determines the container group name that is configured in Azure.

Append Log to Output

Determines whether to add the job log to the outputClosed A tab in the job properties pane of the Monitoring domain where the job output appears that indicates whether a job ended OK, and is used, for example, with jobs that check file location.. If you choose this option, the Container name, Tail, and Timestamps parameters are relevant.

Valid Values:

  • Checked

  • Unchecked

Default: Unchecked

Container Name

(Append Log to Output) Defines the container instance name that is configured in Azure under the Container group and used for the log.

Tail

(Append Log to Output) Defines the number of lines to show from the tail (end) of the Azure Container Instance log. To show all the available logs up to 4MB, leave this field empty.

Default: 10

Timestamps

(Append Log to Output) Determines whether to add a timestamp to every line of the log output.

Valid Values:

  • Checked

  • Unchecked

Default: Unchecked

Status Polling Frequency

Determines the number of seconds to wait before checking the job status.

Default: 60

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2

Job:GCP Cloud Run

GCP Cloud Run is a container management service that enables you to execute, stop, manage, and monitor containerized applications in a cluster.

To deploy and run an GCP Cloud Run job, ensure that you have installed the GCP Cloud Run plug-in with the provision image command or the provision agent::update command.

The following example shows how to define an GCP Cloud Run job.

Copy 
"GCP Cloud Run_Job": 
{
   "Type": "Job:GCP Cloud Run",
   "ConnectionProfile": "GCR",
   "Project ID": "applied-lattice-123456",
   "Location": "us-central1",
   "Job Name": "testjob",
   "Overrides Specification": "{}",
   "Status Polling Frequency": "5"
}

The following table describes the GCP Cloud Run job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:GCP Cloud Run name that connects Control-M to GCP Cloud Run.

Project ID

Defines the ID of the predefined Google Cloud project that holds your configured APIs, authentication information, billing details, and job resources.

Location

Determines the region where the GCP Cloud Run job executes.
Job Name Defines the unique job name set in GCP Cloud Run.
Overrides Specification

Defines the parameters to override in JSON format when the job executes, as shown in the following example:

Copy 
{
   "containerOverrides": [
   {
      object (ContainerOverride)
   } ],
   "taskCount": integer,
   "timeout": string
}

Status Polling Frequency

Determines the number of seconds to wait before Control-M checks the job status.

Default: 5

Job:Kubernetes

The Kubernetes job enables you to run a pod to completion in a Kubernetes-based cluster.

To deploy and run a Kubernetes job, ensure that you have set up Helix Control-M for Kubernetes, as described in Setting Up Control-M for Kubernetes.

The following example shows how to define a Kubernetes job:

Copy 
"Kubernetes_Job": 
{
   "Type": "Job:Kubernetes",
   "ConnectionProfile": "KBN_CCP",
   "Description": "Containerized Hello World",
   "Job Spec Yaml" : "apiVersion: batch/v1\nkind: Job\nmetadata:\n  name: {{job_yaml_file_params:jobname}}\nspec:\n  template:\n    spec:\n      nodeSelector:\n        kubernetes.io/os: linux\n      containers:\n      - name: busybox0\n        image: busybox\n        command: [\"echo\",  \"Hello {{job_yaml_file_params:subject}}\"]\n      restartPolicy: Never\n  backoffLimit: 4\n",
   "Job Spec Parameters" : "{\"jobname\":\"ctmjob-%%ORDERID\",\"subject\":\"Sweden\"}",
   "Get Pod Logs": "Get Logs",
   "OS Exit Code": "No print", 
   "Job Cleanup": "Keep",
   "Job Status Polling Interval": "20"
}

The following table describes the Kubernetes job parameters.

Parameter

Description

Connection Profile

Defines the ConnectionProfile:Kubernetes name that connects Control-M to Kubernetes.

Job Spec YAML

Defines the settings of the job in Kubernetes.

Convert the contents of the YAML file to JSON and include them in the JSON code.

Tip: You can use the JQ command on Linux to convert the YAML file to JSON.
Job Spec Parameters

Defines input parameters required by the Kubernetes job, as name:value pairs.

Copy 
{\"jobname\":\"ctmjob-%%ORDERID\",\"iterations\":\"10\",\"delay\":\"20\"}

Get Pod Logs

Determines whether to fetch logs of the pods of the Kubernetes job upon completion and append to the Control-M job output.

Maximum Output Size (Megabytes): 10

  • Get Logs

  • Do Not Get

Default: Get Logs

OS Exit Code

Determines whether to include an OS return code from the Application pod in the Control-M job output. This is supported for a Kubernetes job without retries (backofflimit: 0) of a single-container pod. Introduced in Control-M for Kubernetes 9.0.21.205 (plug-in version 1.0.02).

Valid values:

  • No print

  • Print code of single pod

Default: No print

Job Cleanup

Determines whether to delete Kubernetes resources that were created for the job.

  • Delete Job

  • Keep

Default: Delete Job
Job Status Polling Interval

Determines the number of seconds between status checks of the Kubernetes job.

Default: 20