Container Orchestration Jobs

The following topics describe job types for container orchestration platforms and services:

Job:AWS ECS

AWS Elastic Container Service (ECS) is a container management service that enables you to execute, stop, manage, and monitor containerized applications in a cluster.

To deploy and run an AWS ECS job, ensure that you have done the following:

The following examples show how to define an AWS ECS job.

  • This JSON defines preset Control-M parameters:

    Copy 
    "AWS ECS_Job_1": 
    {
       "Type": "Job:AWS ECS",
       "ConnectionProfile": "ECS",
       "Action": "Preset Json",
       "ECS Cluster Name": "ECSIntegrationCluster",
       "ECS Task Definition": "ECSIntegrationTask",
       "Launch Type": "FARGATE",
       "Assign Public IP": "True",
       "Network Security Groups": "\"sg-01e4a5bfac4189d10\"",
       "Network Subnets": "\"subnet-045ddaf41d4852fd7\", \"subnet-0b574cca721d462dc\", \"subnet-0e108b6ba4fc0c4d7\"",
       "Override Container": "IntegrationURI",
       "Override Command": "\"/bin/sh -c 'whoami'\"",
       "Environment Variables": "{\"name\": \"var1\", \"value\": \"1\"}",
       "Logs": "Get Logs",
       "Status Polling Frequency":"10",
       "Failure Tolerance":"5"
    }

  • This JSON defines job parameters via a submitted JSON body:

    Copy 
    "AWS ECS_Job_2": 
    {
       "Type": "Job:AWS ECS",
       "ConnectionProfile": "ECS",
       "Action": "Manual Json",
       "Parameters":
       {
          "cluster":"ECSIntegrationCluster",
          "launchType":"FARGATE",
          "networkConfiguration":
          {
             "awsvpcConfiguration":
             {
                "assignPublicIp":"ENABLED",
                "securityGroups":["sg-01e4a5bfac4189d10"],
                "subnets":["subnet-045ddaf41d4852fd7","subnet-0b574cca721d462dc","subnet-0e108b6ba4fc0c4d7"]
             }
          },
          "overrides":
          {
             "containerOverrides":[
             {
                "command":["/bin/sh -c 'whoami'"],
                "environment":[ 
                {   
                   "name":"var1",
                   "value":"hello"
                } ],
                "name":"IntegrationURI"
             } ]
          },
          "taskDefinition":"ECSIntegrationTask"
       },
       "Logs": "Get Logs",
       "Status Polling Frequency":"10",
       "Failure Tolerance":"5"
    }

The following table describes the AWS ECS job parameters.

Parameter

Description

Connection Profile

Defines the ConnectionProfile:AWS ECS name that connects Control-M to AWS ECS.

Action

Determines one of the following actions to perform on AWS ECS:

  • Preset JSON: Runs the task according to the attributes that you define in this job.

  • Manual JSON: Runs the task according to the JSON parameters that you supply in the Parameters attribute to submit as a JSON body.

Parameters

Defines parameters to submit as a JSON body, to control how the AWS ECS task runs.

The JSON body replaces various other parameters described below, provided that you set the Action parameter to Manual Json.

For a description of this JSON syntax, see RunTask in the AWS ECS API Reference.

ECS Cluster Name

Defines the ECS cluster on the AWS ECS platform where the job runs.

An ECS cluster is a logical group of tasks and services.

ECS Task Definition

Defines the task definition on the AWS ECS platform.

The task definition describes the container image, command, environment variables, and other parameters that run your application.

Launch Type

Determines the type infrastructure where your tasks and services run:

  • Fargate: Runs your application on a virtual machine that is automatically provisioned and managed.

  • EC2: Runs your application on a predefined AWS EC2 virtual machine that you provision and manage.

  • External: Runs your application on a different virtual machine that you define in the AWS ECS platform.

Assign Public IP

Determines whether the job has a public internet protocol (IP).

Valid Values:

  • True

  • False

Network Security Groups

Defines which network security group your task is connected to, through the elastic network interface, which is a virtual network card that controls inbound and outbound traffic.

Network Subnets

Defines the virtual subnet, which determines the IP addresses for the task.

Override Container

Defines which override container to use that overrides the default container image, command, or other settings specified in the task definition.

Override Command

Defines the command to run in the container that overrides any command specified in the task definition.

Environment Variables

Defines the environment variables for the container, which are used to manage the container and pass information to the application that runs inside it.

Logs

Determines whether the logs from the AWS ECS platform appear at the end of the Control-M job output.

  • Get Logs

  • Don't Get Logs

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the job.

Default: 10

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 5

Job:Azure Container Instances

Azure Container Instances enables you to run an isolated container in Azure, without having to manage any virtual machines and without having to adopt a higher-level service.

To deploy and run a Azure Container Instances job, ensure that you have done the following:

The following example shows how to define an Azure Container Instances job.

Copy 
"Azure Container Instances_Job_2": 
{
   "Type": "Job:Azure Container Instances",
   "ConnectionProfile": "ACI",
   "Resource Group Name": "Karen_Resource_Group",
   "Container Group Name": "ksu-batch-job-word-counter-in-novel",
   "Append Log to Output": "checked",
   "Container Name": "ksu-batch-job-word-counter-in-novel",
   "Tail": "10"
   "Timestamps": "checked",
   "Status Polling Frequency":"60",
   "Failure Tolerance":"2"
}

The following table describes the Azure Container Instances job parameters.

Parameter

Description

ConnectionProfile

Determines the authorization credentials that are used to connect Control-M to Azure Container Instances, as described in Azure Container Instances Connection Profile Parameters.

Rules:

Characters: 1−30

Case sensitive: Yes

Invalid characters: Blank spaces.

Resource Group Name

Determines the Azure Resource Group that is associated with a specific container instance. A resource group is a container that holds related resources for an Azure solution.

The resource group can include all the resources for the solution, or only those resources that you want to manage as a group.

Container Group Name

Determines the container group name that is configured in Azure.

Append Log to Output

Determines whether to add the job log to the outputClosed A tab in the job properties pane of the Monitoring domain where the job output appears that indicates whether a job ended OK, and is used, for example, with jobs that check file location.. If you choose this option, the Container name, Tail, and Timestamps parameters will be relevant.

Valid Values:

  • Checked

  • Unchecked

Default: Unchecked

Container Name

(Append Log to Output) Defines the container instance name that is configured in Azure under the Container group and used for the log.

Tail

(Append Log to Output) Defines the number of lines to show from the Tail (end) of the Azure Container Instance log. Leave this field empty to show all the available logs up to 4MB.

Default: 10

Timestamps

(Append Log to Output) Determines whether to add a Timestamp to every line of the log output.

Default: Unchecked

Status Polling Frequency

Determines the number of seconds to wait before checking the job status.

Default: 60

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2

Job:GCP Cloud Run

GCP Cloud Run is a container management service that enables you to execute, stop, manage, and monitor containerized applications in a cluster.

To deploy and run a GCP Cloud Run job, ensure that you have done the following:

The following example shows how to define an GCP Cloud Run job.

Copy 
"GCP Cloud Run_Job": 
{
   "Type": "Job:GCP Cloud Run",
   "ConnectionProfile": "GCR",
   "Project ID": "applied-lattice-123456",
   "Location": "us-central1",
   "Job Name": "testjob",
   "Overrides Specification": "{}",
   "Status Polling Frequency": "5"
}

The following table describes the GCP Cloud Run job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:GCP Cloud Run name that connects Control-M to GCP Cloud Run.

Project ID

Defines the ID of the predefined Google Cloud project that holds your configured APIs, authentication information, billing details, and job resources.

Location

Determines the region where the GCP Cloud Run job executes.
Job Name Defines the unique job name set in GCP Cloud Run.
Overrides Specification

Defines the parameters to override in JSON format when the job executes, as shown in the following example:

Copy 
{
   "containerOverrides": [
   {
      object (ContainerOverride)
   } ],
   "taskCount": integer,
   "timeout": string
}

Status Polling Frequency

Determines the number of seconds to wait before Control-M checks the job status.

Default: 5

Job:Kubernetes

The Kubernetes job enables you to run a pod to completion in a Kubernetes-based cluster.

To deploy and run a Kubernetes job, ensure that you have set up Control-M for Kubernetes, as described in Setting Up Control-M for Kubernetes.

The following examples show how to define a Kubernetes job:

  • In this job, you provide the Kubernetes job spec YAML in the job definitions:

    Copy 
    "Kubernetes_Job": 
    {
       "Type": "Job:Kubernetes",
       "ConnectionProfile": "KBN_CCP",
       "Job Spec Type" : "Local file",
       "Description": "Containerized Hello World",
       "Job Spec Yaml" : "apiVersion: batch/v1\nkind: Job\nmetadata:\n  name: {{job_yaml_file_params:jobname}}\nspec:\n  template:\n    spec:\n      nodeSelector:\n        kubernetes.io/os: linux\n      containers:\n      - name: busybox0\n        image: busybox\n        command: [\"echo\",  \"Hello {{job_yaml_file_params:subject}}\"]\n      restartPolicy: Never\n  backoffLimit: 4\n",
       "Job Spec Parameters" : "{\"jobname\":\"ctmjob-%%ORDERID\",\"subject\":\"Sweden\"}",
       "Get Pod Logs": "Get Logs",
       "OS Exit Code": "No print", 
       "Job Cleanup": "Keep",
       "Job Status Polling Interval": "20"
    }

  • In this job, the Kubernetes job spec is retrieved using a web service request with a defined body:

    Copy 
    "Kubernetes_Job": 
    {
       "Type": "Job:Kubernetes",
       "ConnectionProfile": "KBN_CCP",
       "Job Spec Type" : "Remote web service",
       "Description": "Kubernetes job",
       "Spec Request Path" : "/jobs/myspec.yaml",
       "Spec Request Method" : "POST",
       "Spec Request Parameters" : [
          {
             "job-id" : "%%ORDERID"
          }, 
          {
             "iterations" : "5"
          }
       ],
       "Spec Request Headers" : [ 
          {
             "User-Agent" : "kbn-2.0.00"
          } 
       ],
       "WsRestBody" : 
          {
             "RequestDefinition" : "text",
             "BodyRequest" : "{%4E\"Job Identifier\": \"some-unique-id-000x\"%4E}"
          },
       "Get Pod Logs": "Get Logs",
       "OS Exit Code": "No print", 
       "Job Cleanup": "Keep",
       "Job Status Polling Interval": "20"
    }

  • In this job the Kubernetes job spec is retrieved using a web service request with a request body from a local file:

    Copy 
    "Kubernetes_Job": 
    {
       "Type": "Job:Kubernetes",
       "ConnectionProfile": "KBN_CCP",
       "Job Spec Type" : "Remote web service",
       "Description": "Kubernetes job",
       "Spec Request Path" : "/jobs/myspec.yaml",
       "Spec Request Method" : "POST",
       "Spec Request Parameters" : [
          {
             "job-id" : "%%ORDERID"
          }, 
          {
             "iterations" : "5"
          }
       ],
       "Spec Request Headers" : [ 
          {
             "User-Agent" : "kbn-2.0.00"
          } 
       ],
       "WsRestBody" : 
          {
             "RequestDefinition" : "filename",
             "FileName" : "/home/jobdefs/request-body.txt"
          },
       "Get Pod Logs": "Get Logs",
       "OS Exit Code": "No print", 
       "Job Cleanup": "Keep",
       "Job Status Polling Interval": "20"
    }

The following table describes the Kubernetes job parameters.

Parameter

Description

Connection Profile

Defines the ConnectionProfile:Kubernetes name that connects Control-M to Kubernetes.

Job Spec Type

Determines how to obtain the Kubernetes job spec:

  • Local file: Provide the Kubernetes job spec YAML file during job definition.

  • Remote web service: Retrieve the Kubernetes job spec from a remote location at the time of job execution.

Job Spec YAML

(Local file) Defines the settings of the job in Kubernetes.

Convert the contents of the YAML file to JSON and include them in the JSON code.

Tip: You can use the JQ command on Linux to convert the YAML file to JSON.
Job Spec Parameters

(Local file) Defines input parameters required by the Kubernetes job, as name:value pairs.

Copy 
{\"jobname\":\"ctmjob-%%ORDERID\",\"iterations\":\"10\",\"delay\":\"20\"}

Spec Request Path

(Remote web service) Defines the request path to the Kubernetes job spec in the remote location.

Spec Request Method

(Remote web service) Determines one of the following HTTP methods that is used to execute the REST request to retrieve the Kubernetes job spec from a remote location at the time of job execution:

  • GET: Reads or retrieves data from a resource.

  • POST: Adds a new resource.

Default: GET

Spec Request Parameters

(Remote web service) Defines optional URL parameters, as an array of Key:Value pairs.

Spec Request Headers

(Remote web service) Defines optional HTTP headers, as an array of Key:Value pairs.

If you created a connection profile that authenticates with OAuth2, as described in Kubernetes Connection Profile Parameters, you must define the following HTTP header key name and value:

"Authorization" : ""

WsRestBody

(Remote web service) Defines the REST request body, using the following parameters on the next level:

  • RequestDefinition: Determines whether to define a REST request body, and how to do so:

    • text: Textual input of the body

    • none: No request body (only URL parameters and headers)

    • filename: Use a predefined request body in a local file

  • BodyRequest: Defines the request body as textual input, in JSON format.

  • FileName: Defines the full path to a local file that contains a predefined request body.

Get Pod Logs

Determines whether to fetch logs of the pods of the Kubernetes job upon completion and append to the Control-M job output.

Maximum Output Size (Megabytes): 10

  • Get Logs

  • Do Not Get

Default: Get Logs

OS Exit Code

Determines whether to include an OS return code from the Application pod in the Control-M job output. This is supported for a Kubernetes job without retries (backofflimit: 0) of a single-container pod. Introduced in Control-M for Kubernetes 9.0.21.205 (plug-in version 1.0.02).

Valid values:

  • No print

  • Print code of single pod

Default: No print

Job Cleanup

Determines whether to delete Kubernetes resources that were created for the job.

  • Delete Job

  • Keep

Default: Delete Job
Job Status Polling Interval

Determines the number of seconds between status checks of the Kubernetes job.

Default: 20