Automating Code Deployment

This tutorial teaches you how to use a script to automate DevOps tasks. In order to complete this tutorial, you need a valid Control-M endPoint and API token.

Before you begin

Ensure that you meet the following prerequisites:

You have successfully completed API setup, as described in Setting Up the API.
You have Git installed. If not, obtain it from the Git Downloads page.
You have local copies of the tutorial samples from GitHub and a local copy of the source code using the git clone command:
CopyCopied to clipboard
```
git clone https://github.com/controlm/automation-api-quickstart.git
```

Step 1: Set the Control-M EnvironmentLink copied to clipboard

The first task when starting to work with Control-M Automation API is to configure the Control-M environment that you are going to use. An environment is a combination of an endPoint and token.

An endPoint looks like the following:

Copy

https://<controlmEndPointHost>/automation-api

The Control-M SaaS endpoint host has the following format: <tenant-name>-aapi.<area>.ctmsaas.com

Let's add an environment and name it prodEnvironment.

The command below shows you how to do this. In this command, you specify a valid API token, as described in Authentication Service.

Copy

ctm environment saas::add prodEnvironment "https://<controlmEndPointHost>/automation-api" "<keyToken>"

Step 2: Access the Tutorial SamplesLink copied to clipboard

Go to the directory where the tutorial sample is located:

Copy

cd automation-api-quickstart/helix-control-m/102-automate-code-deployment

Step 3: Deploy to prodEnvironmentLink copied to clipboard

Deploy the code to a specific environment. The "-e" is used to specify a destination environment that differs from the default environment.

The command below shows you how to do this and demonstrates a response:

Copy

> ctm deploy AutomationAPISampleFlow.json -e prodEnvironment
 
[
  {
    "deploymentFile": "AutomationAPISampleFlow.json",
    "successfulFoldersCount": 0,
    "successfulSmartFoldersCount": 1,
    "successfulSubFoldersCount": 0,
    "successfulJobsCount": 2,
    "successfulConnectionProfilesCount": 0,
    "successfulDriversCount": 0,
    "isDeployDescriptorValid": false,
    "deployedFolders": [
      "AutomationAPISampleFlow"
    ]
  }
]

Step 4: Retrieve jobs from prodEnvironment Back to the Development Environment Using Deploy DescriptorLink copied to clipboard

You can now retrieve the jobs from the prodEnvironment back to your Development environment using a Deploy Descriptor.

The following command shows how to retrieve the jobs and folders from the prodEnvironment in a new JSON file named prodEnvironmentJobs.json:

Copy

ctm deploy jobs::get -s "server=*&folder=*" -e prodEnvironment > prodEnvironmentJobs.json

Typically, the two environments (in this case, prodEnvironment and the Development environment named testEnvironment) differ in their resources. Therefore, in the following example, we will modify the Host property value to "devhost" (the host in the Development environment) in any job whose name begins with "Command" or "Script". In addition, we will add a "Dev" prefix to the Application property for any job in the source code. Finally, we will set the RunAs user to "devuser".

Copy

{
  "DeployDescriptor":
  [  
    {
      "Comment": "Set run as user in Defaults to the Dev automation user",
      "ApplyOn": {
                "@":"Defaults"
       },
      "Property" :"RunAs",
      "Assign" : "devuser"
    },   
    {
      "Comment": "Modify Application property to comply with Development environment",
      "Property" :"Application",
      "Replace" : [ {"(.*)" : "Dev$1"} ]
    },
    {
      "Comment": "Distribute jobs across hosts available in Development environment based on job names",
      "Property": "Host",
      "Source": "@",
      "Replace": [
                    { "Command.*" : "devhost"},
                    { "Script.*"  : "devhost"}
      ]
    }
  ]
}

Use the deploy transform command to debug the modifications:

Copy

ctm deploy transform prodEnvironmentJobs.json DeployDescriptor.json -e testEnvironment

The following output is returned. Note that the name of the application now begins with "Dev", and the two hosts are now "devhost":

Copy

{
  "AutomationAPISampleFlow": {
    "Type": "Folder",
    "ControlmServer": "IN01",
    "RunAs": "USERNAME",
    "SubApplication": "SampleSubApp",
    "Application": "DevSampleApp",
    "CommandJob": {
      "Type": "Job:Command",
      "SubApplication": "SampleSubApp",
      "Host": "workbench",
      "RunAs": "USERNAME",
      "Application": "DevSampleApp",
      "Command": "echo my 1st job",
      "When": {
        "WeekDays": [ "MON", "TUE", "WED", "THU", "FRI" ],
        "Months": [ "JAN", "OCT", "DEC" ],
        "MonthDays": [ "1", "11", "22" ],
        "ToTime": "2100",
        "FromTime": "0300"
      },
      "IfBase:Folder:CompletionStatus_0": {
        "Type": "If:CompletionStatus",
        "CompletionStatus": "NOTOK",
        "Mail_0": {
          "Type": "Action:Mail",
          "To": "team@mycomp.com",
          "Message": "%%JOBNAME failed"
        }
      }
    },
    "ScriptJob": {
      "Type": "Job:Script",
      "SubApplication": "SampleSubApp",
      "FileName": "SCRIPT_NAME",
      "Host": "workbench",
      "FilePath": "SCRIPT_PATH",
      "RunAs": "USERNAME",
      "Application": "DevSampleApp",
      "When": {
        "WeekDays": [ "MON", "TUE", "WED", "THU", "FRI" ],
        "Months": [ "JAN", "OCT", "DEC" ],
        "MonthDays": [ "1", "11", "22" ],
        "ToTime": "2100",
        "FromTime": "0300"
      },
      "IfBase:Folder:CompletionStatus_0": {
        "Type": "If:CompletionStatus",
        "CompletionStatus": "NOTOK",
        "Mail_0": {
          "Type": "Action:Mail",
          "To": "team@mycomp.com",
          "Message": "%%JOBNAME failed"
        }
      }
    },
    "Flow": {
      "Type": "Flow",
      "Sequence": [
        "CommandJob",
        "ScriptJob"
      ]
    }
  },
  "JobsRunInDockerSample": {
    "Type": "Folder",
    "ControlmServer": "IN01",
    "RunAs": "controlm",
    "SubApplication": "SampleSubApp",
    "Application": "DevSampleApp",
    "CommandJob": {
      "Type": "Job:Command",
      "SubApplication": "SampleSubApp",
      "Host": "workbench",
      "RunAs": "controlm",
      "Application": "DevSampleApp",
      "Command": "whoami ; pwd; ls -l"
    }
  },
  "AutomationAPIFileTransferDatabaseSampleFlow": {
    "Type": "Folder",
    "Variables": [
      {
        "DestDataFile": "DESTINATION_FILE"
      },
      {
        "SrcDataFile": "SOURCE_FILE"
      }
    ],
    "ControlmServer": "IN01",
    "SubApplication": "SampleSubApp",
    "Application": "DevSampleApp",
    "GetData": {
      "Type": "Job:FileTransfer",
      "ConnectionProfileSrc": "SFTP-CP",
      "ConnectionProfileDest": "Local-CP",
      "SubApplication": "SampleSubApp",
      "Host": "HOST",
      "RunAs": "SFTP-CP+Local-CP",
      "Application": "DevSampleApp",
      "Variables": [
        {
          "DestDataFile": "DESTINATION_FILE"
        },
        {
          "SrcDataFile": "SOURCE_FILE"
        }
      ],
      "FileTransfers": [
        {
          "ABSTIME": "0",
          "VERNUM": "0",
          "Dest": "%%DestDataFile",
          "SRCOPT": "0",
          "TransferType": "Binary",
          "CASEIFS": "0",
          "DSTOPT": "0",
          "RECURSIVE": "0",
          "TransferOption": "SrcToDest",
          "Src": "%%SrcDataFile",
          "TIMELIMIT": "0",
          "EXCLUDE_WILDCARD": "0",
          "NULLFLDS": "0",
          "TRIM": "1",
          "IF_EXIST": "0",
          "UNIQUE": "0",
          "PostCommandDest": {
            "action": "chmod",
            "arg2": "%%DestDataFile",
            "arg1": "700"
          },
          "PreCommandDest": {
            "arg1": "%%DestDataFile",
            "action": "rm"
          }
        }
      ],
      "When": {
        "ToTime": "2100",
        "FromTime": "0300"
      }
    },
    "UpdateRecords": {
      "Type": "Job:Database:SQLScript",
      "SQLScript": "/home/USER/automation-api-quickstart/helix-control-m/101-running-file-transfer-and-database-query-job-flow/processRecords.sql",
      "ConnectionProfile": "DB-CP",
      "SubApplication": "SampleSubApp",
      "Host": "HOST",
      "RunAs": "DB-CP",
      "Application": "DevSampleApp",
      "Variables": [
        {
          "DestDataFile": "DESTINATION_FILE"
        },
        {
          "SrcDataFile": "SOURCE_FILE"
        }
      ],
      "When": {
        "ToTime": "2100",
        "FromTime": "0300"
      }
    },
    "Flow": {
      "Type": "Flow",
      "Sequence": [
        "GetData",
        "UpdateRecords"
      ]
    }
  },
  "JobsRunInDockerSample2": {
    "Type": "Folder",
    "ControlmServer": "IN01",
    "RunAs": "controlm",
    "SubApplication": "SampleSubApp",
    "Application": "DevSampleApp",
    "CommandJob": {
      "Type": "Job:Command",
      "SubApplication": "SampleSubApp",
      "Host": "workbench",
      "RunAs": "controlm",
      "Application": "DevSampleApp",
      "Command": "whoami ; pwd; ls -l"
    }
  }
}

To do the actual deployment to the Development environment, use the following command:

Copy

ctm deploy prodEnvironmentJobs.json DeployDescriptor.json -e testEnvironment

Step 5: Automate DeploymentsLink copied to clipboard

Let's automate the deployment of Control-M object definitions from the source directory to prodEnvironment.

Copy

#!/bin/bash
for f in *.json; do
 echo "Deploying file $f";
 ctm deploy $f -e prodEnvironment;
done

This code can be used in Jenkins to push Git changes to Control-M.

Step 6: Automate Deployments with a Python ScriptLink copied to clipboard

You can automate the deployment of Control-M object definitions from the source directory to prodEnvironment with a Python script using the REST API.

Copy

import requests  # pip install requests if you don't have it already
import urllib3
 
urllib3.disable_warnings() # disable warnings when creating unverified requests
   
endPoint = 'https://<controlmEndPointHost>/automation-api'
token=<token>
   
# -----------------
# Built
uploaded_files = [
        ('definitionsFile', ('Jobs.json', open('c:\\src\Jobs.json', 'rb'), 'application/json'))
]
   
r = requests.post(endPoint + '/deploy', files=uploaded_files, headers={'x-api-key': + token}, verify=False)
   
print(r.content)
print(r.status_code)
   
exit(r.status_code == requests.codes.ok)

Where to Go from HereLink copied to clipboard

To learn more about what you can do with the Control-M Automation API, read through Code Reference and Services.