Automating Code Deployment

This tutorial teaches you how to use a script to automate DevOps tasks. In order to complete this tutorial, you need a valid Control-M endPoint and API token.

Before you begin

Ensure that you meet the following prerequisites:

You have successfully completed API setup, as described in Setting Up the API.
You have Git installed. If not, obtain it from the Git Downloads page.
You have local copies of the tutorial samples from GitHub and a local copy of the source code using the git clone command:
Copy
```
git clone https://github.com/controlm/automation-api-quickstart.git
```

Step 1: Set the Control-M Environment

The first task when starting to work with Control-M Automation API is to configure the Control-M environment that you are going to use. An environment is a combination of an endPoint and token.

An endPoint looks like the following:

Copy

https://<controlmEndPointHost>/automation-api

The Helix Control-M endpoint host has the following format: <tenant-name>-aapi.<area>.ctmsaas.com

Let's add an environment and name it prodEnvironment.

The command below shows you how to do this. In this command, you specify a valid API token, as described in Authentication Service.

Copy

ctm environment saas::add prodEnvironment "https://<controlmEndPointHost>/automation-api" "<keyToken>"

Step 2: Access the Tutorial Samples

Go to the directory where the tutorial sample is located:

Copy

cd automation-api-quickstart/helix-control-m/102-automate-code-deployment

Step 3: Deploy to prodEnvironment

Deploy the code to a specific environment. The "-e" is used to specify a destination environment that differs from the default environment.

The command below shows you how to do this and demonstrates a response:

Copy

> ctm deploy AutomationAPISampleFlow.json -e prodEnvironment
 
[
  {
    "deploymentFile": "AutomationAPISampleFlow.json",
    "successfulFoldersCount": 0,
    "successfulSmartFoldersCount": 1,
    "successfulSubFoldersCount": 0,
    "successfulJobsCount": 2,
    "successfulConnectionProfilesCount": 0,
    "successfulDriversCount": 0,
    "isDeployDescriptorValid": false,
    "deployedFolders": [
      "AutomationAPISampleFlow"
    ]
  }
]

Step 4: Retrieve jobs from prodEnvironment Back to the Development Environment Using Deploy Descriptor

You can now retrieve the jobs from the prodEnvironment back to your Development environment using a Deploy Descriptor.

The following command shows how to retrieve the jobs and folders from the prodEnvironment in a new JSON file named prodEnvironmentJobs.json:

Copy

ctm deploy jobs::get -s "server=*&folder=*" -e prodEnvironment > prodEnvironmentJobs.json

Typically, the two environments (in this case, prodEnvironment and the Development environment named testEnvironment) differ in their resources. Therefore, in the following example, we will modify the Host property value to "devhost" (the host in the Development environment) in any job whose name begins with "Command" or "Script". In addition, we will add a "Dev" prefix to the Application property for any job in the source code. Finally, we will set the RunAs user to "devuser".

Copy

{
  "DeployDescriptor":
  [  
    {
      "Comment": "Set run as user in Defaults to the Dev automation user",
      "ApplyOn": {
                "@":"Defaults"
       },
      "Property" :"RunAs",
      "Assign" : "devuser"
    },   
    {
      "Comment": "Modify Application property to comply with Development environment",
      "Property" :"Application",
      "Replace" : [ {"(.*)" : "Dev$1"} ]
    },
    {
      "Comment": "Distribute jobs across hosts available in Development environment based on job names",
      "Property": "Host",
      "Source": "@",
      "Replace": [
                    { "Command.*" : "devhost"},
                    { "Script.*"  : "devhost"}
      ]
    }
  ]
}

Use the deploy transform command to debug the modifications:

Copy

ctm deploy transform prodEnvironmentJobs.json DeployDescriptor.json -e testEnvironment

The following output is returned. Note that the name of the application now begins with "Dev", and the two hosts are now "devhost":

Copy

{
  "AutomationAPISampleFlow": {
    "Type": "Folder",
    "ControlmServer": "IN01",
    "RunAs": "USERNAME",
    "SubApplication": "SampleSubApp",
    "Application": "DevSampleApp",
    "CommandJob": {
      "Type": "Job:Command",
      "SubApplication": "SampleSubApp",
      "Host": "workbench",
      "RunAs": "USERNAME",
      "Application": "DevSampleApp",
      "Command": "echo my 1st job",
      "When": {
        "WeekDays": [ "MON", "TUE", "WED", "THU", "FRI" ],
        "Months": [ "JAN", "OCT", "DEC" ],
        "MonthDays": [ "1", "11", "22" ],
        "ToTime": "2100",
        "FromTime": "0300"
      },
      "IfBase:Folder:CompletionStatus_0": {
        "Type": "If:CompletionStatus",
        "CompletionStatus": "NOTOK",
        "Mail_0": {
          "Type": "Action:Mail",
          "To": "[email protected]",
          "Message": "%%JOBNAME failed"
        }
      }
    },
    "ScriptJob": {
      "Type": "Job:Script",
      "SubApplication": "SampleSubApp",
      "FileName": "SCRIPT_NAME",
      "Host": "workbench",
      "FilePath": "SCRIPT_PATH",
      "RunAs": "USERNAME",
      "Application": "DevSampleApp",
      "When": {
        "WeekDays": [ "MON", "TUE", "WED", "THU", "FRI" ],
        "Months": [ "JAN", "OCT", "DEC" ],
        "MonthDays": [ "1", "11", "22" ],
        "ToTime": "2100",
        "FromTime": "0300"
      },
      "IfBase:Folder:CompletionStatus_0": {
        "Type": "If:CompletionStatus",
        "CompletionStatus": "NOTOK",
        "Mail_0": {
          "Type": "Action:Mail",
          "To": "[email protected]",
          "Message": "%%JOBNAME failed"
        }
      }
    },
    "Flow": {
      "Type": "Flow",
      "Sequence": [
        "CommandJob",
        "ScriptJob"
      ]
    }
  },
  "JobsRunInDockerSample": {
    "Type": "Folder",
    "ControlmServer": "IN01",
    "RunAs": "controlm",
    "SubApplication": "SampleSubApp",
    "Application": "DevSampleApp",
    "CommandJob": {
      "Type": "Job:Command",
      "SubApplication": "SampleSubApp",
      "Host": "workbench",
      "RunAs": "controlm",
      "Application": "DevSampleApp",
      "Command": "whoami ; pwd; ls -l"
    }
  },
  "AutomationAPIFileTransferDatabaseSampleFlow": {
    "Type": "Folder",
    "Variables": [
      {
        "DestDataFile": "DESTINATION_FILE"
      },
      {
        "SrcDataFile": "SOURCE_FILE"
      }
    ],
    "ControlmServer": "IN01",
    "SubApplication": "SampleSubApp",
    "Application": "DevSampleApp",
    "GetData": {
      "Type": "Job:FileTransfer",
      "ConnectionProfileSrc": "SFTP-CP",
      "ConnectionProfileDest": "Local-CP",
      "SubApplication": "SampleSubApp",
      "Host": "HOST",
      "RunAs": "SFTP-CP+Local-CP",
      "Application": "DevSampleApp",
      "Variables": [
        {
          "DestDataFile": "DESTINATION_FILE"
        },
        {
          "SrcDataFile": "SOURCE_FILE"
        }
      ],
      "FileTransfers": [
        {
          "ABSTIME": "0",
          "VERNUM": "0",
          "Dest": "%%DestDataFile",
          "SRCOPT": "0",
          "TransferType": "Binary",
          "CASEIFS": "0",
          "DSTOPT": "0",
          "RECURSIVE": "0",
          "TransferOption": "SrcToDest",
          "Src": "%%SrcDataFile",
          "TIMELIMIT": "0",
          "EXCLUDE_WILDCARD": "0",
          "NULLFLDS": "0",
          "TRIM": "1",
          "IF_EXIST": "0",
          "UNIQUE": "0",
          "PostCommandDest": {
            "action": "chmod",
            "arg2": "%%DestDataFile",
            "arg1": "700"
          },
          "PreCommandDest": {
            "arg1": "%%DestDataFile",
            "action": "rm"
          }
        }
      ],
      "When": {
        "ToTime": "2100",
        "FromTime": "0300"
      }
    },
    "UpdateRecords": {
      "Type": "Job:Database:SQLScript",
      "SQLScript": "/home/USER/automation-api-quickstart/helix-control-m/101-running-file-transfer-and-database-query-job-flow/processRecords.sql",
      "ConnectionProfile": "DB-CP",
      "SubApplication": "SampleSubApp",
      "Host": "HOST",
      "RunAs": "DB-CP",
      "Application": "DevSampleApp",
      "Variables": [
        {
          "DestDataFile": "DESTINATION_FILE"
        },
        {
          "SrcDataFile": "SOURCE_FILE"
        }
      ],
      "When": {
        "ToTime": "2100",
        "FromTime": "0300"
      }
    },
    "Flow": {
      "Type": "Flow",
      "Sequence": [
        "GetData",
        "UpdateRecords"
      ]
    }
  },
  "JobsRunInDockerSample2": {
    "Type": "Folder",
    "ControlmServer": "IN01",
    "RunAs": "controlm",
    "SubApplication": "SampleSubApp",
    "Application": "DevSampleApp",
    "CommandJob": {
      "Type": "Job:Command",
      "SubApplication": "SampleSubApp",
      "Host": "workbench",
      "RunAs": "controlm",
      "Application": "DevSampleApp",
      "Command": "whoami ; pwd; ls -l"
    }
  }
}

To do the actual deployment to the Development environment, use the following command:

Copy

ctm deploy prodEnvironmentJobs.json DeployDescriptor.json -e testEnvironment

Step 5: Automate Deployments

Let's automate the deployment of Control-M object definitions from the source directory to prodEnvironment.

Copy

#!/bin/bash
for f in *.json; do
 echo "Deploying file $f";
 ctm deploy $f -e prodEnvironment;
done

This code can be used in Jenkins to push Git changes to Control-M.

Step 6: Automate Deployments with a Python Script

You can automate the deployment of Control-M object definitions from the source directory to prodEnvironment with a Python script using the REST API.

Copy

import requests  # pip install requests if you don't have it already
import urllib3
 
urllib3.disable_warnings() # disable warnings when creating unverified requests
   
endPoint = 'https://<controlmEndPointHost>/automation-api'
token=<token>
   
# -----------------
# Built
uploaded_files = [
        ('definitionsFile', ('Jobs.json', open('c:\\src\Jobs.json', 'rb'), 'application/json'))
]
   
r = requests.post(endPoint + '/deploy', files=uploaded_files, headers={'x-api-key': + token}, verify=False)
   
print(r.content)
print(r.status_code)
   
exit(r.status_code == requests.codes.ok)

Where to Go from Here

To learn more about what you can do with the Control-M Automation API, read through Code Reference and Services.