Application Workflow Connection Profiles

The following topics describe connection profiles for application workflow platforms and services:

ConnectionProfile:Airflow

Airflow enables you to monitor and manage DAG workflows in Control-M. You can monitor DAG executions in the Airflow tab in the Monitoring domain. You can also view the specific details of each task, open the DAG in the Airflow web server user interface, and view XCom variables from the Airflow tab.

The following example shows how to define a connection profile for standalone instance of an Airflow job:

Copy 
{
   "AIRFLOW_CONNECTION_PROFILE": 
   {
      "Type": "ConnectionProfile:Airflow:Standalone",
      "Host": "dba-airflow-12",
      "Port": "8080",
      "User": "admin",
      "Password": "password",
      "SecuredConnection": true,
      "Centralized": true
   }
}

The following table describes the standalone Airflow job connection profile parameters.

Parameter

Description

Host

Defines the Airflow hostname.

Port

Defines the Airflow port number of the Airflow web server.

User

Defines the username that is used to log in to the Airflow server.

Password

Defines the password to log in to the Airflow server.

SecuredConnection

Determines whether a secure connection to the Airflow server is established.

Valid Values:

  • true

  • false

Centralized

Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents.

You must set this parameter to true.

The following examples show how to define a connection profile for a Google Cloud Composer 1.0 instance and a Google Cloud Composer 2.0 instance of Airflow, respectively:

Copy 
{
   "AIRFLOW_GC_CONNECTION_PROFILE": 
   {
      "Type": "ConnectionProfile:Airflow:GoogleComposer",
      "ServiceAccountKey": "{\"type\":\"service_account\",\"project_id\":\"sso-gcp-dba-ctm1-priv-cc30752\",\"private_key_id\":\"5197d05c5b8212bea944985cec74a34d6c1868aa\",\"private_key\":\"-----BEGIN PRIVATE KEY-----\\nprivate-key\\n-----END PRIVATE KEY-----\\n\",\"client_email\":\"bmc-wla-svc-02@sso-gcp-dba-ctm1-priv-cc30752.iam.gserviceaccount.com\",\"client_id\":\"116650586827623521335\",\"auth_uri\":\"https://accounts.google.com/o/oauth2/auth\",\"token_uri\":\"https://oauth2.googleapis.com/token\",  \"auth_provider_x509_cert_url\":\"https://www.googleapis.com/oauth2/v1/certs\",\"client_x509_cert_url\":\"https://www.googleapis.com/robot/v1/metadata/x509/bmc-wla-svc-02%40sso-gcp-dba-ctm1-priv-cc30752.iam.gserviceaccount.com\"}",
      "ServiceAccountKeyFilename": "1234.log",
      "BaseURL": "http://akjsdlksajdksad",
      "TargetAudience": "153767957930-mq2qc9l7p5i8liqvf19d77fku5nrcd1k.apps.googleusercontent.com",
      "Centralized": true
   }
}
Copy 
{
   "AIRFLOW_GC_CONNECTION_PROFILE": 
   {
      "Type": "ConnectionProfile:Airflow:GoogleComposer",
      "ServiceAccountKey": "{\"type\":\"service_account\",\"project_id\":\"sso-gcp-dba-ctm1-priv-cc30752\",\"private_key_id\":\"5197d05c5b8212bea944985cec74a34d6c1868aa\",\"private_key\":\"-----BEGIN PRIVATE KEY-----\\nprivate-key\\n-----END PRIVATE KEY-----\\n\",\"client_email\":\"bmc-wla-svc-02@sso-gcp-dba-ctm1-priv-cc30752.iam.gserviceaccount.com\",\"client_id\":\"116650586827623521335\",\"auth_uri\":\"https://accounts.google.com/o/oauth2/auth\",\"token_uri\":\"https://oauth2.googleapis.com/token\",  \"auth_provider_x509_cert_url\":\"https://www.googleapis.com/oauth2/v1/certs\",\"client_x509_cert_url\":\"https://www.googleapis.com/robot/v1/metadata/x509/bmc-wla-svc-02%40sso-gcp-dba-ctm1-priv-cc30752.iam.gserviceaccount.com\"}",
      "ServiceAccountKeyFilename": "1234.log",
      "BaseURL": "http://akjsdlksajdksad",
      "Centralized": true
   }
}

The following table describes additional Airflow job connection profile parameters.

Parameter

Description

ServiceAccountKey

Defines a JSON body that contains the required service account credentials to access the Google Cloud Composer account.

ServiceAccountKeyFilename

Defines the name of the file that contains the service account credentials.

BaseURL

Defines the Airflow web UI URL in Google Composer.

TargetAudience

Defines the JWT (JSON Web Token) audience for the IAP (Identity-Aware Proxy) Google Cloud version 1.0 service.

If you do not know the target audience value, define any text value and test the connection profile. The error message indicates the required target audience value.

Centralized

Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents.

You must set this parameter to true.

ConnectionProfile:Apache NiFi

Apache NiFi is an open-source tool that automates data flow across systems in real time.

The following examples show how to define a connection profile for an Apache NiFi job:

Copy 
"NFI": 
{
   "Type": "ConnectionProfile:Apache NiFi",
   "NiFi URL": "https://localhost",
   "Port": "8443",
   "Username": "abcd", 
   "Password": "*****",
   "Connection Timeout": "50",
   "Centralized": true
}

The following table describes the Apache NiFi connection profile parameters.

Parameter

Description

NiFi URL

Defines the Apache authentication endpoint for NiFi.

https://localhost

Port

Defines the port number of the Apache NiFi server.

Username

Defines the Apache NiFi server username.

Password

Defines the Apache NiFi server password. You can use Secrets in Code to hide this value in the code.

Connection Timeout

Determines the number of seconds to wait after Control-M initiates a connection request to Apache NiFi before a timeout occurs.

Default: 50 seconds

Centralized

Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents.

You must set this parameter to true.

ConnectionProfile:Astronomer

Astronomer is a workload automation service based on Apache Airflow that enables you to create, schedule, and manage your workflows.

The following example shows how to define a connection profile for an Astronomer job:

Copy 
"Astronomer": 
{
   "Type": "ConnectionProfile:Astronomer",
   "Deployment URL": "https://clybeh1ok01ke01k6wr9szi10.astronomer.run/dns2rtpk",
   "Bearer Token": "eyJhbGciOiJSUzl1NilslmtpZcl6",
   "Connection Timeout": "30",
   "Description": "",
   "Centralized": true
}

The following table describes the Astronomer connection profile parameters.

Parameter

Description

Deployment URL

Defines the Astronomer deployment endpoint URL in the Airflow environment.

https://clybeh1ok01ke01k6wr9szi10.astronomer.run/dns2rtpk

Bearer Token

Defines a security token to authenticate API requests to Astronomer, which allows access to protected resources. You can use Secrets in Code to hide this value in the code.

Connection Timeout

Determines the number of seconds to wait after Control-M initiates a connection request to Astronomer before a timeout occurs.

Default: 30 seconds

Centralized

Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents.

You must set this parameter to true.

ConnectionProfile:AWS Step Functions

AWS Step Functions enables you to create visual workflows that can integrate other AWS services.

The following examples show how to define a connection profile for an AWS Step Functions job:

  • This JSON defines a connection profile for a job based on authentication using an AWS access key and secret:

    Copy 
    {
       "STEPFUNCTIONSCCP": 
       {
          "Type": "ConnectionProfile:AWS Step Functions",
          "Authentication": "SECRET",
          "AWS Secret": "myAwsSecret12345",
          "AWS Access Key": "MYAWSACCESSKEY1234",
          "AWS Region": "eu-west-2",
          "Step Functions URL": "https://states.AWSRegion.amazonaws.com",
          "Connection Timeout": "20",
          "Description": "",
          "Centralized": true
       }
    }

  • This JSON defines a connection profile fora job based on authentication using an AWS IAM role from inside an EC2 instance:

    Copy 
    {
       "STEPFUNCTIONSCCP": 
       {
          "Type": "ConnectionProfile:AWS Step Functions",
          "Authentication": "NOSECRET",
          "IAM Role": "STEPFXIAMROLE",
          "AWS Region": "eu-west-2",
          "Step Functions URL": "https://states.AWSRegion.amazonaws.com",
          "Connection Timeout": "20",
          "Description": "",
          "Centralized": true
       }
    }

The following table describes the AWS Step Functions job connection profile parameters.

Parameter

Description

Authentication

Determines one of the following types of authentication for the connection with AWS Step Functions:

  • SECRET: Authentication using an AWS access key and secret.

  • NOSECRET: Authentication using an AWS IAM role from within the AWS infrastructure.

IAM roles (NOSECRET) is currently not supported for this plug-in on ECS container instances and for EKS services.

AWS Access Key

(SECRET Authentication) Defines the access key assigned to the account in the AWS Step Functions integration setup.

AWS Secret

(SECRET Authentication) Defines the secret access key assigned to the account in the AWS Step Functions integration setup. You can use Secrets in Code to hide this value in the code.

IAM Role

(NOSECRET Authentication) Defines the Identity and Access Management (IAM) role for connection to AWS Step Functions.

AWS Region

Determines the region where the AWS Step Functions jobs are located.

Step Functions URL

Determines the authentication endpoint for AWS Step Functions, based on the following format:

https://states.{{region}}.amazonaws.com

For more information about regional endpoints available for the AWS Step Functions service, refer to the AWS documentation.

Connection Timeout

Determines the number of seconds to wait after Control-M initiates a connection request to AWS Step Functions before a timeout occurs.

Default: 20

Centralized

Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents.

You must set this parameter to true.

ConnectionProfile:Azure Logic Apps

Azure Logic Apps enables you to design and automate cloud-based workflows and integrations.

The following examples show how to define a connection profile for an Azure Logic Apps job:

  • This JSON defines a connection profile that authenticates with an Azure Service Principal identity:

    Copy 
    {
        "AZURE_LOGIC_APPS": 
        {
           "Type": "ConnectionProfile:Azure Logic Apps",
           "Subscription ID": "e76056e0-70de-4da8-b02e-61263a150b1f", 
           "Authentication Method": "PRINCIPAL",
           "Resource Group": "tb-resourcegroup", 
           "Tenant ID": "92b796c5-5839-40a6-8dd9-c1fad320c69b", 
           "Azure Login url": "https://login.microsoftonline.com", 
           "Application ID": "7f477fa3-1a1f-4877-ba80-f39bb563f1b5", 
           "Client Secret": "*****", 
           "Connection timeout": "20", 
           "Description": "",
           "Centralized": true
        }
    }

  • This JSON defines a connection profile that authenticates with an Azure Managed Identity:

    Managed Identity authentication is based on an Azure token that is valid, by default, for 24 hours. Token lifetime can be extended by Azure.

    Copy 
    {
        "AZURE_LOGIC_APPS": 
        {
           "Type": "ConnectionProfile:Azure Logic Apps",
           "Subscription ID": "e76056e0-70de-4da8-b02e-61263a150b1f", 
           "Authentication Method": "MANAGEDID",
           "Resource Group": "tb-resourcegroup", 
           "Specify Managed Identity Client ID": "&client_id=",
           "Managed Identity Client ID": "72d448f0-ac32-45ea-9158-f8653e4ee16",  
           "Connection timeout": "20", 
            "Description": "",
           "Centralized": true
        }
    }

The following table describes the Azure Logic Apps job connection profile parameters.

Parameter

Description

Subscription ID

Defines the Azure account subscription ID, which is located in the Azure portal.

Authentication Method

Defines one of the following types of authentication to use for the connection with Azure Logic Apps:

  • PRINCIPAL: Authentication using a service principal.

  • MANAGEDID: Authentication using a managed identity.

Resource Group

Determines the resource group where your logic app is located.

Tenant ID

(Service Principal) Defines the Azure Tenant ID in Azure AD.

Azure Login url

(Service Principal) Defines the Azure AD authentication endpoint base URL.

Application ID

(Service Principal) Defines the application ID of the registered application.

The service principal must be an Azure Logic Apps workspace user with a Contributor or Owner role.

Client Secret

(Service Principal) Defines the password associated with the Azure user and the application. You can use Secrets in Code to hide this value in the code.

Specify Managed Identity Client ID

(Managed Identity) Determines whether the client ID for your Managed Identity is specified by the Managed Identity Client ID parameter.

Include this parameter only if you are using the Managed Identity authentication method and you have multiple Managed Identities defined on your Azure virtual machine. Set its value to &client_id=.

Managed Identity Client ID

(Managed Identity) Determines which client ID to use as the Managed Identity.

This parameter requires a value only if you have multiple Managed Identities defined on your Azure virtual machine and you included the Specify Managed Identity Client ID parameter.

If you have only one Managed Identity, it is detected automatically.

Connection Timeout

Determines the number of seconds to wait before a timeout occurs after Control-M initiates a request to Azure Logic Apps.

Default: 50

Centralized

Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents.

You must set this parameter to true.

ConnectionProfile:GCP Composer

Google Cloud (GCP) Composer is a managed workflow orchestration service built on Apache Airflow that enables you to automate workflow tasks.

The following example shows how to define a connection profile for a GCP Composer job:

Copy 
"GCP Composer": 
{
   "Type": "ConnectionProfile:GCPComposer",
   "Composer URL": " https://c2b6dd3a789740c087b09992a7df889d-example.composer.googleusercontent.com",
   "Identity Type": ""Identity Type": "Service Account",
   "Service Account Key": "<service Account Key>",
   "Connection Timeout": "30",
   "Description": "",
   "Centralized": true
}

The following table describes the GCP Composer job connection profile parameters.

Parameter

Description

Composer URL

Defines the GCP authentication endpoint for GCP Composer.

https://c2b6dd3a789740c087b09992a7df889d-example.composer.googleusercontent.com
Identity Type

Determines one of the following authentication types using GCP Access Control:

  • Service Account: Authenticates using an application ID (service account) and client secret.

  • IAM: Authenticates based on a detected IAM role, which removes the need to provide additional credentials.

    IAM is available on GCP VMs only.

Service Account Key

(Service Account) Defines a service account that is associated with an RSA key pair.

Connection Timeout

Determines the number of seconds to wait after Control-M initiates a connection request to GCP Composer before a timeout occurs.

Default: 30

Centralized

Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents.

You must set this parameter to true.

ConnectionProfile:GCP Workflows

GCP Workflows enables you to design and automate cloud-based workflows and integrations.

The following example shows how to define a connection profile for a GCP Workflows job:

Copy 
{
   "GCPWF": 
   {
      "Type": "ConnectionProfile:GCP Workflows",
      "Identity Type": "service_account",
      "GCP API URL": "https://workflowexecutions.googleapis.com",
      "Service Account Key": "*****",
      "Connection Timeout": "20",
      "Description": "",
      "Centralized": true
   }
}

The following table describes the GCP Workflows job connection profile parameters.

Parameter

Description

Identity Type

Determines one of the following authentication types using GCP Access Control:

  • service_account: Authenticates using an application ID (service account) and client secret.

  • os_user: Authenticates based on a detected IAM role, which removes the need to provide additional credentials.

GCP API URL

(Service Account) Defines the Google Cloud Platform (GCP) authentication endpoint for GCP Workflows.

Default: https://workflowexecutions.googleapis.com

Service Account Key

(Service Account) Defines a service account that is associated with an RSA key pair.

Connection Timeout

Determines the number of seconds to wait after Control-M initiates a connection request to GCP Workflows before a timeout occurs.

Default:: 20

Centralized

Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents.

You must set this parameter to true.