Application Workflow Connection Profiles

The following topics describe connection profiles for application workflow platforms and services:

ConnectionProfile:Airflow
ConnectionProfile:Apache Airflow
ConnectionProfile:Apache NiFi
ConnectionProfile:Astronomer
ConnectionProfile:AWS MWAA
ConnectionProfile:AWS Step Functions
ConnectionProfile:Azure Logic Apps
ConnectionProfile:GCP Composer
ConnectionProfile:GCP Workflows

ConnectionProfile:Airflow Link copied to clipboard

Airflow enables you to monitor and manage DAG workflows in Control-M. You can monitor DAG executions in the Airflow tab in the Monitoring domain. You can also view the specific details of each task, open the DAG in the Airflow web server user interface, and view XCom variables from the Airflow tab.

The following example shows how to define a connection profile for standalone instance of an Airflow job:

Copy

{
   "AIRFLOW_CONNECTION_PROFILE": 
   {
      "Type": "ConnectionProfile:Airflow:Standalone",
      "Host": "dba-airflow-12",
      "Port": "8080",
      "User": "admin",
      "Password": "password",
      "SecuredConnection": true,
      "Centralized": true
   }
}

The following table describes the standalone Airflow job connection profile parameters.

Parameter	Description
Host	Defines the Airflow hostname.
Port	Defines the Airflow port number of the Airflow web server.
User	Defines the username that is used to log in to the Airflow server.
Password	Defines the password to log in to the Airflow server.
SecuredConnection	Determines whether a secure connection to the Airflow server is established. Valid Values: true false
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.

The following examples show how to define a connection profile for a Google Cloud Composer 1.0 instance and a Google Cloud Composer 2.0 instance of Airflow, respectively:

Copy

{
   "AIRFLOW_GC_CONNECTION_PROFILE": 
   {
      "Type": "ConnectionProfile:Airflow:GoogleComposer",
      "ServiceAccountKey": "{\"type\":\"service_account\",\"project_id\":\"sso-gcp-dba-ctm1-priv-cc30752\",\"private_key_id\":\"5197d05c5b8212bea944985cec74a34d6c1868aa\",\"private_key\":\"-----BEGIN PRIVATE KEY-----\\nprivate-key\\n-----END PRIVATE KEY-----\\n\",\"client_email\":\"bmc-wla-svc-02@sso-gcp-dba-ctm1-priv-cc30752.iam.gserviceaccount.com\",\"client_id\":\"116650586827623521335\",\"auth_uri\":\"https://accounts.google.com/o/oauth2/auth\",\"token_uri\":\"https://oauth2.googleapis.com/token\",  \"auth_provider_x509_cert_url\":\"https://www.googleapis.com/oauth2/v1/certs\",\"client_x509_cert_url\":\"https://www.googleapis.com/robot/v1/metadata/x509/bmc-wla-svc-02%40sso-gcp-dba-ctm1-priv-cc30752.iam.gserviceaccount.com\"}",
      "ServiceAccountKeyFilename": "1234.log",
      "BaseURL": "http://akjsdlksajdksad",
      "TargetAudience": "153767957930-mq2qc9l7p5i8liqvf19d77fku5nrcd1k.apps.googleusercontent.com",
      "Centralized": true
   }
}

Copy

{
   "AIRFLOW_GC_CONNECTION_PROFILE": 
   {
      "Type": "ConnectionProfile:Airflow:GoogleComposer",
      "ServiceAccountKey": "{\"type\":\"service_account\",\"project_id\":\"sso-gcp-dba-ctm1-priv-cc30752\",\"private_key_id\":\"5197d05c5b8212bea944985cec74a34d6c1868aa\",\"private_key\":\"-----BEGIN PRIVATE KEY-----\\nprivate-key\\n-----END PRIVATE KEY-----\\n\",\"client_email\":\"bmc-wla-svc-02@sso-gcp-dba-ctm1-priv-cc30752.iam.gserviceaccount.com\",\"client_id\":\"116650586827623521335\",\"auth_uri\":\"https://accounts.google.com/o/oauth2/auth\",\"token_uri\":\"https://oauth2.googleapis.com/token\",  \"auth_provider_x509_cert_url\":\"https://www.googleapis.com/oauth2/v1/certs\",\"client_x509_cert_url\":\"https://www.googleapis.com/robot/v1/metadata/x509/bmc-wla-svc-02%40sso-gcp-dba-ctm1-priv-cc30752.iam.gserviceaccount.com\"}",
      "ServiceAccountKeyFilename": "1234.log",
      "BaseURL": "http://akjsdlksajdksad",
      "Centralized": true
   }
}

The following table describes additional Airflow job connection profile parameters.

Parameter	Description
ServiceAccountKey	Defines a JSON body that contains the required service account credentials to access the Google Cloud Composer account.
ServiceAccountKeyFilename	Defines the name of the file that contains the service account credentials.
BaseURL	Defines the Airflow web UI URL in Google Composer.
TargetAudience	Defines the JWT (JSON Web Token) audience for the IAP (Identity-Aware Proxy) Google Cloud version 1.0 service. If you do not know the target audience value, define any text value and test the connection profile. The error message indicates the required target audience value.
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.

ConnectionProfile:Apache AirflowLink copied to clipboard

Apache Airflow enables you to create, schedule, and monitor complex data processing and analytics pipelines. It provides an environment to define, manage, and execute workflows as Directed Acyclic Graphs (DAGs) to control task dependencies and execution order.

For more information about this plug-in, see Control-M for Apache Airflow.

The following example shows how to define a connection profile for an Apache Airflow job:

Copy

"Apache Airflow": 
{
   "Type": "ConnectionProfile:Apache Airflow",
   "Airflow URL": "https://localhost",
   "Username": "abcd", 
   "Password": "*****",
   "Connection Timeout": "50",
   "Centralized": true
}

The following table describes the Apache Airflow connection profile parameters.

Parameter	Description
Airflow URL	Defines the Apache authentication endpoint for Airflow. https://localhost
Username	Defines the username that is used to log in to the Airflow server.
Password	Defines the password to log in to the Airflow server.
Connection Timeout	Determines the number of seconds to wait after Control-M initiates a connection request before a timeout occurs. Default: 60 seconds
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.

ConnectionProfile:Apache NiFiLink copied to clipboard

Apache NiFi is an open-source tool that automates data flow across systems in real time.

For more information about this plug-in, see Control-M for Apache NiFi.

The following example shows how to define a connection profile for an Apache NiFi job:

Copy

"NFI": 
{
   "Type": "ConnectionProfile:Apache NiFi",
   "NiFi URL": "https://localhost",
   "Port": "8443",
   "Username": "abcd", 
   "Password": "*****",
   "Connection Timeout": "50",
   "Centralized": true
}

The following table describes the Apache NiFi connection profile parameters.

Parameter	Description
NiFi URL	Defines the Apache authentication endpoint for NiFi. https://localhost
Port	Defines the port number of the Apache NiFi server.
Username	Defines the Apache NiFi server username.
Password	Defines the Apache NiFi server password. You can use Secrets in Code to hide this value in the code.
Connection Timeout	Determines the number of seconds to wait after Control-M initiates a connection request before a timeout occurs. Default: 50 seconds
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.

ConnectionProfile:AstronomerLink copied to clipboard

Astronomer is a workload automation service based on Apache Airflow that enables you to create, schedule, and manage your workflows.

For more information about this plug-in, see Control-M for Astronomer.

The following example shows how to define a connection profile for an Astronomer job:

Copy

"Astronomer": 
{
   "Type": "ConnectionProfile:Astronomer",
   "Deployment URL": "https://clybeh1ok01ke01k6wr9szi10.astronomer.run/dns2rtpk",
   "Bearer Token": "eyJhbGciOiJSUzl1NilslmtpZcl6",
   "Connection Timeout": "30",
   "Description": "",
   "Centralized": true
}

The following table describes the Astronomer connection profile parameters.

Parameter	Description
Deployment URL	Defines the Astronomer deployment endpoint URL in the Airflow environment. https://clybeh1ok01ke01k6wr9szi10.astronomer.run/dns2rtpk
Bearer Token	Defines a security token to authenticate API requests to Astronomer, which allows access to protected resources. You can use Secrets in Code to hide this value in the code.
Connection Timeout	Determines the number of seconds to wait after Control-M initiates a connection request before a timeout occurs. Default: 30 seconds
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.

ConnectionProfile:AWS MWAALink copied to clipboard

AWS Managed Workflows for Apache Airflow (MWAA) is an orchestration service built on Apache Airflow, designed to create, schedule, and monitor data pipelines and workflows.

For more information about this plug-in, see Control-M for Amazon MWAA.

The following examples show how to define a connection profile for an AWS MWAA job:

This JSON defines a connection profile for a job based on authentication using an AWS access key and secret:

Copy

{
   "AMW": 
   {
      "Type": "ConnectionProfile:AWS MWAA",
      "Authentication Method": "Secret",
      "AWS Secret": "*****",
      "AWS Access Key": "MYAWSACCESSKEY1234",
      "AWS Region": "eu-west-2",
      "AWS MWAA URL": "https://env.airflow.AwsRegion.amazonaws.com",
      "Connection Timeout": "30",
      "Description": "",
      "Centralized": true
   }
}

This JSON defines a connection profile fora job based on authentication using an AWS IAM role from inside an EC2 instance:

Copy

{
   "AMW": 
   {
      "Type": "ConnectionProfile:AWS MWAA",
      "Authentication Method": "IAM",
      "IAM Role": "<IAMRole>",
      "AWS Region": "eu-west-2",
      "AWS MWAA URL": "https://env.airflow.AwsRegion.amazonaws.com",
      "Connection Timeout": "30",
      "Description": "",
      "Centralized": true
   }
}

The following table describes the AWS MWAA job connection profile parameters.

Parameter	Description
Authentication Method	Determines one of the following authentication methods: Secret: Authenticates with an AWS access key and secret, which are used for services outside the AWS infrastructure. IAM: Authenticates with an AWS IAM role from within the AWS infrastructure, which removes the need to provide additional credentials.
AWS Secret	(Secret Authentication) Defines the secret access key assigned to the account with the relevant permissions to the AWS service. You can use Secrets in Code to hide this value in the code.
AWS Access Key	(Secret Authentication) Defines the access key assigned to the account with the relevant permissions to the AWS service.
IAM Role	(IAM Authentication) Defines the Identity and Access Management (IAM) role name for the AWS service connection.
AWS Region	Determines the AWS region where the job is located. us-east-2
AWS MWAA URL	Determines the authentication endpoint for Amazon MWAA, based on the following format: https://states.<region name>.amazonaws.com https://states.eu-east-1.amazonaws.com
Connection Timeout	Determines the number of seconds to wait after Control-M initiates a connection request before a timeout occurs. Default: 60
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.

ConnectionProfile:AWS Step FunctionsLink copied to clipboard

AWS Step Functions enables you to create visual workflows that can integrate other AWS services.

For more information about this plug-in, see Control-M for AWS Step Functions.

The following examples show how to define a connection profile for an AWS Step Functions job:

This JSON defines a connection profile for a job based on authentication using an AWS access key and secret:

Copy

{
   "STEPFUNCTIONSCCP": 
   {
      "Type": "ConnectionProfile:AWS Step Functions",
      "Authentication": "SECRET",
      "AWS Secret": "myAwsSecret12345",
      "AWS Access Key": "MYAWSACCESSKEY1234",
      "AWS Region": "eu-west-2",
      "Step Functions URL": "https://states.AWSRegion.amazonaws.com",
      "Connection Timeout": "20",
      "Description": "",
      "Centralized": true
   }
}

This JSON defines a connection profile fora job based on authentication using an AWS IAM role from inside an EC2 instance:

Copy

{
   "STEPFUNCTIONSCCP": 
   {
      "Type": "ConnectionProfile:AWS Step Functions",
      "Authentication": "NOSECRET",
      "IAM Role": "STEPFXIAMROLE",
      "AWS Region": "eu-west-2",
      "Step Functions URL": "https://states.AWSRegion.amazonaws.com",
      "Connection Timeout": "20",
      "Description": "",
      "Centralized": true
   }
}

The following table describes the AWS Step Functions job connection profile parameters.

Parameter	Description
Authentication	Determines one of the following authentication methods: SECRET: Authenticates with an AWS access key and secret, which are used by services outside the AWS infrastructure. NOSECRET: Authenticates with an AWS IAM role from within the AWS infrastructure, which removes the need to provide additional credentials. Default: SECRET IAM roles (NOSECRET) is currently not supported for this plug-in on ECS container instances and for EKS services.
AWS Access Key	(SECRET Authentication) Defines the access key assigned to the account with the relevant permissions to the AWS service.
AWS Secret	(SECRET Authentication) Defines the secret access key assigned to the account with the relevant permissions to the AWS service. Defines the secret access key assigned to the account with the relevant permissions to the AWS service.
IAM Role	(NOSECRET Authentication) Defines the Identity and Access Management (IAM) role name for the AWS service connection.
AWS Region	Determines the AWS region where the job is located. us-east-2
Step Functions URL	Determines the authentication endpoint for AWS Step Functions, based on the following format: https://states.{{region}}.amazonaws.com For more information about regional endpoints available for the AWS Step Functions service, refer to the AWS documentation.
Connection Timeout	Determines the number of seconds to wait after Control-M initiates a connection request before a timeout occurs. Default: 20
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.

ConnectionProfile:Azure Logic AppsLink copied to clipboard

Azure Logic Apps enables you to design and automate cloud-based workflows and integrations.

For more information about this plug-in, see Control-M for Azure Logic Apps.

The following examples show how to define a connection profile for an Azure Logic Apps job:

This JSON defines a connection profile that authenticates with an Azure Service Principal identity:

Copy

{
    "AZURE_LOGIC_APPS": 
    {
       "Type": "ConnectionProfile:Azure Logic Apps",
       "Subscription ID": "e76056e0-70de-4da8-b02e-61263a150b1f", 
       "Authentication Method": "PRINCIPAL",
       "Resource Group": "tb-resourcegroup", 
       "Tenant ID": "92b796c5-5839-40a6-8dd9-c1fad320c69b", 
       "Azure Login url": "https://login.microsoftonline.com", 
       "Application ID": "7f477fa3-1a1f-4877-ba80-f39bb563f1b5", 
       "Client Secret": "*****", 
       "Connection timeout": "20", 
       "Description": "",
       "Centralized": true
    }
}

This JSON defines a connection profile that authenticates with an Azure Managed Identity:

Managed Identity authentication is based on an Azure token that is valid, by default, for 24 hours. Token lifetime can be extended by Azure.

Copy

{
    "AZURE_LOGIC_APPS": 
    {
       "Type": "ConnectionProfile:Azure Logic Apps",
       "Subscription ID": "e76056e0-70de-4da8-b02e-61263a150b1f", 
       "Authentication Method": "MANAGEDID",
       "Resource Group": "tb-resourcegroup", 
       "Specify Managed Identity Client ID": "&client_id=",
       "Managed Identity Client ID": "72d448f0-ac32-45ea-9158-f8653e4ee16",  
       "Connection timeout": "20", 
        "Description": "",
       "Centralized": true
    }
}

The following table describes the Azure Logic Apps job connection profile parameters.

Parameter	Description
Subscription ID	Defines the Azure account ID of your organization. You can retrieve the subscription ID from the Subscription menu in the Azure portal.
Authentication Method	Determines one of the following authentication methods: PRINCIPAL: Access protected Azure services and resources based on roles assigned to the Service Principal by the Azure administrator. The Service Principal is also known as an App Registration. Use this method if the Control-M/Agent is installed on-premises or with another (non-Azure) cloud vendor. MANAGEDID: Access protected Azure services and resources using a key created and managed by the Azure platform, without login credentials. Use this method if the Control-M/Agent is installed on an Azure virtual machine that has a Managed Identity with the required permissions.
Resource Group	Determines the resource group where your logic app is located.
Tenant ID	(Service Principal) Defines the Azure Tenant ID in Azure AD.
Azure Login url	(Service Principal) Defines the Azure AD authentication endpoint base URL.
Application ID	(Service Principal) Defines the application ID of the registered application. The service principal must be an Azure Logic Apps workspace user with a Contributor or Owner role.
Client Secret	(Service Principal) Defines the password associated with the Azure user and the application. You can use Secrets in Code to hide this value in the code.
Specify Managed Identity Client ID	(Managed Identity) Determines whether the client ID for your Managed Identity is specified by the Managed Identity Client ID parameter. Include this parameter only if you are using the Managed Identity authentication method and you have multiple Managed Identities defined on your Azure virtual machine. Set its value to &client_id=.
Managed Identity Client ID	(Managed Identity) Determines which client ID to use as the Managed Identity. This parameter requires a value only if you have multiple Managed Identities defined on your Azure virtual machine and you included the Specify Managed Identity Client ID parameter. If you have only one Managed Identity, it is detected automatically.
Connection Timeout	Determines the number of seconds to wait after Control-M initiates a connection request before a timeout occurs. Default: 50
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.

ConnectionProfile:GCP Composer Link copied to clipboard

Google Cloud (GCP) Composer is a managed workflow orchestration service built on Apache Airflow that enables you to automate workflow tasks.

For more information about this plug-in, see Control-M for GCP Composer.

The following example shows how to define a connection profile for a GCP Composer job:

Copy

"GCP Composer": 
{
   "Type": "ConnectionProfile:GCPComposer",
   "Composer URL": " https://c2b6dd3a789740c087b09992a7df889d-example.composer.googleusercontent.com",
   "Identity Type": ""Identity Type": "Service Account",
   "Service Account Key": "<service Account Key>",
   "Connection Timeout": "30",
   "Description": "",
   "Centralized": true
}

The following table describes the GCP Composer job connection profile parameters.

Parameter	Description
Composer URL	Defines the GCP authentication endpoint for GCP Composer. https://c2b6dd3a789740c087b09992a7df889d-example.composer.googleusercontent.com
Identity Type	Determines one of the following authentication types that utilize GCP Access Control: Service Account: Authenticates with an application ID (service account) and client secret. IAM: Authenticates based on a detected IAM role, which removes the need to provide additional credentials. IAM is available on GCP VMs only.
Service Account Key	(Service Account) Defines a service account that is associated with an RSA key pair.
Connection Timeout	Determines the number of seconds to wait after Control-M initiates a connection request before a timeout occurs. Default: 30
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.

ConnectionProfile:GCP WorkflowsLink copied to clipboard

GCP Workflows enables you to design and automate cloud-based workflows and integrations.

For more information about this plug-in, see Control-M for GCP Workflows.

The following example shows how to define a connection profile for a GCP Workflows job:

Copy

{
   "GCPWF": 
   {
      "Type": "ConnectionProfile:GCP Workflows",
      "Identity Type": "service_account",
      "GCP API URL": "https://workflowexecutions.googleapis.com",
      "Service Account Key": "*****",
      "Connection Timeout": "20",
      "Description": "",
      "Centralized": true
   }
}

The following table describes the GCP Workflows job connection profile parameters.

Parameter	Description
Identity Type	Determines one of the following authentication types with GCP Access Control: service_account: Authenticates with an application ID (service account) and client secret. os_user: Authenticates based on a detected IAM role, which removes the need to provide additional credentials.
GCP API URL	(Service Account) Defines the Google Cloud Platform (GCP) authentication endpoint for GCP Workflows. Default: https://workflowexecutions.googleapis.com
Service Account Key	(Service Account) Defines a service account that is associated with an RSA key pair.
Connection Timeout	Determines the number of seconds to wait after Control-M initiates a connection request before a timeout occurs. Default:: 20
Centralized	Determines whether to create a centralized connection profile, which is stored in the Control-M/EM database and is available to all Agents. You must set this parameter to true.