Control-M for AWS Glue DataBrew

AWS Glue DataBrew is a cloud-based extract, transform, load (ETL) service that you can use to visualize your data and publish it to the Amazon S3 Data Lake.

Control-M for AWS Glue DataBrew enables you to do the following:

  • Execute AWS Glue DataBrew jobs.

  • Manage AWS Glue DataBrew credentials in a secure connection profile.

  • Connect to any AWS Glue DataBrew endpoint.

  • Introduce all Control-M capabilities to Control-M for AWS Glue DataBrew, including advanced scheduling criteria, complex dependencies, resource pools, lock resources, and variables.

  • Integrate AWS Glue DataBrew jobs with other Control-M jobs into a single scheduling environment.

  • Monitor the status, results, and output of AWS Glue DataBrew jobs.

  • Attach an SLA job to the AWS Glue DataBrew jobs.

  • Run 50 AWS Glue DataBrew jobs simultaneously per Agent.

Setting up Control-M for AWS Glue DataBrew

This procedure describes how to deploy the AWS Glue DataBrew plug-in, create a connection profile, and define an AWS Glue DataBrew job in Control-M SaaS and Automation API.

Before You Begin

Begin

  1. Do one of the following:

    • Install: Run one of the following provision image commands:

      • Linux: ctm provision image ADB_plugin.Linux

      • Windows: ctm provision image ADB_plugin.Windows

    • Upgrade: Run the following command:

      ctm provision agent::update

  2. Create an AWS Glue DataBrew connection profile in Control-M SaaS or Automation API, as follows:

  3. Define an AWS Glue DataBrew job in Control-M SaaS or Automation API, as follows:

To remove this plug-in from an Agent, see Removing a Plug-in. The plug-in ID is ADB112022.

Change Log

The following table provides details about changes that were introduced in new versions of this plug-in:

Plug-in Version

Details

1.0.02

  • Added AWS Assume IAM Role cross-account authentication

  • Added HTTP Codes, Rerun Interval, and Attempt Reruns parameters to the connection profile to rerun an execution step with an HTTP code

1.0.01

Handles incorrect URLs in centralized connection profiles

1.0.00

Initial version