Control-M for Airbyte

Airbyte is an open-source extract, transform, and load (ETL) service that enables you to build data pipelines and load data to a data warehouse, data lake, database, or analytics tool.

Control-M for Airbyte enables you to do the following:

Execute Airbyte jobs that sync or reset data pipelines.
Manage Airbyte credentials in a secure connection profile.
Connect to any Airbyte endpoint.
Introduce all Control-M capabilities to Control-M for Airbyte, including advanced scheduling criteria, complex dependencies, Resource Pools, Lock Resources, and variables.
Integrate Airbyte jobs with other Control-M jobs into a single scheduling environment.
Monitor the status, results, and output of Airbyte jobs.
Attach an SLA job to the Airbyte jobs.

Control-M for Airbyte Compatibility

The following table lists the prerequisites that are required to use the Airbyte plug-in, each with its minimum required version.

Component	Version
Control-M/EM	9.0.21.200
Control-M/Agent	9.0.21.200
Control-M Application Integrator	9.0.21.200
Control-M Automation API	9.0.21.200

Control-M for Airbyte is supported on Control-M Web and Control-M Automation API, but not on the Control-M client.

To download the required installation files for each prerequisite, see Obtaining Control-M Installation Files.

Setting up Control-M for Airbyte

This procedure describes how to deploy the Airbyte plug-in, create a connection profile, and define an Airbyte job in Control-M Web and Automation API.

Integration plug-ins released by BMC require an Application Integrator installation. However, these plug-ins are not editable and you cannot import them into Application Integrator. To deploy these integrations to your Control-M environment, import them directly into Control-M with Control-M Automation API.

Before You Begin

Verify that Automation API is installed, as described in Automation API Installation.

Begin

Create a temporary directory to save the downloaded files.
Download the Airbyte plug-in from the Control-M for Airbyte download page in the EPD site.
Install the Airbyte plug-in with the Automation API Provision service:
1. Log in to the Control-M/EM Server machine as an Administrator and store the downloaded zip file in the one of the following locations:
  - Linux: $HOME/ctm_em/AUTO_DEPLOY
  - Windows: <EM_HOME>\AUTO_DEPLOY
2. Log in to the Control-M/Agent machine and run the provision image command, as follows:
  - Linux: ctm provision image Airbyte_plugin.Linux
  - Windows: ctm provision image Airbyte_plugin.Windows
Create an Airbyte connection profile in Control-M Web or Automation API, as follows:
- Web: Create a Centralized Connection Profile with Airbyte Connection Profile Parameters
- Automation API: ConnectionProfile:Airbyte
Define an Airbyte job in Control-M Web or Automation API, as follows:
- Web: Create a Job with Airbyte Job parameters
- Automation API: Job:Airbyte

To remove this plug-in from an Agent, see Removing a Plug-in. The plug-in ID is ABY122023.

Change Log

The following table provides details about changes that were introduced in new versions of this plug-in:

Plug-in Version	Details
1.0.01	Airbyte added as a Connection Name and Connection Id when you click the Load button.
1.0.00	Initial version