Control-M for GCP BigQuery

Google Cloud Platform (GCP) BigQuery is a cloud-computing platform that enables you to process, analyze, and store your data.

Control-M for GCP BigQuery enables you to do the following:

Execute SQL statements, stored procedures or functions, and table functions; and copy, load, and export data from tables in Google Cloud Storage.
Manage GCP BigQuery credentials in a secure connection profile.
Connect to any GCP BigQuery endpoint.
Introduce all Control-M capabilities to Control-M for GCP BigQuery, including advanced scheduling criteria, complex dependencies, Resource Pools, Lock Resources, and variables.
Integrate GCP BigQuery jobs with other Control-M jobs into a single scheduling environment.
Monitor the status, results, and output of GCP BigQuery jobs.
Attach an SLA job to the GCP BigQuery jobs.
Run 50 GCP BigQuery jobs simultaneously per Agent.

Control-M for GCP BigQuery Compatibility

The following table lists the prerequisites that are required to use the GCP BigQuery plug-in, each with its minimum required version.

Component	Version
Control-M/EM	9.0.20.200
Control-M/Agent	9.0.20.200
Control-M Application Integrator	9.0.20.201
Control-M Automation API	9.0.20.250

Control-M for GCP BigQuery is supported on Control-M Web and Control-M Automation API, but not on the Control-M client.

To download the required installation files for each prerequisite, see Obtaining Control-M Installation Files.

Setting up Control-M for GCP BigQuery

This procedure describes how to deploy the GCP BigQuery plug-in, create a connection profile, and define a GCP BigQuery job in Control-M Web and Automation API.

Integration plug-ins released by BMC require an Application Integrator installation. However, these plug-ins are not editable and you cannot import them into Application Integrator. To deploy these integrations to your Control-M environment, import them directly into Control-M with Control-M Automation API.

Before You Begin

Verify that Automation API is installed, as described in Automation API Installation.

Begin

Create a temporary directory to save the downloaded files.
Download the GCP BigQuery plug-in from the Control-M for GCP BigQuery download page in the EPD site.
Install the GCP BigQuery plug-in via one of the following methods:
- (9.0.21 or higher) Use the Automation API Provision service, as follows:
  1. Log in to the Control-M/EM Server machine as an Administrator and store the downloaded zip file in the one of the following locations:
    - Linux: $HOME/ctm_em/AUTO_DEPLOY
    - Windows: <EM_HOME>\AUTO_DEPLOY
  2. Log in to the Control-M/Agent machine and run the provision image command, as follows:
    - Linux: ctm provision image GBQ_plugin.Linux
    - Windows: ctm provision image GBQ_plugin.Windows
- (9.0.20.200 or lower) Run the Automation API Deploy service, as described in deploy jobtype.
Create a GCP BigQuery connection profile in Control-M Web or Automation API, as follows:
- Web: Create a Centralized Connection Profile with GCP BigQuery Connection Profile Parameters
- Automation API: ConnectionProfile:GCP BigQuery
Define a GCP BigQuery job in Control-M Web or Automation API, as follows:
- Web: Create a Job with GCP BigQuery Job parameters
- Automation API: Job:GCP BigQuery

To remove this plug-in from an Agent, see Removing a Plug-in. The plug-in ID is GBQ102022.

Change Log

The following table provides details about changes that were introduced in new versions of this plug-in:

Plug-in Version	Details
1.0.02	Minor changes in job output
1.0.01	Minor changes in the job execution steps
1.0.00	Initial version