Business Intelligence and Analytics Jobs

The following topics describe job types for business intelligence and analytics platforms and services:

Job:AWS QuickSight

AWS QuickSight is a cloud-based data analytics platform that enables you to visualize, analyze, and share large workloads of data.

To deploy and run an AWS QuickSight job, ensure that you have installed the AWS QuickSight plug-in with the provision image command or the provision agent::update command.

The following example shows how to define an AWS QuickSight job:

Copy 
"AWS QuickSight_Job_2":
{
   "Type": "Job:AWS QuickSight",
   "ConnectionProfile": "QUICKSIGHT",
   "AWS Dataset ID": "f351ce9e-1500-4291-b0e1-78b2d6f48861",
   "Refresh Type": "Full Refresh",
   "Status Polling Frequency": "30",
   "Failure Tolerance": "2",
}

The following table describes the AWS QuickSight job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:AWS QuickSight name that connects Control-M to AWS QuickSight.

AWS Dataset ID

Defines the ID of the AWS QuickSight job that is created in an AWS QuickSight workspace.

Refresh Type

Determines which of the following refresh functions to perform:

  • Full Refresh: Updates the entire data set, which refreshes all the rows in the table.

  • Incremental Refresh: Adds the rows that have not been imported to the table.

Status Polling Frequency

(Optional) Defines the number of seconds to wait before checking the status of the QuickSight job.

Default: 30

Failure Tolerance

Defines the number of times to check the job status before ending Not OK.

Default: 2

Job:Microsoft Power BI

The Microsoft Power BI job enables you to integrate Power BI workflows with your existing Control-M workflows.

To deploy and run a Microsoft Power BI job, ensure that you have installed the Microsoft Power BI plug-in with the provision image command or the provision agent::update command.

The following examples show how to define a Microsoft Power BI job:

Copy 
"Microsoft Power BI_Job_2": 
{ 
   "Type": "Job:Microsoft Power BI", 
   "ConnectionProfile": "POWERBI",
   "Dataset Refresh/ Pipeline Deployment": "Dataset Refresh",
   "Workspace Name": "Demo", 
   "Workspace ID": "a7989345-8cfe-44e7-851d-81560e67973f", 
   "Dataset ID": "9976ce6c-e21a-4c33-9b8c-37c8303231cf",
   "Parameters": "{\"type\":\"Full\",\"commitMode\":\"transactional\",\"maxParallelism\":20,\"retryCount\":2}",
   "Connection Timeout": "10", 
   "Status Polling Frequency": "10"
}
Copy 
"Microsoft Power BI_Job_2": 
{ 
   "Type": "Job:Microsoft Power BI", 
   "ConnectionProfile": "POWERBI",
   "Dataset Refresh/ Pipeline Deployment": "Pipeline Deployment",
   "Pipeline ID": "83f36385-4e38-43g4-8263-10aa12e3175c",
   "Connection Timeout": "10", 
   "Status Polling Frequency": "10"  
}

The following table describes the Microsoft Power BI job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:Microsoft Power BI name that connects Control-M to Power BI.

Dataset Refresh/ Pipeline Deployment

Determines one of the following options for execution in Power BI:

  • Dataset Refresh

  • Pipeline Deployment

Group Name

(Dataset) Defines a Power BI workspace where you want to refresh data.

Group ID

(Dataset) Defines the ID for the specified Power BI workspace (defined in the PowerBI Group Name).

Dataset Name

Defines a Power BI data set that you want to refresh in the specified workspace.

Parameters

(Dataset) Defines specific parameters to pass when the job runs, defined as JSON pairs of parameter name and value.

Copy 
 {"param1":"value1", "param2":"value2"}

For no parameters, specify {}.

For more information about available parameters, see Datasets - Refresh Dataset in the Microsoft Power BI documentation.

To specify parameters, the dataset must be in Premium group.

Copy 
{
   "type":"Defragment",
   "commitMode":"transactional",
   "maxParallelism":20,
   "retryCount":2,
   "objects":[
      {"table":"Contributions"},
      {"table":"Traffic_Popular"}
   ]
}

Connection Timeout

(Optional) Determines the maximum number of seconds to wait for REST API requests to respond, before disconnecting.

Default: 10

Status Polling Frequency

(Optional) Determines the number of seconds to wait before checking the status of the job.

Default: 10

Pipeline ID

Defines the ID of a Power BI pipeline that you want to deploy from dev to test and then to production.

Job:Microsoft Power BI SP

The Microsoft Power BI job enables you to include service principal authentication to integrate Power BI workflows with your existing Control-M workflows.

To deploy and run a Microsoft Power BI job, ensure that you have installed the Microsoft Power BI plug-in with the provision image command or the provision agent::update command.

The following example shows how to define a Microsoft Power BI SP job for Dataset Refresh with service principal authentication:

Copy 
"Microsoft Power BI SP_Job_2":
{
   "Type": "Job:Microsoft Power BI SP",
   "ConnectionProfile": "POWERBI",
   "Action": "Dataset Refresh - Service Principal",
   "Variables": [
   {
      "UCM-WORKSPACENAME": "Github"
   },
   {
      "UCM-WORKSPACEID": "1c2b505a-da3f-4dcc-8a7a-a6be1083c339"
   },
   {
      "UCM-DATASETID": "9976ce6c-e21a-4c33-9b8c-37c8303231cf"
   }
   ],
   "Parameters": "{}",
   "Status Polling Frequency": "60", 
   "Failure Tolerance": "2" 
}

The following example shows how to define a Microsoft Power BI SP job for Pipeline Deployment with service principal authentication:

Copy 
"Microsoft Power BI SP_Job_2":
{
   "Type": "Job:Microsoft Power BI SP",
   "ConnectionProfile": "POWERBI",
   "Action": "Pipeline Deployment - Service Principal",
   "Pipeline ID": "625ec832-b4fe-4e22-933f-3fd6c7b009fd",
   "Status Polling Frequency": "60", 
   "Failure Tolerance": "2" 
}

The following table describes the Microsoft Power BI SP job parameters.

Attribute

Action

Description

Connection profile

All actions

Determines the ConnectionProfile:Microsoft Power BI SP name that connects Control-M to Power BI.

Action

NA

Determines one of the following execution options in Power BI:

  • Dataset Refresh

  • Dataflow Refresh

  • Pipeline Deployment

  • Dataset Refresh - Service Principal

  • Dataflow Refresh - Service Principal

  • Pipeline Deployment - Service Principal

Workspace ID

  • Dataset Refresh

  • Dataflow Refresh

Defines the ID for the specified Power BI Workspace (defined in the PowerBI Group Name).

Workspace ID - SP

  • Dataset Refresh - Service Principal

  • Dataflow Refresh - Service Principal

Defines the ID for the specified Power BI SP Workspace (defined in the PowerBI Group Name).

Dataset ID

Dataset Refresh

Defines a Power BI dataset to refresh in the specified Workspace.

This attribute is also known as the semantic model in Power BI.

Dataset ID - SP

Dataset Refresh - Service Principal

Defines a Power BI SP dataset to refresh in the specified Workspace.

This attribute is also known as the semantic model in Power BI.

Dataflow ID

Dataflow Refresh

Defines a unique identifier assigned to a dataflow within the Power BI service.

A dataflow is a collection of tables that are created and managed in Workspaces in the Power BI service.

Dataflow ID - SP

Dataflow Refresh - Service Principal

Defines a unique identifier assigned to a dataflow within the Power BI SP service.

A dataflow is a collection of tables that are created and managed in Workspaces in the Power BI SP service.

Pipeline ID

Pipeline Deployment

Defines the ID of a Power BI pipeline to deploy for test and production.

Pipeline ID - SP

Pipeline Deployment - Service Principal

Defines the ID of a Power BI SP pipeline to deploy for test and production.

Parameters

  • Dataset Refresh

  • Dataset Refresh - Service Principal

Defines specific parameters, in JSON format, to pass when the job executes.

For more information about available parameters, see Datasets - Refresh Dataset in the Microsoft Power BI documentation.

To specify parameters, the dataset must be in the Premium group.

Format: {"param1":"value1", "param2":"value2"}

Copy 
{
  "type":"Full",
  "commitMode":"transactional",
  "maxParallelism":20,
  "retryCount":2,
}

For no parameters, type {}.

Status Polling Frequency

 All actions

(Optional) Determines the number of seconds to wait before checking the status of the job.

Default: 60
Failure Tolerance All actions

Determines the number of times to check the job status before the job ends Not Ok.

Default: 2

Job:Qlik Cloud

Qlik Cloud supports a cloud-based extract, transform, load (ETL) service, called Qlik Cloud Data Services, and enables you to visualize your data using Qlik Sense.

To deploy and run a Qlik Cloud job, ensure that you have installed the Qlik Cloud plug-in with the provision image command or the provision agent::update command.

The following example shows how to define a Qlik Cloud job:

Copy 
"Qlik Cloud_Job": 
{
   "Type": "Job:Qlik Cloud",
   "ConnectionProfile": "QLIK-TEST",
   "Reload Type": "Full",
   "App Name": "Demo1",
   "Print Log to Output": "Yes",
   "Status Polling Frequency": "10",
   "Tolerance": "2"
}

The following table describes the Qlik Cloud job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:Qlik Cloud name that connects Control-M to Qlik Cloud.

Reload Type

Determines one of the following options to load data into the environment:

  • Full: deletes all the tables in the current environment and reloads the data from the data sources.

  • Partial: keeps the tables in the current environment and executes the Load and Select statements, preceded by an Add, Merge, or Replace prefix.

App Name

Defines the Qlik Sense app name, which contains one or more workspaces, called sheets.

Print Log to Output

Determines whether the job logs are included in the Control-M output.

Values:

  • Yes

  • No

Default: Yes

Status Polling Frequency

(Optional) Determines the number of seconds to wait before checking the status of the job.

Default: 10

Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2

Job:Tableau

Tableau is a cloud-based data analytics platform that enables you to visualize, analyze, and share large workloads of data.

To deploy and run a Tableau job, ensure that you have installed the Tableau plug-in with the provision image command or the provision agent::update command.

The following example shows how to define a Tableau job:

Copy 
"Tableau_Refresh_Datasource": 
{
   "Type": "Job:Tableau",
   "ConnectionProfile": "TABLEAU_CP",
   "Action": "Refresh Datasource",
   "Datasource Name": "BQ_Dataset",
   "Status Polling Frequency": "10",
   "Failure Tolerance": "2"
}

The following table describes the Tableau job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:Tableau name that connects Control-M to Tableau.

Action

Determines one of the following Tableau actions to perform:

  • Refresh Datasource: Updates the Tableau workflow with the most recent version of a data source.

  • Run Flow: Executes the Tableau workflow.

Datasource Name

Defines the name of the data source that is refreshed.

Tableau can connect to the following types of data sources:

  • Files, such as CSV, Excel, and PDF.

  • Databases, such as Oracle, SQL Server, and DB2.

  • Cloud storage, such as Azure, GCP, and AWS.

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the job.

Default: 30

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 1