Business Intelligence and Analytics Jobs

The following topics describe job types for business intelligence and analytics platforms and services:

Job:AWS QuickSight

AWS QuickSight is a cloud-based data analytics platform that enables you to visualize, analyze, and share large workloads of data.

To deploy and run an AWS QuickSight job, ensure that you have installed the AWS QuickSight plug-in with the provision image command or the provision agent::update command.

The following example shows how to define an AWS QuickSight job:

Copy 
"AWS QuickSight_Job_2":
{
   "Type": "Job:AWS QuickSight",
   "ConnectionProfile": "QUICKSIGHT",
   "AWS Dataset ID": "f351ce9e-1500-4291-b0e1-78b2d6f48861",
   "Refresh Type": "Full Refresh",
   "Status Polling Frequency": "30",
   "Failure Tolerance": "2",
}

The following table describes the AWS QuickSight job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:AWS QuickSight name that connects Control-M to AWS QuickSight.

AWS Dataset ID

Defines the ID of the AWS QuickSight job that is created in an AWS QuickSight workspace.

Refresh Type

Determines which of the following refresh functions to perform:

  • Full Refresh: Updates the entire data set, which refreshes all the rows in the table.

  • Incremental Refresh: Adds the rows that have not been imported to the table.

Status Polling Frequency

(Optional) Defines the number of seconds to wait before checking the status of the QuickSight job.

Default: 30

Failure Tolerance

Defines the number of times to check the job status before ending Not OK.

Default: 2

Job:Microsoft Power BI

The Microsoft Power BI job enables you to integrate Power BI workflows with your existing Control-M workflows.

To deploy and run a Microsoft Power BI job, ensure that you have installed the Microsoft Power BI plug-in with the provision image command or the provision agent::update command.

The following examples show how to define a Microsoft Power BI job:

Copy 
"Microsoft Power BI_Job_2": 
{ 
   "Type": "Job:Microsoft Power BI", 
   "ConnectionProfile": "POWERBI",
   "Dataset Refresh/ Pipeline Deployment": "Dataset Refresh",
   "Workspace Name": "Demo", 
   "Workspace ID": "a7989345-8cfe-44e7-851d-81560e67973f", 
   "Dataset ID": "9976ce6c-e21a-4c33-9b8c-37c8303231cf",
   "Parameters": "{\"type\":\"Full\",\"commitMode\":\"transactional\",\"maxParallelism\":20,\"retryCount\":2}",
   "Connection Timeout": "10", 
   "Status Polling Frequency": "10"
}
Copy 
"Microsoft Power BI_Job_2": 
{ 
   "Type": "Job:Microsoft Power BI", 
   "ConnectionProfile": "POWERBI",
   "Dataset Refresh/ Pipeline Deployment": "Pipeline Deployment",
   "Pipeline ID": "83f36385-4e38-43g4-8263-10aa12e3175c",
   "Connection Timeout": "10", 
   "Status Polling Frequency": "10"  
}

The following table describes the Microsoft Power BI job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:Power BI name that connects Control-M to Power BI.

Dataset Refresh/ Pipeline Deployment

Determines one of the following options for execution in Power BI:

  • Dataset Refresh

  • Pipeline Deployment

Group Name

(Dataset) Defines a Power BI workspace where you want to refresh data.

Group ID

(Dataset) Defines the ID for the specified Power BI workspace (defined in Group Name).

Dataset Name

Defines a Power BI data set that you want to refresh under the specified workspace.

Parameters

(Dataset) Defines specific parameters to pass when the job runs, defined as JSON pairs of parameter name and value.

Copy 
 {"param1":"value1", "param2":"value2"}

For no parameters, specify {}.

For more information about available parameters, see Datasets - Refresh Dataset in the Microsoft Power BI documentation.

To specify parameters, the dataset must be in Premium group.

Copy 
{
   "type":"Defragment",
   "commitMode":"transactional",
   "maxParallelism":20,
   "retryCount":2,
   "objects":[
      {"table":"Contributions"},
      {"table":"Traffic_Popular"}
   ]
}

Connection Timeout

(Optional) Determines the maximum number of seconds to wait for REST API requests to respond, before disconnecting.

Default: 10

Status Polling Frequency

(Optional) Determines the number of seconds to wait before checking the status of the job.

Default: 10

Pipeline ID

Defines the ID of a Power BI pipeline that you want to deploy from dev to test and then to production.

Job:Qlik Cloud

Qlik Cloud supports a cloud-based extract, transform, load (ETL) service, called Qlik Cloud Data Services, and enables you to visualize your data using Qlik Sense.

To deploy and run a Qlik Cloud job, ensure that you have installed the Qlik Cloud plug-in with the provision image command or the provision agent::update command.

The following example shows how to define a Qlik Cloud job:

Copy 
"Qlik Cloud_Job": 
{
   "Type": "Job:Qlik Cloud",
   "ConnectionProfile": "QLIK-TEST",
   "Reload Type": "Full",
   "App Name": "Demo1",
   "Print Log to Output": "Yes",
   "Status Polling Frequency": "10",
   "Tolerance": "2"
}

The following table describes the Qlik Cloud job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:Qlik Cloud name that connects Control-M to Qlik Cloud.

Reload Type

Determines one of the following options to load data into the environment:

  • Full: deletes all the tables in the current environment and reloads the data from the data sources.

  • Partial: keeps the tables in the current environment and executes the Load and Select statements, preceded by an Add, Merge, or Replace prefix.

App Name

Defines the Qlik Sense app name, which contains one or more workspaces, called sheets.

Print Log to Output

Determines whether the job logs are included in the Control-M output.

Values:

  • Yes

  • No

Default: Yes

Status Polling Frequency

(Optional) Determines the number of seconds to wait before checking the status of the job.

Default: 10

Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 2

Job:Tableau

Tableau is a cloud-based data analytics platform that enables you to visualize, analyze, and share large workloads of data.

To deploy and run a Tableau job, ensure that you have installed the Tableau plug-in with the provision image command or the provision agent::update command.

The following example shows how to define a Tableau job:

Copy 
"Tableau_Refresh_Datasource": 
{
   "Type": "Job:Tableau",
   "ConnectionProfile": "TABLEAU_CP",
   "Action": "Refresh Datasource",
   "Datasource Name": "BQ_Dataset",
   "Status Polling Frequency": "10",
   "Failure Tolerance": "2"
}

The following table describes the Tableau job parameters.

Parameter

Description

ConnectionProfile

Defines the ConnectionProfile:Tableau name that connects Control-M to Tableau.

Action

Determines one of the following Tableau actions to perform:

  • Refresh Datasource: Updates the Tableau workflow with the most recent version of a data source.

  • Run Flow: Executes the Tableau workflow.

Datasource Name

Defines the name of the data source that is refreshed.

Tableau can connect to the following types of data sources:

  • Files, such as CSV, Excel, and PDF.

  • Databases, such as Oracle, SQL Server, and DB2.

  • Cloud storage, such as Azure, GCP, and AWS.

Status Polling Frequency

Determines the number of seconds to wait before checking the status of the job.

Default: 30

Failure Tolerance

Determines the number of times to check the job status before ending Not OK.

Default: 1