Workflow Insights Data Exporter

Workflow Insights Data Exporter enables you to export some or all of the data that Workflow Insights collects into OpenSearch-indexed CSV (see note) files, which you can load into other dashboard programs outside of Control-M. OpenSearch indexes are database views that can also be explored via Workflow Insights Dashboards, as described in Workflow Insights Dashboard Options and Workflow Insights Dashboard Filters.

Workflow Insights data records are saved even when Data Exporter is not enabled, which results in a large data export when Data Exporter is enabled for the first time or after it has been disabled for a long period of time. To configure the number of months data records are saved for, see Configuring Workflow Insights.

The OpenSearch-indexed file shows that it has a CSV extension. However, it is actually a .jsonl file and you need to work in the .jsonl format.
If you use Elasticsearch in Workflow Insights, you must first upgrade to OpenSearch, as described in Upgrading from Elasticsearch to OpenSearch.
You must have Control-M/EM 9.0.21.200 or higher installed.

The following API commands enable you configure Workflow Insights Data Exporter:

config workflowinsights:dataexport:parameters::get
config workflowinsights:dataexport:parameters::set
config workflowinsights:dataexport:status::get

config workflowinsights:dataexport:parameters::get

The config workflowinsights:dataexport:parameters::get command enables you to get a list, in JSON format, of the Workflow Insights Data Exporter system parameters and their current values.

To change these parameter values, copy the JSON-formatted API response to a JSON file, change the parameter values, save the file, and send it with the config workflowinsights:dataexport:parameters::set command. You must have Control-M/EM 9.0.21.200 or higher installed.

CLI Syntax

Copy

ctm config workflowinsights:dataexport:parameters::get

Response

The following example shows the JSON-formatted API response:

Copy

{
   "parameters": [
   {
      "name": "WIExportEnabled",
      "value": "true"
   },
   {
      "name": "WIExportTargetPath",
      "value": "/h/user/writable/wiexport"
   },
   {
      "name": "WIExportMaxFileSize",
      "value": "64"
   },
   {
      "name": "WIExportMaxPeriodPerFile",
      "value": "24"
   },
   {
      "name": "WIExportInterval",
      "value": "60"
   },
   {
      "name": "WIExportIndex",
      "value": "all"
   },
   {
      "name": "WIIndexList",
      "value": "ctm_alert,ctm_user_action,ctm_job_execution,ctm_job_order,ctm_service,ctm_definition_update"
   },
   {
      "name": "WIExportMaxRetries",
      "value": "5"
   },
   {
      "name": "WIExportRetryInterval",
      "value": "24"
    } ]
}

The following table describes the Workflow Insights Data Exporter system parameters that are shown in the example above, and that you can change via the Workflow Insights Data Exporter command.

Parameter	Description
WIExportEnabled	Determines whether Data Exporter is enabled, as follows: true: Data Exporter is enabled. false: Data Exporter is disabled. Default: false
WIExportTargetPath	Defines the preexisting directory path where the OpenSearch-indexed CSV are saved. /h/user/writable/wiexport This value must be accessible from any machine that runs a Workflow Insights service.
WIExportMaxFileSize	Determines the maximum size (in MB) of the OpenSearch-indexed files that are exported in CSV format. Valid Values: 1–128 Default: 64
WIExportMaxPeriodPerFile	Determines the number of hours of data to export per OpenSearch-indexed CSV file. Valid Values: 1–24 Default: 24 A value of 8 produces three CSV files with timestamps from the following periods each day: Midnight–8:00 AM 8:00 AM–4:00 PM 4:00 PM–midnight
WIExportInterval	Defines how often, in minutes, Data Exporter queries Workflow Insights for new data. Default: 60
WIExportIndex	Determines one or more of the following Workflow Insights indexes to export: ctm_alert: Workflow Alerts index. ctm_user_action: User Actions index. ctm_job_execution: Job Executions index. ctm_job_order: Job Run index. ctm_service: SLA Management Services index. ctm_definition_update: Workflow Definition Updates index. all: All of the above-mentioned indexes. Default: all
WIIndexList	Displays all the Workflow Insights indexes that you can export with the WIExportInterval parameter, as described in the WIExportIndex parameter. This is a read-only parameter that you can leave or delete from the JSON-file that is sent with the Workflow Insights Data Exporter command.
WIExportMaxRetries	Determines the number of times Data Exporter attempts to export an OpenSearch-indexed CSV file after the first attempt fails. Valid Values: 0–10 Default: 5 If all attempts fail, the WIExportEnabled parameter is set to false.
WIExportRetryInterval	Determines the number of seconds to wait before Data Exporter reattempts to export an OpenSearch-indexed CSV file. Valid Values: 5–60 Default: 24

REST API Syntax

cURL:

Copy

AuthHeader="x-api-key: $token"
# AuthHeader="Authorization: Bearer $token"  #for a session token

curl -H "$AuthHeader" -H "Content-Type: application/json" -X GET
"%endpoint%/config/workflowinsights/dataexport/parameters"

To determine the correct AuthHeader value—"Authorization: Bearer $token" or "x-api-key: $token"—see Authentication Tokens.

config workflowinsights:dataexport:parameters::set

The config workflowinsights:dataexport:parameters::set command enables you to set the Workflow Insights system parameters with an attached JSON file.

To see the parameters that you want to set, first run the config workflowinsights:dataexport:parameters::get command, and then copy the JSON-formatted API response to a JSON file. After you change the parameter values, save and send the file with the config workflowinsights:dataexport:parameters::set command. You must have Control-M/EM 9.0.21.200 or higher installed.

CLI Syntax

Copy

ctm config workflowinsights:dataexport:parameters::set -f <New_WI_System_Parameters>

Where New_WI_System_Parameters is the full directory path and filename of the JSON parameter definitions file.

The Workflow Insights Data Exporter system parameters are described in the Parameter table.

If annotation is enabled for the Configuration Management category in the CCM, you must also provide an annotation to justify your action. For more information, see Annotation Input.

REST API Syntax

cURL:

Copy

AuthHeader="x-api-key: $token"
# AuthHeader="Authorization: Bearer $token"  #for a session token

curl -H "$AuthHeader" -H "Content-Type: application/json" 
--data "@workflowSystemParameters.json" -X POST
"%endpoint%/config/workflowinsights/dataexport/parameters"

To determine the correct AuthHeader value—"Authorization: Bearer $token" or "x-api-key: $token"—see Authentication Tokens.

config workflowinsights:dataexport:status::get

The config workflowinsights:dataexport:status::get command enables you to determine whether Workflow Insights Data Exporter is enabled, and to see the type and time of the most recently exported OpenSearch-indexed CSV files. You must have Control-M/EM 9.0.21.200 or higher installed.

CLI Syntax

Copy

ctm config workflowinsights:dataexport:status::get

Response

The following example shows the JSON-formatted API response when you export all OpenSearch indexes:

Copy

{
   "enabled": true,
   "details": [
   {
      "name": "ctm_job_execution",
      "timestamp": "2023-12-24T08:23:10.895Z"
   },
   {
      "name": "ctm_alert",
      "timestamp": "2023-12-24T08:23:30.895Z"
   },
   {
      "name": "ctm_job_order",
      "timestamp": "2023-12-24T07:42:20.915Z"
   },
   {
      "name": "ctm_service",
      "timestamp": "2023-11-12T11:27:59.294Z"
   },
   {
      "name": "ctm_user_action",
      "timestamp": "2023-11-16T09:12:21.904Z"
   },
   {
      "name": "ctm_definition_update",
      "timestamp": "2023-11-12T11:27:32.077Z"
   } ]
}

The following table describes the parameters shown in the example response above.

Parameter	Description
enabled	Determines whether Data Exporter is enabled, as follows: true: Data Exporter is enabled. false: Data Exporter is disabled. Default: false
name	Determines which of the following Workflow Insights indexes is exported: ctm_alert: Workflow Alerts index. ctm_user_action: User Actions index. ctm_job_execution: Job Executions index. ctm_job_order: Job Run index. ctm_service: SLA Management Services index. ctm_definition_update: Workflow Definition Updates index. all: All of the above-mentioned indexes. Default: all
timestamp	Determines the time that the most recent Workflow Insights index was created, in ISO 8601 time format, as follows: <Year>-<Month>-<Day>-T-<Hour>:<Minute>:<Second>.<Millisecond>Z Where T separates the date from the time and Z stands for Universal Coordinated Time (UTC). 2023-11-12T11:27:32.077Z

Parameter

Description

enabled

Determines whether Data Exporter is enabled, as follows:

true: Data Exporter is enabled.
false: Data Exporter is disabled.

Default: false

name

Determines which of the following Workflow Insights indexes is exported:

ctm_alert: Workflow Alerts index.
ctm_user_action: User Actions index.
ctm_job_execution: Job Executions index.
ctm_job_order: Job Run index.
ctm_service: SLA Management Services index.
ctm_definition_update: Workflow Definition Updates index.
all: All of the above-mentioned indexes.

Default: all

timestamp

Determines the time that the most recent Workflow Insights index was created, in ISO 8601 time format, as follows:

<Year>-<Month>-<Day>-T-<Hour>:<Minute>:<Second>.<Millisecond>Z

Where T separates the date from the time and Z stands for Universal Coordinated Time (UTC).

2023-11-12T11:27:32.077Z

REST API Syntax

cURL:

Copy

AuthHeader="x-api-key: $token"
# AuthHeader="Authorization: Bearer $token"  #for a session token

curl -H "$AuthHeader" -H "Content-Type: application/json" -X GET
"%endpoint%/config/workflowinsights/dataexport/status"

To determine the correct AuthHeader value—"Authorization: Bearer $token" or "x-api-key: $token"—see Authentication Tokens.