Job Archiving Configuration

The config archive service enables you to manage the setup of Job Archiving (provided by the Control-M Workload Archiving add-on), manage the rules in Workload Archiving policies, and delete accumulated data from the Workload Archiving server. This feature is offered in an environment with Control-M 9.0.20 or higher.

For information about searching and retrieving archived data after performing the setup, see Archive Service.

The following API commands enable you to configure Job Archiving:

config archive:rules::get

The config archive:rules::get command enables you to get the details of all defined Workload Archiving policy rules.

The output is returned in JSON format. You can use this output as a template for your own Archive Rule files when creating new rules or editing existing rules. For more information about the syntax of Workload Archiving policy rules, see Archive Rule File.

CLI Syntax

Copy
ctm config archive:rules::get

REST API Syntax

cURL:

Copy
AuthHeader="x-api-key: $token"
# AuthHeader="Authentication: Bearer $token"  #for a session token

curl -H "$AuthHeader" $endpoint/config/archive/rules

To determine the correct AuthHeader value—"Authentication: Bearer $token" or "x-api-key: $token"—see Authentication Tokens.

config archive:statistics::get

The config archive:statistics::get command enables you to get statistics regarding job archiving for all defined Workload Archiving policy rules.

CLI Syntax

Copy
ctm config archive:statistics::get

REST API Syntax

cURL:

Copy
AuthHeader="x-api-key: $token"
# AuthHeader="Authentication: Bearer $token"  #for a session token

curl -H "$AuthHeader" $endpoint/config/archive/statistics

To determine the correct AuthHeader value—"Authentication: Bearer $token" or "x-api-key: $token"—see Authentication Tokens.

Response

The following example shows the job archiving statistics that are returned when only one rule is defined:

Copy
{
  "rulesStatisticList": {
    "ruleStatistics": [
      {
        "ruleName": "Collect all DS jobs",
        "retention": 1,
        "retentionType": "Year",
        "oldestItem": "2019-01-28 12:29:01 PM",
        "newestItem": "2019-02-08 07:00:11 AM",
        "totalJobs": 30,
        "dataSize": "1 MB"
      }
    ]
  },
  "summary": {
    "totalNumberOfJobs": 30,
    "totalDataSize": "1 MB",
    "actualDbSize": "9 MB",
    "diskUsage": "37%"
  }
}

config archive:rule::add

The config archive:rule::add command enables you to create a new Workload Archiving policy rule based on rule settings that you define through an Archive Rule file.

CLI Syntax

Copy
ctm config archive:rule::add -f <archiveRule>

Where archiveRule is the full path and name of a .json payload file that contains definitions for the rule, as described in Archive Rule File.

If annotation is enabled for the Configuration Management category in Control-M, you must also provide an annotation to justify your action. For more information, see Annotation Input.

REST API Syntax

cURL:

Copy
AuthHeader="x-api-key: $token"
# AuthHeader="Authentication: Bearer $token"  #for a session token

curl -H "$AuthHeader" -X POST -H "Content-Type: application/json"
-d @ruleDefinition.json "$endpoint/config/archive/rule"

To determine the correct AuthHeader value—"Authentication: Bearer $token" or "x-api-key: $token"—see Authentication Tokens.

config archive:rule::update

The config archive:rule::update command enables you to update the definitions of an existing Workload Archiving policy rule based on rule settings that you define through an Archive Rule file.

To help you author your Archive Rule file, you can use the output for a specific rule from the config archive:rules::get command as your template. You can edit or remove lines, as applicable. However, ensure that the name property appears with the correct rule name. For full details about all rule settings, see Archive Rule File.

CLI Syntax

Copy
ctm config archive:rule::update <ruleName> [-f <archiveRule>]

The following table describes the config archive:rule::update command parameters.

Parameter

Description

ruleName

Name of the archiving policy rule that you want to update.

archiveRule

Full path and name of a .json payload file that contains definitions for the rule, as described in Archive Rule File.

If annotation is enabled for the Configuration Management category in Control-M, you must also provide an annotation to justify your action. For more information, see Annotation Input.

REST API Syntax

cURL:

Copy
ruleName=rule5
AuthHeader="x-api-key: $token"
# AuthHeader="Authentication: Bearer $token"  #for a session token

curl -H "$AuthHeader" -X POST -H "Content-Type: application/json"
-d @ruleDefinition.json "$endpoint/config/archive/rule/$ruleName"

To determine the correct AuthHeader value—"Authentication: Bearer $token" or "x-api-key: $token"—see Authentication Tokens.

config archive:rule::delete

The config archive:rule::delete command enables you to delete an existing Workload Archiving policy rule.

CLI Syntax

Copy
ctm config archive:rule::delete <ruleName> <deleteRuleDataFlag>

The following table describes the config archive:rule::delete command parameters.

Parameter

Description

ruleName

Name of the archiving policy rule that you want to update.

deleteRuleDataFlag

The scope of deletion, one of the following values:

  • deleteRuleData: Delete the rule and all data collected based on this rule.

  • deleteRuleWithoutData: Delete the rule only (but do NOT delete any collected job data).

If annotation is enabled for the Configuration Management category in Control-M, you must also provide an annotation to justify your action. For more information, see Annotation Input.

REST API Syntax

cURL:

Copy
ruleName=rule5
AuthHeader="x-api-key: $token"
# AuthHeader="Authentication: Bearer $token"  #for a session token

curl -H "$AuthHeader" 
-X DELETE "$endpoint/config/archive/rule/$ruleName?deleteRuleDataFlag=deleteRuleWithoutData"

To determine the correct AuthHeader value—"Authentication: Bearer $token" or "x-api-key: $token"—see Authentication Tokens.

config archive::cleanup

The config archive::cleanup command enables you to delete job data (including job output and job logs) that has accumulated in the Workload Archiving server, based on search criteria that you specify.

CLI Syntax

Copy
ctm config archive::cleanup -s <search query>

-s is used to run a search using the query string format "field1=criteria1&field2=criteria2&field3=criteria3".

The following table describes the fields that you can use in the search query.

Field

Criteria Guidelines

  • jobname

  • jobnameExceptions

  • server

  • ctmExceptions

  • folder

  • folderExceptions

  • application

  • applicationExceptions

  • subApplication

  • subApplicationExceptions

  • library

  • libraryExceptions

  • jobStatus

  • Supported wildcards are the * (asterisk) and ? (question mark).

  • Many of the fields are paired by another field for specifying exceptions, that is, items to exclude from cleanup.

  • For many fields (jobname, server, folder, application, subApplication, and library), a value of "*" (all) is NOT supported.

  • For the jobStatus field, the following job statuses are relevant:

    • Ended OK

    • Ended Not OK

The server field was previously named ctm (deprecated name).

  • ruleName

  • The ruleName field cannot be combined with any other field in a query.

  • Supported wildcards are * and ?. The * (all rules) wildcard is not supported.

If annotation is enabled for the Configuration Management category in Control-M, you must also provide an annotation to justify your action. For more information, see Annotation Input.

REST API Syntax

cURL:

Copy
AuthHeader="x-api-key: $token"
# AuthHeader="Authentication: Bearer $token"  #for a session token

curl -H "$AuthHeader" -X DELETE
"$endpoint/config/archive/cleanup?jobname=jobA*&jobnameExceptions=jobABC*&application=B*"

To determine the correct AuthHeader value—"Authentication: Bearer $token" or "x-api-key: $token"—see Authentication Tokens.

Archive Rule File

The Archive Rule File code sample shows the available properties and objects that you can include within payload JSON files that you can submit when you configure Job Archiving. Details about each of the properties and objects in this file are provided below.

Copy
{
    "name": "All DS jobs",
    "description": "Collect all DS jobs",
    "maxOutputSize": 20,
    "maxOutputSizeType": "MB",
    "trimType": "Omit",
    "retention": 1,
    "retentionType": "Months",
    "isActive": true,
    "archivedType": "Log",
    "ruleParameters": [
        {
          "field": "ctm",
          "criteria": "*",
          "exceptions": [
            "fds*"
          ]
        },
        {
          "field": "jobName",
          "criteria": "A*",
          "exceptions": [
            "Ac*",
            "Ad*"
          ]
        },
        {
          "field": "jobType",
          "criteria": "Job:FileTransfer"
        },
        {
          "field": "application",
          "criteria": "*",
          "exceptions": [
            "c*"
          ]
        },
        {
          "field": "subApplication",
          "criteria": "s*",
          "exceptions": [
            "s4*"
          ]
        },
        {
          "field": "folder",
          "criteria": "*",
          "exceptions": [
            "f*"
          ]
        },
        {
          "field": "library",
          "criteria": "bmc*",
          "exceptions": [
            "bmc6*",
            "bmc7*"
          ]
        },
        {
          "field": "jobStatus",
          "criteria": "Ended OK"
        }
    ]
}

General Rule Settings

The following table describes the General Rule Property settings, which you can use to start your Archive Rule file for the rule that you are defining.

Property

Description

name

A unique name for the rule.

This is the only property that is required for a new rule.

It cannot be modified when updating an existing rule.

description

An optional textual description of the rule.

maxOutputSize

Maximum size of output to retrieve for jobs that match this rule.

Valid Values: 1–100

maxOutputSizeType

Unit of measurement for maximum output size.

Valid values: KB, MB

trimType

What to do when retrieved job output exceeds the maximum size.

Valid Values:

  • Beginning: Trims the excess size from the beginning of the output.

  • End: Trims the excess size from the end of the output.

  • Omit: Omits this output from the job outputs that are retrieved.

retention

Amount of time to keep the data that is retrieved based on this rule in the Workload Archiving server.

Valid Values: any positive integer.

retentionType

Unit of measurement for the retention period.

Valid Values: Days, Months, Years.

isActive

Activity status of the rule.

Valid Values: true (enabled) | false (disabled)

archivedType

Type(s) of data to archive for jobs that match this rule.

Valid Values:

  • Log

  • Output

  • Log and Output.

Rule Parameters for Job Filtering

Under the ruleParameters object, you can define a group of rule parameters that filter jobs based on various job properties and determine which jobs to archive. These parameters are defined with the following properties:

  • field: Job property by which to filter.

  • criteria: Filter value.

  • exceptions: Values to exclude from the filter.

All rule parameters are optional, and the default for all criteria is * (ALL jobs).

The following table summarizes the guidelines for including rule parameters in an Archive Rule file.

Field

Criteria

Exceptions

ctm

Any string.

* and ? wildcards are supported.

One or more strings, separated by commas.

jobType

One of the following job types:

  • All

  • Distributed:All (all jobs on distributed-type machines).

  • Job:CommandAndScript

  • Job:FileWatcher

  • Job:Informatica

  • Job:SAP:R3

  • Job:SAP:BW

  • Job:Hadoop

  • Job:FileTransfer

  • Job:Database

  • Job:AWS

In the output of the Config Service command, you might see other job types that are not supported by Control-M Automation API (from jobs that were created through Control-M). Do not include unsupported job types in your Archive Rule files. 

Not applicable.

jobName

Any string.

* and ? wildcards are supported.

One or more strings, separated by commas.

application

Any string.

* and ? wildcards are supported.

One or more strings, separated by commas.

subApplication

Any string.

* and ? wildcards are supported.

One or more strings, separated by commas.

folder

Any string.

* and ? wildcards are supported.

One or more strings, separated by commas.

library

Any string.

* and ? wildcards are supported.

One or more strings, separated by commas.

jobStatus

One of the following statuses:

  • All

  • Ended OK.

  • Ended Not OK.

Not applicable.