Alerts

The Alerts tool enables you to view, manage, and monitor alerts. Alerts are sent to indicate that a problem or exception has occurred while processing a job or SMART folder, or to relay important job processing information that needs to be handled.

Alerts are generated in the following scenarios:

  • A job or SMART folder processing definition includes instructions in the Notify parameter to send a message to the Alerts window.

  • Alerts are automatically generated when a job terminates with the status Ended Not OK.

  • SLA Management services are used to relay messages of special importance, see Service Management.

  • The following alert appears when scheduling criteria of a specific job or folder was not met at New Day time:

    One or More Jobs in Daily System Did Not Run

In addition to the display of alerts in Helix Control-M, you can also transfer alerts to an external client for monitoring through external tools. For more information, see Setting Up External Alerts.

Configuring Alerts

This procedure describes how to enable or disable alerts and to filter the alert display.

Begin

  1. From the Control-M ribbon, in the upper-right corner, click .

    The Alerts window appears.

  2. Click Configure.

    The Alerts Server Configuration pane appears.

  3. Select the Enable Alerts checkbox.

  4. Set the criteria to filter the alerts as follows:

  5. Click Save.

    The alerts appear in the Alerts window according to the criteria set.

Handling Alerts

This procedure describes how to handle alerts:

Begin

  1. From the Control-M ribbon, in the upper-right corner, click

    The number of new alerts appear next to the Alerts icon.

    The Alerts window appears.

  2. Click Configure to filter alerts using specific criteria. For more information, see Configuring Alerts.

  3. Select one or more alerts to view its general properties in the right pane and set the urgency and comment for each alert, as follows:

    • From the Urgency drop-down list, select Normal, Urgent, or Critical to set the alert severity status.

    • In the Comment field, type your comments concerning the alert.

  4. Select an alert and right-click to set the handling status as one of the following:

    • Set Reviewed: The alert was viewed but not closed

    • Set Closed: The alert issue has been resolved

    • Set New: The alert has not been reviewed

You can view jobs that are related to a selected alert, or handle new alerts, as follows:

  • View Job: View the job that is related to the selected alert

  • Pause: Temporarily stop receiving new alerts

  • Resume: Continue receiving new alerts

Setting Up External Alerts

This procedure describes how to set up the transfer of alerts to an external client for monitoring through external tools, such as an event management system.

Before You Begin

  • If the host of your external client uses a proxy server, ensure that the HTTP_PROXY, HTTPS_PROXY, and WSS_PROXY environment variables are defined. Values for these variables are in the format {protocol}://{host}:{port}. For example, http://172.19.0.169:3128 or https://server.com:3128. For authentication to the proxy, include login credentials, as in the following example:
    HTTP_PROXY= http://username:password@proxy.example.com:1234

  • If you want to opt out of proxying on specific hosts or particular destination ports, use the NO_PROXY environment variable, set with a comma-separated list of hosts. For example, NO_PROXY=hostname.example.com,10.1.0.0/16,172.30.0.0/16:443

Begin

  1. Enable sending alerts to external event management systems in one of the following ways:

  2. (Optional) Open the stream of alerts from Helix Control-M using the following Automation API command:

    ctm run alerts:stream::open

    For more information, see External Alert Management.

    A successful response returns a WebSocket URL for the connection, as in the following example:

    Copy 
    "url": "wss://xyz-66095-alerts.us1.ci.ctmsaas.com"

    • You can have only one open Alerts stream at a time. If you want to open another stream, you must first close the previous stream, using the ctm run alerts:stream::close Automation API command.

      To close a stream, you must use the same API token that was used to open the stream. If you do not have access to the API token that was used to open the stream (for example, you have meanwhile regenerated the token), add the Force option to the API command, as follows:

      ctm run alerts:stream::close true

    • This step is optional. If you do not open the stream at this point, it opens automatically when you start the Alerts listener (last step below).

  3. (Optional) If you want to control the alert fields, their names, and order of appearance, submit a JSON template that overrides the default alert details. Use the following Automation API command:

    ctm run alerts:stream:template::set

  4. Set the Helix Control-M environment to listen to for alerts using the ctm run alerts:listener:environment::set Automation API command.

  5. Create a script to receive alert data and handle each new alert, and set your script using the ctm run alerts:listener:script::set Automation API command.

    The script is executed every time an alert is received. During script execution, the script receives alert details as parameters. For example:

    Copy 
    script_path/script.sh field_name1:field_value1 field_name2:field_value2

    Ensure that you are authorized to execute the script.

    A script begins with the following lines of code:

    #!/bin/bash
echo "Saving alerts data to the log file"
echo $* >> log.out
echo "Opening ticket in Jira"

    After the script invokes the interactive shell, saves alert data to the log, and notifies that it is going to open a ticket, the script proceeds with commands to open the ticket.

    For a variety of additional script samples that were submitted to the Automation API Community, see External Monitoring Tools Examples in GitHub.

    • For troubleshooting purposes, it is recommended that you redirect the script standard error output (stderr) to another log.

    • If you are migrating from Control-M (on-premises) to Helix Control-M and want to continue using the same script from before the migration, you can copy your existing script to the Helix Control-M machine and set it there.

  6. Start the Alerts listener for the defined environment using one of the following:

    Every time the listener receives an alert on your client, it triggers the Alert script.

    • You can have only one listener running at a time. If you want to open another listener (for example, from a different machine), you must first stop the running listener, using the ctm run alerts:listener::stop Automation API command. You must use the same API token that was used to start the listener

    • The Alerts listener logs are available at the following location:

      <$HOME>/.ctm/logs/

      To customize these logs, use the ctm environment configure Automation API command to set log parameters — listenerLogLevel, listenerLogSizeMb, and listenerMaxFileCount.

    • When listenerLogLevel is set to debug level, the exit status from the Alert script is included in the Alerts listener log.

    • If no alerts are received for 1 hour, notification is sent to your Helix Control-M Trust Page.

    As an alternative to the BMC-provided Alerts listener (Steps 4-6 above), you can do one of the following:

    • Connect to the gateway using a wscat command:

      Copy 
      wscat -c <WebSocket_URL> -P --slash -H x-api-key:<token>

    • Use your own Alerts Listener implementation on a WebSocket client. For a customizable example, see Customizable Alerts Listener Implementation in GitHub.

Configuring the Alerts Listener as a Linux Service

This procedure describes how to configure an alerts_listener system service on a Linux operating system. You can use this service instead of the Automation API commands to start or stop alerts listening or to check the status of the Alerts listener.

Before You Begin

Ensure successful completion of the following steps described in Setting Up External Alerts:

  • Enable external alerts.

  • (Optional) Open the stream of alerts from Helix Control-M.

  • (Optional) Submit a template to configure alert details.

  • Set the Helix Control-M environment.

  • Submit a script with details of external alert handling.

Begin

  1. Locate the alerts_listener.rc file in <$HOME>/<user>/ctm_cli/<version>/ctm-cli/bin.

    If you want to run this file from a different location, copy it into another directory.

  2. Run the alerts_listener.rc service with any of the following commands:

    • start

    • stop

    • status

    alerts_listener.rc start

    The Alerts listener logs are available at the following location:

    <$HOME>/.ctm/logs/

    To customize these logs, use the ctm environment configure Automation API command to set log parameters — listenerLogLevel, listenerLogSizeMb, and listenerMaxFileCount.

Configuring the Alerts Listener as a Windows Service

Configuration of an alerts_listener system service on a Windows operating system for use instead of the Automation API commands involves the use of a third-party wrapper executable. For more information, see Configuring the Alerts Listener as a Windows Service in GitHub.