Control-M Architecture

The following diagram shows the logical architecture of a typical Control-M environment:

Control-M Components

Control-M components represent client applications, servers, a database, and other infrastructure that support functionality.

The following table describes the Control-M components:

Component

Description

Control-M/Enterprise Manager

Provides a central point of access and control for Control‑M/Servers. Control-M/Enterprise Manager also enables you to view, monitor, manage, and intervene in batch workflow processing across the entire enterprise.

Control-M/Server

Schedules jobs, manages job processing workflows, and provides notifications of job workflow status.

Control‑M/Servers are the scheduling engines that schedule jobs on other computers that can serve as Control-M/Agents and Remote Hosts.

Control-M/Agent and Remote Hosts

Run and track jobs according to the associated Control‑M/Server. Though Control-M/Servers are responsible for job scheduling and processing, they do not need to limit their activity to the computer where they reside. Control-M/Servers can schedule jobs to execute on other computers under their control.

You can have a dedicated Control‑M/Agent installed on each computer that is controlled by a Control‑M/Server, or you can use Agentless computers—known as Remote Hosts. Jobs are assigned to specific host IDs that identify the Control‑M/Agents and Remote Hosts.

For information about Control-M/Agents that run other platforms, see Control-M/Agent and Other Platforms.

Plug-ins

Enable you to connect to external applications and run Control-M jobs.

Control-M Add-Ons

Enable you to use Business Service Management (BSM) features.

Control-M/Enterprise Manager

Control-M/Enterprise Manager (Control-M/EM) is a Control-M component that provides a central point of control for Control‑M/Servers. Control‑M/EM enables users to do the following:

  • View the status of job schedules and execution in Control‑M environments.

  • Issue requests for additional information.

  • Make changes to Production.

  • Handle problems. Control‑M/EM also passes global events among Control‑M/Servers.

The following figure shows the main sub-components of Control-M/EM:

The Control-M/EM includes the following types of sub-components:

  • Control-M/EM Clients: Enables you to configure your Control-M environment to generate reports and define, run, and monitor your job flows.

  • Control-M/EM Servers: Enables Control-M/EM components to communicate with Control‑M/EM clients and other Control-M/EM components.

  • Infrastructure Components: Enables Control-M/EM clients to communicate with the relevant Control-M/EM Server and includes the Control-M/EM database.

Control-M/EM Clients

The following table describes the interfaces to your real-time batch environment and consists of the following:

Component

Description

Control-M

Enables you to define and schedule your production, which Control-M uses to control job processing and handling. You can monitor your production, and intervene when necessary.

Control-M Configuration Manager (CCM)

Enables you to manage, monitor, configure, and maintain all Control-M components, as well as define security settings and user authorizations.

Control-M Self Service

Enables you to view your services with a Web-based application and analyze the problematic services and jobs. After you have determined what the problems are, you can resolve them by performing various service and job actions, depending on your authorizations.

Control-M Workload Change Manager

Enables you to:

  • Request changes to business workflows in Control-M by creating and submitting requests in Control-M Workload Change Manager web application.

  • Define standards that comply with your organization's standards when defining folder or job definitions in Control-M.

Reports

Enables you to view jobs running Control-M Reporting Facility jobs.

These jobs cannot be run. If you want to run Control-M Report jobs in a Control-M client, you need to define a Control-M Report job in Control-M Automation API and then add the command to an OS job type.

Utilities

Enables you to run batch commands to perform specific actions in Control-M.

Control-M Automation API

Enables developers and DevOps engineers to build and deploy job workflows as code and manage the Control-M environment through Command Line Interface and REST API calls.

Control-M/EM Servers

The following table describes the Control-M/EM Servers and their functionalities including handling communication between Control‑M/EM clients and other Control‑M/EM components:

Component

Description

GUI Server (GSR)

Handles communication between Control-M/EM clients and Control-M/Servers. You can configure multiple GUI Servers in one Control-M/EM environment for load balancing purposes.

The GUI Server connects to each Gateway to transmit requests and receive responses to and from Control-M/Server.

Global Conditions Server (GCS)

Identifies and distributes global events, which are used to create job dependencies between Control-M/Servers, as described in Global Events.

The GCS connects to each Gateway to receive and transmit prerequisite events to the appropriate Control-M/Server.

Gateway

Handles communication between Control-M/EM Servers (GSR and GCS) and Control-M/Servers.

A different Gateway is defined for each Control-M/Server (including Control-M for z/OS).

Configuration Manager Server (CMS)

Receives information from the Control-M/Server and Control-M for z/OS Configuration Agents, and handles requests from the CCM.

SLA Manager

Monitors critical batch services when your site uses the Control-M SLA Manager Add-on.

Forecast Server

Helps you simulate job patterns to validate the schedules for running jobs on a future date. This is supported when your site uses the Control-M/Forecast Add-on.

Self Service Server

Monitors Self Service services when your site uses the Control-M Self Service Add-on.

Web Server

Provides access for the following applications, using an HTTP/S protocol:

  • Control-M Client

  • Control-M Web

  • Control-M Configuration Manager (CCM)

  • Control-M Reports

  • Control-M/EM API

  • Automation API

Services

Services is a set of services that expose shared functionality to various Control-M/EM applications and enable you to control and monitor them in the CCM or in the services-cli utility.

For a detailed description of each service, see Control-M/EM Services.

Control-M/Server

At the heart of Control-M processing is Control-M/Server, which is the scheduling engine that schedules jobs, manages job processing flows, performs load balancing, and handles requests from Control‑M/EM. Each Control-M/Server is resident on a particular computer and is specific to its operating system. The following lists the Control-M Server types:

  • Control-M/Server: Defines a distributed system component of Control‑M that maintains the Control‑M/Server database (including Active Jobs), schedules jobs, performs load balancing, sends job-handling requests to agent platforms, and handles requests from Control-M/EM.

    One or more hosts can be controlled by a single Control‑M/Server. In older implementations, a Control‑M/Server can consist of a single computer. In a server/agent implementation of Control‑M, Control‑M/Server consists of a server platform and all the agents, and any Agentless platforms that it handles.

  • Control-M for z/OS: Defines a Mainframe system component that uses the Control-M client to define and handle job definitions and requests.

The following image shows Control-M/Server and Control-M for z/OS and its related components:

Control-M/Agent and Remote Hosts

Although Control-M/Server is responsible for job scheduling and processing, it does not have to limit the activity to the computer on which it resides. It can schedule jobs on other computers that comes under its control. Control-M/Agents are commonly used to place computers under the control of a Control-M/Server. Control‑M/Agents submit jobs, perform requests from each Control‑M/Server and perform post-processing analysis of completed jobs.

As an alternative to having a dedicated Control-M/Agent installed on each computer under the control of a Control-M/Server, Control-M/Server can submit jobs to Agentless computers known as Remote Hosts. A Remote Host enables you to run jobs on a computer without having to install and update a Control-M/Agent, as well as less management.

The main advantages of Control-M/Agents over Remote Hosts are that agents support counters and multiple types of shouts:

  • Control-M/Agents can utilize features that require a Control-M counter. If you need any of these features, you should choose a Control-M/Agent over a Remote Host. For example, Control-M/Agents can use the File Watching feature, run jobs using Control-M/Agent utilities, and run jobs requiring Application plug ins for Control-M/Agent.

  • Control-M/Agents support many types of shouts. (Remote Host support is limited to e-mail shouts and Control-M/Server shouts.)

The Control-M/Agent manages the following job handling requests:

  • Instructions to submit a job on the Agent computer

  • Requests for information about jobs on the Agent computer that have been submitted, are currently executing, or have recently been completed

  • Requests to view or edit job script statements

  • Requests to view job output or job documentation

  • Requests to stop jobs that are currently executing

  • Group a number of Control-M/Agents together into a host group to balance processing load

Control-M/Agent and Other Platforms

Control-M supports the following Control-M/Agents that run on other platforms:

  • Control-M/Agent for UNISYS 2200: Supports running jobs on other platforms, and enables you to submit jobs for execution on the agent platform, monitor the jobs, and perform post-processing analysis of output files. The completion status of jobs and the results of post-processing analysis are transmitted to the Control-M/Server. For more information, see Control-M/Agent for UNISYS 2200

  • Control-M/Agent for IBM i (AS/400): Enables you to submit jobs for execution on the agent platform, monitor the jobs, and perform post-processing analysis of output files. The completion status of jobs and the result of post-processing analysis are transmitted back to the Control-M/Server. You can view and modify the following OS/400 type of jobs: External, Full, Multiple Commands, Program, VT. For more information, see Control-M/Agent for IBM i (AS/400)

  • Control-M/Agent for TANDEM Guardian: Supports running jobs on other platforms, and enables you to define jobs, retrieve job parameters, and control jobs running on the Guardian operating system. For more information, see Control-M/Agent for TANDEM Guardian

Infrastructure Components

The following table describes the Control-M/EM Infrastructure components:

Component

Description

Control-M/EM Configuration Agent

Provides Control-M/EM Server information to the Control-M Configuration Manager (CCM) and handles administrator requests from the Control-M Configuration Manager.

Control-M/EM database

Provides a repository of operational data relating to the functioning of all Control-M installations. Contents of the database include Active Jobs folders (job processing definitions), resources, and events. The database is maintained by Control-M/EM.

Naming Service (NS)

Enables each client to locate and connect to the GUI server via the Control-M Web Server.

Control-M Services

Services in Control-M are a set of microservices that are independent and can perform individual tasks without making requests from larger components. This enables your environment to use fewer resources and run more efficiently.

Services are available in the following environments:

  • Control-M/EM Services are Services Configuration Agent, Apache Kafka, Apache Zookeeper, Services Health Monitor, Protocol Translator, Validation Service, Reports Service, Control-M Web, and Scheduling Service.

  • Control-M/Server Services are Services Configuration Agent, Apache Kafka, Apache Zookeeper, Services Health Monitor, and Scheduling Service.

  • Control-M Full Install: Apache Kafka and Apache Zookeeper services are installed as part of the Control-M/Server installation and managed by it. If Control-M/Server has shut down gracefully, the service management is moved to Control-M/EM until Control-M/Server is restarted. In CCM the Apache Kafka and Apache Zookeeper services appear under Control-M/EM and Control-M/Server.

Control-M/EM Services

Control-M/EM services are a set of microservices that are independent and can perform individual tasks without making requests from larger components. This enables your environment to use fewer resources and run more efficiently.

The following table describes the Control-M/EM services that can be managed in the CCM, or in services-cli:

Service

Description

Services Configuration Agent

Functions as a watchdog process for all services, which is monitored by the Control-M/EM Configuration Agent. This service enables you to monitor and send commands for all services such as shutting down, starting up, ignoring services, viewing the status, and setting the log level by running the emsca utility.

Apache Kafka

An open source for publish and subscribe data streams between different applications.

If you are running Kafka on UNIX, a bash shell must be installed before you install Control-M/EM.

Apache Zookeeper

A centralized service for distributed systems, which Apache Kafka uses to coordinate between Apache Kafka instances.

Services Health Monitor

Produces health reports for all the service statuses from life checks sent through Apache Kafka.

Protocol Translator

Enables legacy applications to connect with subsystems, such as if you are using the CMS Server for health information, you can send commands to Apache Kafka to monitor services through Thrift.

Validation Service

Receives a definition folder validation request and sends a validation result response (using Kafka).

Reports Service

Creates and generates ad hoc reports, as described in Reports.

Control-M Web

Enables the Control-M Web application to function.

Scheduling Service

Calculates the dates that match the scheduling criteria of a job or SMART folder. The View Schedule option sends a request to the Scheduling Service to receive the dates that match the scheduling criteria for the selected jobs or folders.

Elastic Search

A scalable search engine, which searches, analyzes, and visualizes documents and datasets.

Kibana

Kibana is a free and open frontend application that sits on top of the Elastic Stack providing search and data visualization capabilities for data indexed in Elasticsearch.

Workflow Insights Service

Runs the Workflow Insights add-on in the Control-M environment, as described in Workflow Insights.

Application Integrator

Builds and customizes your own plug-ins for external applications beyond the built-in Control-M plug-ins.

Authorization Service

Responsible for external IdP communication and API token management.

EM-CTM Request Service

A channel between the Control-M/EM and the Control-M/Server, which is responsible for the user action requests log, why, and order.

EM-MFT Updates Service

A channel between the Control-M/EM and the Control-M/Server, which is responsible for the MFT file updates.

The Services Configuration Agent is a watchdog process for all services, which is monitored by the Control-M/EM Configuration Agent and responsible for starting all the services. When the Services Configuration Agent detects that if a service process is down but set to Up, the SCA starts it up and vice versa.

All services send life checks through Kafka. The Service Health Monitor monitors the life checks and produces a report with information of all services statuses. This information is sent to the Protocol Translator which translates it to a thrift message in the CMS. If the Service Health Monitor is not running, a message appears in the Control-M client, and services cannot be displayed.

The Protocol Translator has a REST API that is used by the services-cli utility to pass commands via Kafka to the services. Services can implement commands such as log level changes, show stack, and additional custom commands. The Protocol Translator also passes commands from the CMS through Thrift to Kafka and passes validation requests from the GUI Server through Kafka to the Validation service.

The system is optimized to a maximum of 10,000 jobs simultaneously in a single folder when using the emdef utility. If you need more, you might require adjusting memory parameters manually.

The following diagram shows the services architecture:

Control-M/Server Services

Control-M/Server services are a set of microservices that are independent and can perform individual tasks without making requests from larger components. This enables your environment to use fewer resources and run more efficiently.

The following table describes the Control-M/Server services that can be managed in the CCM:

Service

Description

Services Configuration Agent

Functions as a watchdog process for all services, which is monitored by the Control-M/Server Configuration Agent. This service enables you to monitor and send commands for all services such as shutting down, starting up, ignoring services, viewing the status, and setting the log level.

Apache Kafka

An open source for publish and subscribe data stream between different applications.

If you are running Kafka on UNIX, a Bash shell must be installed before you install Control-M/Server.

Apache Zookeeper

A centralized service for distributed systems, which Apache Kafka uses to coordinator between Apache Kafka instances.

Ctms API Gateway Service

Sends all requests to the relevant service.

Ctms App Updates Service

Provides a dedicated channel for Control-M/EM to query Server for MFT updates without any impact to job updates.

Ctms Job Info Service

Handles the waiting information (why) and job log requests.

Ctms Job Order Service

Handles all (New Day, ctmudly, ctmorder, request order from Control-M/EM) order requests.

Scheduling Service

Centralized service for distributed systems, which Apache Kafka uses to coordinate between Apache Kafka instances.

Services Health Monitor

Produces health reports for all the service statuses from life checks, which are sent through Apache Kafka.

The Services Configuration Agent is a watchdog process for all services, which is monitored by the Control-M/Server Configuration Agent and responsible for starting all the services. When the Services Configuration Agent detects that if a service process is down but set to Up, the SCA starts it up and vice versa.

All services send life checks through Kafka. The Service Health Monitor monitors the life checks and produces a report with information of all services statuses. If the Service Health Monitor is not running, a message appears in the Control-M client, and services cannot be displayed.

The following diagram shows the Control-M/Server services architecture: