File Watcher (ctmfw)

The ctmfw (Control-M File Watcher) utility, enables you to detect file creation, file deletion, or the successful completion of a file transfer activity. You can use this utility before you activate a job or perform a task, such as the creation of an eventClosed An conditional entity that creates a sequential relationship between jobs by enabling the successor job to execute after the predecessor job has executed., which depends on a created file.

The Control-M Fire Watcher utility does not detect files with a modify time older than 50 years.

The ctmfw utility runs as a process on the Agent computer, which waits for the creation or deletion of a specified file.

The ctmfw utility does the following:

  • File Transfer Activity: Monitors the size of a file after it is detected. When the file reaches a predefined minimum size and does not increase in size for a specified period of time, the File Watcher utility either finishes with a status of OK or executes a specified If-Action, such as the creation or deletion of an event or a command execution.

  • File Creation: Ignores the file size if a wildcard is specified as part of the filename unless the mon_size_wildcard parameter is set to Y.

  • File Deletion: Monitors for the deletion of a file after detection.

The ctmfw utility runs from the CLI and can detect one or more files, as described in Watching a Single File and Watching Multiple Files.

There are two usages for this utility:

  • Service: (Windows only) As a service, ctmfw takes its parameters (rules) during startup from the rull.dat file, whose full pathname is specified in <Agent>\data\ctmfw.cfg. For more information, see Usage as a Service.

  • Running as a Utility: ctmfw is invoked from the CLI. Rules can be provided in the command line or by a rule file. You can watch for one or more files, as described in Watching a Single File and Watching Multiple Files.

    To allow other users to run ctmfw, they must have write permissions to $CONTROLM, and the CONTROLM environment variable must be set. For more information, see Running Agent Utilities as Other Users on UNIX. For details about ctmfw processing and return codes that are issued after the utility detects the creation or deletion of a file in the specified time frame, see ctmfw Utility Processing and Return Codes.

Usage as a Service

As a service, ctmfw takes its parameters (rules) during startup from the rull.dat file whose full path name is specified in <Agent>\data\ctmfw.cfg on Windows.

To change one or more rules, change the contents of the rull.dat file. The rull.dat file provided with Agent is a sample file and must be changed to reflect your requirements. The full pathname to the ctmfw.cfg configuration file must be specified under the following Windows registry key that is generated automatically by the installation script:

HKEY_LOCAL_MACHINE\SOFTWARE\BMC Software\CONTROL-M/FileWatcher\SYSPRM\File Watcher Configuration File

The default value for this key is <CONTROL-M/Agent_install_directory>\DATA\ctmfw.cfg

You need administrative permissions to update the registry key. It is recommended that this default value not be changed.

The configuration file must contain the following line:

-input <ruleFileName>

The variable <ruleFileName> is the full path name of a rule file containing the File Watcher rules.

The following example shows a sample rull.dat file:

INTERVAL 5

FROM_TIME 0001

MIN_SIZE 50

MIN_DETECT 5

WAIT_TIME 2

ON_FILEWATCH NONEXIST CREATE 10 3 1

THEN

DO_CMD "BAD FILE WAS CREATED IN 1 MINUTE"

ELSE

DO_CMD "GOOD FILE WAS NOT CREATED IN 1 MINUTE"

END_ON

The FileWatcher service running under the local system account cannot detect network resources (files located on remote systems). If you want the FileWatcher to detect network resources, configure the FileWatcher service to run under a regular user account instead of Local System Account.

When running as a service, ctmfw generates an execution log file. This file is saves in the Agent proclog directory under the following name:

U_CTMFW_<process_id>.log

The following example shows a Trace file output:

2017/03/10 13:04:24 182 FW:set INTERVAL=3

2017/03/10 13:04:24 182 FW:set MIN_SIZE=4

2017/03/10 13:04:24 182 FW:set FROM_TIME=0909

2017/03/10 13:04:24 182 FW:ctmfw:command line 'ON_FILEWATCH tst CREATE'(arg#=3).

id=1.

2017/03/10 13:04:24 182 FW:ctmfw:command line 'ON_FILEWATCH prd CREATE 0 0 1 NOW

10'(arg#=8)id=2

2017/03/10 13:04:24 182 FW:ctmfw:command line 'ON_FILEWATCH abc DELETE 0 0 1 NOW

10'(arg#=8)id=3

2017/03/10 13:04:24 182 FW:File 'test' exists, its current size is 265.

id=1.

2017/03/10 13:04:24 182 FW:File 'test' has reached the minimum size of 4.

size=265 bytes id=1.

2017/03/10 13:04:24 182 FW:File 'abc' does not exist. id=3.

2017/03/10 13:04:36 182 FW:File transfer was completed. The size of file 'test'

is 265. id=1.

2017/03/10 13:05:09 182 FW:Executing:< dir >

2017/03/10 13:05:27 182 FW:File 'prd' was not CREATED within the time limit.

id=2.

2017/03/10 13:05:27 182 FW:File prd will be scanned at 1315. id=2.

2017/03/10 13:05:27 182 FW:File 'abc' was not DELETED within the time limit.

id=3.

2017/03/10 13:05:27 182 FW:File abc will be scanned at 1315. id=3.

2017/03/10 13:05:30 182 FW:File prd, is out of time window. next time:1315, id=2.

2017/03/10 13:05:30 182 FW:File abc, is out of time window. next time:1315, id=3.

2017/03/10 13:15:01 182 FW:File prd, entered the time window from '1315' for

monitoring, id=2.

2017/03/10 13:15:01 182 FW:File abc, entered the time window from '1315' for

monitoring, id=3.

Logs in the proclog directory are retained for one day, by default. If the maximum days to retain output files parameter is set to a number higher than 1, logs are retained for the number of days specified in that parameter.

Watching a Single File

This procedure describes how to watch a single file with the ctmfw File Watcher utility. Rules can be provided in the command line or by a Rule file.

Begin

  1. Run the following command:

    ctmfw FILE (absolute path)

  2. Enter the required parameters.

    Copy
    < mode (CREATE|DELETE)> Default: CREATE
    < minimum detected size <number>
    [' '|Bytes(B)|Kilo(K)|Mega(M)|Giga(G)] >Default:0
    < interval between file search (seconds) > Default: 60sec
    < interval between filesize comparison iterations (seconds) > Default: 10sec
    < number of iterations while the size is static > Default: 3 iterations
    < time limit for the process (minutes).> Default: 0 (no time limit)
    Effective while the file does not exists or,
    the file size is static and the minimum size
    was not reached >
    < monitor file size , minimal and maximal age, when wildcard is used > Default: N
    < starting time for detecting files (HHMM or YYYYMMDDHHMM > Default: NOW
    < absolute stop time (HHMM or YYYYMMDDHHMM > Default: +0000 ( No stop time )
    < minimal age of file (modified time)
    format:xxxxYxxxxMxxxxDxxxxHxxxxMin > Default: NO_MIN_AGE
    < maximal age of file ( timestamp monitoring )
    format:xxxxYxxxxMxxxxDxxxxHxxxxMin > Default: NO_MAX_AGE

    All parameters must be assigned a value, even if that value is zero. If only six values are specified, the default value for mon_size_wildcard is used. If five parameters are specified, default values for wait_time and mon_size_wildcard are used.

    ctmfw /home/watchedfile.txt CREATE 100 10

    is resolved by using default values for mon_int, min_detect, wait_time, and mon_size_wildcard as follows:

    ctmfw /home/samplefile.txt CREATE 100 10 10 3 0 N

ctmfw Parameters

The following table lists the ctmfw utility rule file parameters:

Parameter

Description

FILE

Determines the path of the file to be detected. The filename can include the * wildcard character to represent any number of characters (including no characters) or ? to represent any one character.

The path and filename must not exceed 214 characters.

mode

  • CREATE (default): Detects the creation of a file. The file size is ignored if the filename parameter contains wildcards, (unless the monitor file size when a wildcard is used parameter is set to Y).

    If a mask is specified as the filename and the monitor file size, and a wildcard is used, the parameter can be set to the following:

    • N: Ends OK after detection of the first file that matches the specified mask.

    • Y: Ends OK after detection of the first file that matches the filename and file size.

  • DELETE: Detects deletion of a file. When the ctmfw utility is run in this mode, it first checks for files that match the specified name. After a specified file is detected, the ctmfw utility checks at the specified interval for deletion of that file.

    If a mask is specified as the filename, the ctmfw utility will end successfully only after all detected files that match the specified mask have been deleted.

minimum detected size

Determines the minimum file size in bytes. This parameter is ignored if the FILE parameter contains wildcards (unless the monitor file size when wildcard is used parameter is set to Y) or if the mode parameter is set to DELETE.

Default: 0 (any size detected).

interval between file searches

Determines, in seconds, the interval between successive attempts to detect the existence/deletion of a file.

Default: 60

interval between filesize comparison iterations

Determines, in seconds, the interval between attempts to monitor the size of a file after it is detected. This parameter is ignored when using wildcards in FILE or when using DELETE mode.

Default: 10

number of iterations while size is static

Determines the number of attempts at monitoring the file size, where the size remains static and greater than or equal to minimum detected size, (indicating successful creation of the file). This parameter is ignored when using wildcards in FILE or when using DELETE mode.

Default: 3

time limit for the process

Determines, in minutes, the maximum amount of time taken to run the process without detecting the file at its minimum size (CREATE), or detecting its deletion (DELETE). If the file is not detected/deleted in this specified time frame, the process terminates with an error return code, as described in Return codes.

Default: 0 (no time limit).

monitor file size when wildcard is used

Determines whether the file size should be monitored if the filename contains wildcards. This parameter is ignored if the filename does not contain a wildcard.

Valid values:

  • N Does not monitor file size (Default)

  • Y Monitors the file size

If this parameter is set to Y and more than one file matches the specified mask, the ctmfw utility selects the first file that is detected, monitors its file size, and ignores all other matching files.

starting time for detecting files

Determines an absolute time at which the utility starts monitoring the file. For example, 200712061400, means that at 2 PM on December 6th, 2007 the File Watcher utility will start watching the file.

Alternatively, you can use the HHMM format, in which case the current date is used.

absolute stop time

Determines an absolute time at which the file is no longer watched. For example, 200702061400, would mean that at 2 PM on February 6th, 2007 the File Watcher utility will stop watching the file. Alternatively, you can use the HHMM format, in which case the current date is used.

maximal age of file

Determines the maximum amount of time that can pass since the file you want to watch was last modified. For example, 2y3d5h means that after 2 years, 3 days, and 5 hours has passed, the file will no longer be watched. Entering a value of 2H10Min, means that after 2 hours and 10 minutes has passed, the file will no longer be detected.

This parameter is ignored if the mode parameter is set to DELETE.

Default: 0

If using the default value, files older than 50 years cannot be detected by the utility. If you want to detect files older than 50 years, you must define the number of years, such as 51y.

minimal age of file

Determines the minimum amount of time that must have passed since the file you want to watch was last modified. For example, 2y3d5h means that 2years, 3 days, and 5 hours must pass before the file will be watched. Entering a value of 2H10Min, means that 2 hours and 10 minutes must pass before the file will be detected. This parameter is ignored if the mode parameter is set to DELETE.

Default: 0

Watching Multiple Files

This procedure describes how to watch multiple files with the ctmfw File Watcher utility.

Begin

  • Type the following command to invoke the ctmfw utility for multiple files:

    ctmfw -input <ruleFileName>

    The variable <ruleFileName> is the complete path name of the file containing the definitions for each file to be detected. The .Rule file contains the following sections:

    • Global parameters, whose default values apply to all the files in the rule file. For more information, see ctmfw Rule File Global Parameters.

    • ON_FILEWATCH statements identifying which files to detect, specific criteria for each file, and the action to take upon detection or non-detection. Any number of ON_FILEWATCH statements can appear in a Rule file. For more information about ON_FILEWATCH parameters, see File Watcher (ctmfw).

    If any mandatory parameter is omitted from a Rule file, the default value for that parameter is used. Parameters entered for ON_FILEWATCH statements override the default values.

ctmfw Rule File Global Parameters

The following table lists the ctmfw Rule file global parameters:

Parameter

Description

CYCLIC_INTERVAL

Determines, in minutes, the interval between multiple operations of detecting the file. This interval must be greater than the value for WAIT_TIME. If the cyclic_interval is 0, only one attempt to detect the file will be performed.

Default: 0

FROM_TIME

Determines the starting time for detecting all the files (default FROM_TIME). Used with WAIT_TIME to identify the time frame for detecting and monitoring the files. This parameter is expressed in 24-hour, hhmm format.

Default: 0000 or Now

INTERVAL

Determines, in seconds, the Sleep interval between successive scans for all the files. This parameter replaces individual sleep_int and mon_int parameters for each file.

Default: 10

MAX_AGE

Determines the maximum amount of time that can pass since the file you want to watch was last modified.

  • If MAX_AGE = 0, any change to the file timestamp means that the event is met.

  • IF MAX_AGE = 10 Min and if the amount of time of the watched file that has passed is less than 10 minutes, then the event is met.

This parameter is ignored if the mode parameter is set to DELETE.

Default: 0

MIN_AGE

Determines the minimum amount of time that must have passed since the file you want to watch was last modified. For example, 2y3d5h means that 2years, 3 days, and 5 hours must pass before the file will be watched.

This parameter is ignored if the mode parameter is set to DELETE.

Default: 0

MIN_SIZE

Determines whether the file size should be monitored if the filename contains wildcards. This parameter is ignored if the filename does not contain a wildcard.

Valid values:

  • N – Does not monitor file size (Default)

  • Y – Monitors the file size

If this parameter is set to Y and more than one file matches the specified mask, the ctmfw utility randomly selects one matching file, monitors its file size, and ignores all other matching files.

STOP_TIME

Determines the absolute time at which the file is no longer watched. For example, 200702061400, means that at 2 PM on February 6th, 2007 the FileWatcher utility will stop watching the file.

You can also use the HHMM format, which uses the current date, plus the HHMM entered.

Default: 0 (meaning, no stop time)

STOP_TIME can only be used as a global parameter.

WAIT_TIME

Determines, in minutes, the maximum time to run the process without detecting the file at its minimum size (CREATE), or detecting its deletion (DELETE). If the file is not detected/deleted in this specified time frame, the process terminates with an error return code, as described in ctmfw- return codes table below.

Default: 0 (no time limit).

ctmfw Utility Processing

If the file is detected and the file size remains static within the time frame (CREATE) or the file has been deleted (DELETE), the DO commands in the THEN block are executed. If the file is not detected/deleted within the time frame, the statements following the ELSE block are executed.

CTMFW terminates when either all the files in the Rule file have been processed or a DO_OK/DO-NOTOK action is executed. If any ON_FILEWATCH statement contains a cyclic_interval, CTMFW only terminates on a DO_OK/DO-NOTOK action. If the cyclic_interval Global parameter is specified and not 0, CTMFW only terminates on a DO_OK/DO-NOTOK action or when the STOP_TIME was reached.

  • The sleep interval between succeeding scans must be 10 seconds.

  • If the ctmfw utility detects that the datafile.txt file in the /home/controlm directory is created in the specified time interval, then the command interpreter must execute the command to move the contents of the ~<controlm_owner>/ctm_server/datafile.txt file to~<controlm_owner>/ctm_server/workfile.txt.

  • If the ctmfw utility detects that the datafile.txt file in the ~<controlm_owner>/controlm directory is not created in the specified time interval then command that executed is "FILE WAS NOT CREATED".

  • When the ctmfw utility detects that the ~<controlm_owner>/ctm_server/tempfile.txt file is deleted, then the following command is executed "FILE WAS DELETED".

Copy
#******************************************************************
INTERVAL 10
ON_FILEWATCH ~<controlm_owner>\ctm_server\datafile.txt CREATE
THEN
DO_CMD move \ctm\datafile.txt \ctm\workfile.txt
ELSE
DO_CMD "FILE WAS NOT CREATED"
END_ON
ON_FILEWATCH \ctm\tempfile.txt DELETE
THEN
DO_CMD "FILE WAS DELETED"
END_ON
#*******************************************************************

Valid Actions

The following table lists the ctmfw valid actions:

If the file is detected and the size remains static within the time frame (CREATE) or the file has been deleted (DELETE), the DO commands in the THEN block are executed. If the file is not detected or deleted within the time frame, the statements following the ELSE block are executed. ctmfw terminates when all the files in the Rule file have been processed.

If an ON_FILEWATCH statement contains a cyclic_interval parameter, ctmfw will only stop monitoring a file on a DO_OK or DO_NOTOK action.

Action

Description

DO_CMD <command>

Execute a valid command under the command interpreter. Full path names are required for files.

DO_EXIT [exit code]

Terminate ctmfw with the user-defined exit code

DO_NOTOK [exit code]

Terminate an ON_FILEWATCH statement with status NOTOK. Exit code is optional and replaces the standard return code, as described in the ctmfw-return codes table below.

DO_OK

Terminate an ON_FILEWATCH statement with status OK. If there is more than one file in the Rule file, the result displayed is that of an AND algorithm.

Return Codes

The return codes listed in the following table are issued by the ctmfw utility after detecting if a file is created or deleted in the specified time frame.

Return Code

Description

0

File was successfully created or deleted (file arrived in the specified time frame and file size is above or equal to the minimum specified size).

1

  • Utility failed, for example, because of a syntax error.

  • A DO_NOTOK statement occurred, but no user-defined exit code was provided for the DO_NOTOK statement.

2

Indicates that the ctmfw request timed out. That is, the file was not detected in the specified time frame.