Distributed Shell

Defines all the parameters required to execute Distributed Shell jobs.

The following table describes the Distributed Shell parameters.

Parameter

Description

Shell Type

Determines whether the execution type is from a command or from a script file.

Command

Defines the shell command entry to run for the job execution.

Script Full Path

Defines the full path to the script file which is executed. The script file is located in the HDFS.

Shell Script Arguments

Defines the shell script arguments.

Files/Archives

Defines the full path to the file/archive to upload as a dependency to the HDFS working directory.

Options

Defines the additional option to set when executing the job.

Environment Variables

Defines the environment variables for the shell script/command.

Append Yarn aggregated logs to output

Determines whether to add yarn aggregated logs to the jobs output.

Parent Topic

Hadoop Job parameters