Spark

Defines the parameters to execute Spark jobs

The following table describes the Spark parameters.

Parameter

Description

Program Type

Defines the Spark program type

Application Jar

Defines the path to the jar including your application and all the dependencies

Main Class to Run

Defines application's main class

Full Path to Script

Defines the full path to the python script to execute

Application Arguments

Defines the parameters that are added at the end of the Spark command line either after the main class for Java / Scala Applications or after the script of the Python Script

Command Line Options

Defines the sets of parameters and values that are added to the command line

Parent Topic

Hadoop Job parameters