Configuring a Job for Dynamic Resource Allocation
There are two ways to customize dynamic resource allocation properties for a specific job:
- Include property values in the - spark-submitcommand, using the- -confoption.- This approach loads the default - spark-defaults.conffile first, and then applies property values specified in your- spark-submitcommand. Here is an example:- spark-submit --conf “property_name=property_value”
- Create a job-specific - spark-defaults.conffile and pass it to the- spark-submitcommand.- This approach uses the specified properties file, without reading the default property file. Here is an example: - spark-submit --properties-file <property_file>

