Use the following instructions to configure Hadoop:
These configuration variables are under the [hadoop] section in the
hue.ini configuration file.
Configure HDFS Cluster.
Hue supports only one HDFS cluster currently.
Ensure that you define the HDFS cluster under the
[hadoop][[hdfs_clusters]][[[default]]]sub-section. Use the following variables to configure the HDFS cluster:- fs_defaultfs
This is equivalent to
fs.defaultFS(fs.default.name) in Hadoop configuration. For example,hdfs://fqdn.namenode.host:8020- webhdfs_url
You can also set this to be the WebHDFS URL. The default value is the HTTP port on the NameNode. For example,
http://fqdn.namenode.host:50070/webhdfs/v1- hadoop_hdfs_home
This is the home of your Hadoop HDFS installation. It is the root of the Hadoop untarred directory or usually
/usr/lib/hadoop.- hadoop_bin
Use this as the HDFS Hadoop launcher script, which is usually
/usr/bin/hadoop.- hadoop_conf_dir
This is the configuration directory of the HDFS, typically
/etc/hadoop/conf.
Configure YARN (MR2) Cluster.
Hue supports only one YARN cluster currently.
Ensure that you define the YARN cluster under the
[hadoop][[yarn_clusters]][[[default]]]sub-section. Use the following variables to configure the YARN cluster:- resourcemanager_host
The host running the ResourceManager.
- resourcemanager_port
The port for the ResourceManager IPC service.
- submit_to
Set this property to true. Hue will be submitting jobs to this YARN cluster. But note that JobBrowser will not be able to show MR2 jobs.
- hadoop_mapred_home
This is the home of your Hadoop MapReduce installation. It is the root of HDP Hadoop-MapReduce directory (
/usr/lib/hadoop-mapreduce). Ifsubmit_toistruefor this cluster, this configuration value is set as the$HADOOP_MAPRED_HOMEfor BeeswaxServer and child shell processes.- hadoop_bin
Use this as the YARN/MR2 Hadoop launcher script (
/usr/bin/hadoop).- hadoop_conf_dir
This is the configuration directory of the YARN/MR2 service, typically set to
/etc/hadoop/conf.- resourcemanager_api_url
The URL of the ResourceManager API. For example,
http://fqdn.resourcemanager.host:8088.- proxy_api_url
The URL of the ProxyServer API. For example,
http://fqdn.resourcemanager.host:8088.- history_server_api_url
The URL of the HistoryServer API. For example,
http://fqdn.historyserver.host:19888.- node_manager_api_url
The URL of the NodeManager API. For example,
http://fqdn.resourcemanager.host:8042.
