When moving from the Spark Tech Preview to the full HDP version of Spark, make sure that the hive.metastore.uris property in your hive-site.xml file is set to the Hive Metastore URI in your cluster. The hive-site.xml file typically resides in /usr/hdp/current/spark-client/conf/.

Example:

<configuration> 
<property>
    <name>hive.metastore.uris</name>
    <value>thrift://blue1:9083</value>
    <description>URI for client to contact metastore server</description>
</property>
</configuration>

When you install Spark 1.3.1 using Ambari or the manual installation process, Spark creates and populates the hive-site.xml file -- you no longer need to create hive-site.xml.