For backwards compatibility, output to S3 defaults to using the original file committer.

To switch to the Directory Committer in Ambari, edit core-site.xml and set the property fs.s3a.committer.name to directory.

<property>
  <name>fs.s3a.committer.name</name>
  <value>directory</value>
</property>

The other values for this option are file, partitioned and magic; file is the original file output committer, which uses file and directory renames to commit output. The partitioned committer is a variant of the directory committer, with different conflict resolution policies. The magic committer is a potentially faster committer, but which should be considered less tested.