Web-based Hive with OneFS


Ambari Hive View is a lightweight tool for interacting with your Hortonworks or Open Data Platform cluster's Hive database. With just a few configuration steps you can get it working with Isilon's OneFS storage.


Ambari User


When you open Hive View and execute queries, the current Ambari Server user will be accessing Hive data on OneFS. If you accepted the defaults during Ambari setup, you are currently signed into Ambari Server as the user admin.


In default OneFS configuration, admin is a System zone user with limited file system action privileges. Predictably, trying to use Hive View as admin in the System zone is not going to work.

Here are a few errors you will see while attempting to use Hive View as admin user in OneFS's System zone.


In Hive View:

  • Permission denied: user=admin, access=READ, path="/user/admin/.AUTO_HIVE_INSTANCE.defaultSettings"
  • E090 HDFS020 Could not write file /user/admin/hive/jobs/hive-job-15-2016-08-17_12-00/query.hql [HdfsApiException]


In OneFS hdfs.log (example is from 8.0 or higher):

  • twilight-sparkle-18: 2016-08-17T12:00:51-07:00 <30.6> twilight-sparkle-18 hdfs[5695]: [hdfs] RPC V9 getFileInfo user: admin exception: org.apache.hadoop.security.AccessControlException cause: Permission denied: user=admin, access=READ, path="/user/admin/hive/jobs/hive-job-1-2017-08-17_12-00" <FS Exception at CreateHandleFromPath():397 status: STATUS_ACCESS_DENIED = 0xC0000022 Path: user/admin/hive/jobs/hive-job-1-2016-08-17_12-00>


We recommend that you create a new user in Ambari, for example ambari-view-user. This walk through assumes that you are authenticating with local users.


Enter the Manage Ambari page via the drop down on the top right of Ambari's UI. Create the user in the Manage Users page.




On your OneFS cluster create admin-view-user as well. You'll need to create a user directory and determine an appropriate group. We'll assume the group is Users.


isi auth users create --name ambari-view-user --primary-group Users --home-directory /ifs/%zone%/%hdfs_root%/user/ambari-view-user --enabled=yes --zone=%zone%


Give group ownership for the new user's HDFS user directory to hadoop.


chgrp hadoop /ifs/%zone%/%hdfs_root%/user/ambari-view-user


Kerberos and Proxy users


For now OneFS does not support Hive View in a Kerberos environment. If you correctly configure proxy users for the Hive View, you will still encounter an error stating that GetDelegationToken is not supported.


The Ambari documentation for Hive View has two preparation steps regardless of where you store HDFS data: create HDFS proxy users and user directories. The user directory was just taken care of, so all that is left is the proxy user.


After OneFS webhdfs implements GetDelegationToken, the proxy user configuration will be necessary in both Ambari and on OneFS. HiveServer2 checks the proxy configuration for its local HDFS client before attempting to access HDFS. OneFS needs a user account for the Ambari Server principal, and a proxy user setting that allows that account to act as ambari-view-user (or admin). The Hive proxy user needs to have ambari-view-user (or admin) added as a member. Follow the instructions in Ambari for configuring Views for Kerberos, and Hive View as well. Without the proxyusers settings added in Ambari, Hive View will fail to start even if the proxy users are correctly configured on OneFS.


HDFS and WebHDFS protocol access


Through Ambari 2.2.x


In Ambari 2.1 and Ambari 2.2, the Hive View works correctly when you use the default view Cluster Configuration. But if you need to use custom Hive View Cluster Configuration, be sure to use hdfs:// protocol and port 8020 for the WebHDFS FileSystem URI. This is within the Views page, accessible from the Manage Ambari page.




Ambari 2.4.0


Ambari 2.4.0 has a new Hive View 1.5.0. Both the old and new Hive View have changed the default HDFS protocol behavior. It now always uses WebHDFS if it is enabled, whereas views previously matched the HDFS Site's fs.defaultFS setting.


OneFS's WebHDFS is not currently compatible with Hive View. To work around this problem, use custom Cluster Configuration in the Hive View settings with hdfs:// as indicated in the above image.


Please see Hortonworks' Ambari documentation for details on the required settings in Hive View Cluster Configuration.