User unable to write to HDFS Directory | Privilege is Granted in Sentry | Sentry HDFS sync is Enabled

0 votes
4 views
asked Aug 28, 2017 in Hadoop by admin (4,410 points)
SummaryA user is still not able to write to a directory that has been granted URI privilege via Sentry in the Sentry HDFS sync enabled cluster. This can be resolved by creating a dummy external table to link to the URI, then granting permissions on the table (rather than URI).

Symptoms
  • Sentry HDFS sync is enabled in the cluster
  • It has been confirmed the user has access to a directory:
    0: jdbc:hive2://ausplcdhedge03.us.dell.com:10> show grant role test_role;
    +---------------------------------+-------+-----------+--------+----------------+----------------+------------+--------------+------------------+---------+
    | database                        | table | partition | column | principal_name | principal_type | privilege  | grant_option | grant_time       | grantor |
    +---------------------------------+-------+-----------+--------+----------------+----------------+------------+--------------+------------------+---------+
    | hdfs://nameservice1/path/to/dir |       |           |        | test_role      | ROLE           | *          | false        | 1468340836037000 | --      |
    +---------------------------------+-------+-----------+--------+----------------+----------------+------------+--------------+------------------+---------+
  • Attempts to write to the directory via HDFS command will get the following error:
    2016-07-12 17:55:25,549 INFO [Thread-58] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: 
    Setting job diagnostics to Job setup failed : org.apache.hadoop.security.AccessControlException: 
    Permission denied: user=username, access=WRITE, inode="/path/to/dir":hive:hive:drwxrwxr-x
    at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:257)
    at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:238)
    at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:216)
    at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:145)
    at org.apache.sentry.hdfs.SentryAuthorizationProvider.checkPermission(SentryAuthorizationProvider.java:174)
Applies To

Hive
Sentry

Cause

There is currently no table in the Hive Metastore that links to the URI.    Sentry HDFS Synchronization will only sync HDFS ACL privileges for locations that contain databases or tables.

Instructions
  1. Create a dummy external table to link to the URI.
  2. Grant permissions on the table (rather than URI) so that privilege will be synced:
    CREATE DATABASE dummy; -- have any dummy tables under this database
    USE dummy;
    CREATE EXTERNAL TABLE dummy (a int) LOCATION "/path/to/dir";
    GRANT ALL ON TABLE dummy TO ROLE test_role;
  3.  Verify the user assigned to the role has write permission through the ACL and the hdfs command works.

For more details about Sentry HDFS Sync, please refer to the Synchronizing HDFS ACLs and Sentry Permissions from our official documentation.

Please log in or register to answer this question.

...