Couldn't Fetch Delegation Token Error with Sqoop Import into Hive Action in Oozie Workflow

0 votes
29 views
asked Aug 28, 2017 in Hadoop by admin (4,410 points)
Summary

Symptoms

When attempting to run an Oozie workflow containing a Sqoop1 action with --as-parquetfile and --hive-import on a secured cluster, it fails with the following exception:

9533 [uber-SubtaskRunner] ERROR org.apache.sqoop.mapreduce.ParquetJob - Couldn't fetch delegation token. 
MetaException(message:Delegation Token can be issued only with kerberos authentication. Current AuthenticationMethod: TOKEN)


NOTE: When trying these actions outside of Oozie on their own, ( i.e. Hive Action and Sqoop import action), the operations are successful. 

Applies To

Oozie workflows with Sqoop1 Actions with Parquet on CDH 5.x on a secured cluster
 

Cause

The cause of this issue is due to how ParquetJob checks whether there is a Hive token in the job credentials by calling conf.getCredentials.getToken() with the alias "HCat Token". It is a code-level issue where it is not finding delegation tokens issued by Oozie as Oozie uses a different alias. This issue occurs when a Sqoop import operation is called by Oozie using --as-parquet-file and --hive-import as options on a secure cluster.

Instructions

Upstream Jira SQOOP-3177 has been filed to resolve the reported issue long-term, however the recommended best practice is to perform the following:

Modify the existing workflow that performs the operation and split the workflow into to parts:

1) Sqoop1 Action (import into HDFS using --target-dir)
2) HS2 Action (load data from --target-dir into Hive table)

Please log in or register to answer this question.

...