Decommission or Recommission DataNode | HDFS Access | AnalysisException: Failed to load metadata for table

0 votes
13 views
asked Aug 21, 2017 in Hadoop by admin (4,410 points)

summary

After decommission or recommission was executed, HDFS access getting the following errors. 
ERROR: AnalysisException: Failed to load metadata for table: <table name> 
CAUSED BY: TableLoadingException: Failed to load metadata for table: <table name> 
CAUSED BY: CatalogException: Failed to create partition: 
CAUSED BY: RuntimeException: couldn't determine block locations for path 'hdfs://nameservice1/<file path>': java.lang.ArrayStoreException 
at java.util.ArrayList.toArray(ArrayList.java:389)

Symptoms

As an example, during the execution of a decommission or recommission of a Datanode, the following errors can be seen during an Impala Query when accessing blocks on an HDFS File System. The issue can also occur with other methods of HDFS block access.

ERROR: AnalysisException: Failed to load metadata for table: <table name>
CAUSED BY: TableLoadingException: Failed to load metadata for table: <table name>
CAUSED BY: CatalogException: Failed to create partition:
CAUSED BY: RuntimeException: couldn't determine block locations for path 'hdfs://nameservice1/<file path>': java.lang.ArrayStoreException
at java.util.ArrayList.toArray(ArrayList.java:389)
at org.apache.hadoop.hdfs.protocol.LocatedBlock.addCachedLoc(LocatedBlock.java:205)
at org.apache.hadoop.hdfs.server.namenode.CacheManager.setCachedLocations(CacheManager.java:907)

Applies To

CDH 5.5.1 or earlier
CDH 5.4.8 or earlier
CDH 5.3.9 or earlier

  • Cause

The deletion of a cached block is not reflected in a timely manner.
 

Instructions

  1. Upgrade to CDH 5.3.10, CDH 5.4.11, CDH 5.5.6, or CDH 5.6 and higher.
  2. Workaround:

Move the target file to another location (temporarily), then move it back to the original location.

Example:

$ sudo -u test hdfs dfs -mv /user/test/target.txt /tmp
$ sudo -u test hdfs dfs -mv /tmp/target.txt /user/test

Please log in or register to answer this question.

...