Hive | MSCK Repair table fails with "Unexpected component partitionvalue"

0 votes
asked Aug 30, 2017 in Hadoop by admin (4,410 points)


Hive command "MSCK REPAIR TABLE" command fails with:

10781 [main] WARN hive.ql.exec.DDLTask - Failed to run metacheck:
org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Unexpected component partitionvalue)
at org.apache.hadoop.hive.ql.exec.DDLTask.msck(
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(
at org.apache.hadoop.hive.ql.exec.Task.executeTask(
at org.apache.hadoop.mapred.YarnChild$
at Method)
at org.apache.hadoop.mapred.YarnChild.main(
Caused by: MetaException(message:Unexpected component partitionvalue)
at org.apache.hadoop.hive.metastore.Warehouse.makeValsFromName(
at org.apache.hadoop.hive.ql.exec.DDLTask.msck(
... 34 more
10794 [main] ERROR org.apache.hadoop.hive.ql.Driver - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Applies To

If you list out the contents of the HDFS directory of the table, you might see that there are directories for each partition.
MSCK expects the directory structure to be in format "partitioncolumn=value". For example a table with partitions year and month:
should be in format

This was the default behavior in some older versions however the MSCK command did not fail before, but silently did nothing.


The solution is to:
1. Rename directories to conform
2. create partitions manually with running "ALTER TABLE ... ADD PARTITION" commands.

It is not advised to use MSCK as part of "data load" instead of "ALTER TABLE ... ADD PARTITION".
MSCK is not designed for this usage and does not scale well as it needs to search/crawl through the hdfs directories. It is advised to use "ALTER TABLE ... ADD PARTITION" instead when loading a new partition.

Please log in or register to answer this question.