Showing ??? for column comment when describe table in beeline

0 votes
0 views
asked Aug 30, 2017 in Hadoop by admin (4,410 points)
Summary
Applies To

Hive Metastore Database

Symptoms

Create a table with non latin comment for columns:

CREATE TABLE <TBL> (col_name data_type COMMENT <non latin comments>)...

Beeline shows '???' for those columns when DESCRIBE <TBL>

Besides Beeline, below tools have the same problem:
HIVE CLI
HUE Hive Query Editor
HUE Metastore Manager

Cause

The root cause is that Metastore Database is not configured to support UTF-8 characters. Taking MySQL as an example:

>show create table COLUMNS_V2;
.....
COMMENT varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
‚Äč.....

Instructions

1) Change the character set to UTF-8 for the COMMENT column of COLUMNS_V2 table in the Metastore Database

>alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;

2) Update existing column comments for Hive tables by running below query in beeline:

>alter table <TBL> change <COL> <COL> <COL_TYPE> <NEW_COMMENT>

3) Restart Hive Metastore Service

Afterwards, column comments will be displayed correctly in Beeline, HIVE CLI, HUE Hive Query Editor and HUE Metastore Manager.

On some Linux environments, beeline may need additional configuration to display the characters correctly:

>HADOOP_CLIENT_OPTS=-Dfile.encoding=UTF-8 beeline ...

Please log in or register to answer this question.

...