Beeline Fails to Start | lang.OutOfMemoryError: Java heap space | Large History File

0 votes
3 views
asked Aug 30, 2017 in Hadoop by admin (4,410 points)
SummaryA large history file containing very long rows will consume a lot of heap, preventing Beeline from starting.
Applies To
Symptoms

Beeline fails to start with the following error:

Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit 
at java.util.Arrays.copyOf(Arrays.java:2271) 
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) 
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) 
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:122) 
at org.apache.hive.beeline.BeeLine.getConsoleReader(BeeLine.java:854) 
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:766) 
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:480) 
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:463) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:606) 
at org.apache.hadoop.util.RunJar.run(RunJar.java:221) 
at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 

Cause

A large Beeline history file (default ~/.beeline/history)  consumes heap memory.
But if the file has extremely long rows in it, it may cause this issue.

Note: Beeline only loads the latest 500 rows from the history file, so, this issue does not happen very often.

Instructions

Option 1:

Move the Beeline history file to another location.

Option 2:

Delete the Beeline history file.

Please log in or register to answer this question.

...