Hive | Loading an updated UDF Class

0 votes
2 views
asked Aug 30, 2017 in Hadoop by admin (4,410 points)
SummaryWhen replacing an already loaded JAR file, Hive does not read the updated class in the new UDF JAR file.
Applies To
  • Hive
  • JAR file
Symptoms

With CDH 5.7.0 and later, there is a new HS2 (HiveServer2) property that allows for reloading of jar files: "hive.reloadable.aux.jars.path". In that case please follow the official documentation for your release of CDH:
https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_hive_udf.html#concept_zb2_rxr_lw

When replacing an already loaded JAR file, Hive does not read the updated class in the new UDF JAR file.

Cause

Hive keeps the loaded UDF classes in memory, and does not see the updated class definitions if JAR file is replaced/updated.

Instructions

Option 1:

Restart HiveServer2 after the JAR file is updated.

Note: This option is unfavorable in development systems where constant updates and testing is required.

Option 2: 

Use versioning of UDF JAR files.

Example:

A JAR file with name udf-classes.jar, can be renamed to:

udf-classes-1.0.jar
udf-classes-1.0.1.jar
udf-classes-2.0.jar

So when a new version is released, simply drop the function, re-create it using the new JAR, and the new function will be ready to be used.

For example:

-- initially created function "test"
CREATE FUNCTION test AS 'package_name' USING JAR '/uri/udf-classes-1.0.jar';

-- after new jar is in place, update the "test" function
DROP FUNCTION test;
CREATE FUNCTION test AS 'package_name' USING JAR '/uri/udf-classes-2.0.jar';

Please log in or register to answer this question.

...