Hue job browser not displaying current jobs after CDH upgrade

0 votes
asked Aug 19, 2017 in Hadoop by admin (4,410 points)
SummaryAfter CDH upgrade, users may notice they can not see recent submitted jobs from Hue Job Browser

After an upgrade to one of the releases below, the Hue Job Browser may not displaying recent jobs.  YARN ResourceManager and JobHistory Servers are displaying jobs as expected.

Applies To
  • Hue Job Browser
  • CDH 5.7.4 and higher
  • CDH 5.8.2 and higher
  • CDH 5.9.0 and higher
  • CDH 5.10.0 and higher

An improvement to Hue performance was made to lower the number of jobs displayed on the main page to 1000 (HUE-4540).   Because the Resource Manager returns the oldest rows first, as a result recently submitted jobs may not get displayed in Hue.


Workaround 1

An improvement was made to HUE to allow filtering the jobs by start time (HUE-5602).  With this improvement, users can filter jobs by start time (day, hour, minute) to find the jobs they need.
Upgrade to one of the following releases to obtain this improvement:
  • CDH 5.9.2 and higher
  • CDH 5.10.1 and higher
  • CDH 5.11.0 and higher

Workaround 2

Apply a username filter to display the most recent jobs for a user.

Workaround 3

Change the Resource Manager retention to be 1000 to match what Hue displays. Hue will get every job that the Resource Manager has in its own history.  To make this change in the configuration, in Cloudera Manager:
  1. Navigate to YARN > Configuration
  2. Search for "Max Completed Applications"
  3. Set this value to 1000
  4. Save Changes and restart the Resource Manager.
WARNING:   With this change if your cumulative jobs per day goes beyond 1000, only the latest 1000 jobs will be available in Hue.   The jobs will still be visible in the JobHistory Server's UI.

Please log in or register to answer this question.