Cloudera Manager Alerts for Unexpected Results Due to Region JMX Race Condition

0 votes
0 views
asked Aug 28, 2017 in Hadoop by admin (4,410 points)
SummaryCloudera Manager UI shows an alert when the CM agent gets a Null Pointer Exception response for a Web Metric JXM call to a Region Server. This error is benign and may be ignored.

Symptoms

If Cloudera Manager UI shows the following message:

The health test result for REGION_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent got an unexpected response from this role's web server

 

Applies To
  • HBase
  • Region Server
  • Cloudera Manager
  • Cloudera Manager Agent
Cause

The agent logs contains the following exception:

Sep 19, 1:53:08.836 AM ERROR org.mortbay.log
Nested in javax.management.RuntimeMBeanException: getMBeanInfo threw RuntimeException:
java.lang.NullPointerException



This exception is caused by HADOOP-11367[1], which is a race condition in the JMX code.
-------------
[1] - 'Fix a race condition in MetricsSourceAdapter.updateJmxCache' : https://issues.apache.org/jira/browse/HADOOP-11361

Instructions

This error message is essentially benign and can be ignored
 
To remove this error from the logs, the Region Server 'Web Meteric Collection' must be disabled:

  1. Navigate to HBase > Configuration
  2. Enter "Web Metric Collection" in the Search box
  3. Uncheck this property for 'Region Server'
  4. Click Save Changes

Please log in or register to answer this question.

...