Cannot Use HDFS Disk Usage Report Search Files and Manage Directories | Error Could not access index for service 'hdfs1'

0 votes
4 views
asked Aug 19, 2017 in Hadoop by admin (4,410 points)
SummaryIssue stems from an inconsistent Lucene Index. Resolved by rebuilding index.
Symptoms
Navigating to Cloudera Manager > Clusters Tab > Reports > Search Files and Manage Directories (button), gives the following error:
The Reports Manager (reportsmanager (hostname)) could not complete the request: Could not access index for service 'hdfs1', nameservice 'nameservice1'.

Exception:
2014-10-07 15:00:04,321 INFO com.cloudera.headlamp.HeadlampIndex: Starting up search service
2014-10-07 15:00:06,332 ERROR com.cloudera.headlamp.HeadlampIndex: Error starting search service
java.io.IOException: read past EOF
at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:207)
at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
at org.apache.lucene.store.ChecksumIndexInput.readByte(ChecksumIndexInput.java:40)
at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:71)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:260)
at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:79)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:736)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:428)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:288)
at org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:82)
at com.cloudera.headlamp.HeadlampServiceImpl.setData(HeadlampServiceImpl.java:203)
at com.cloudera.headlamp.HeadlampIndex.loadNewSearchIndex(HeadlampIndex.java:241)
at com.cloudera.headlamp.HeadlampIndex.<init>(HeadlampIndex.java:96)
at com.cloudera.headlamp.HeadlampIndexManager.getOrCreateIndex(HeadlampIndexManager.java:153)
at com.cloudera.headlamp.HeadlampIndexManager.reindexIndexes(HeadlampIndexManager.java:229)
at com.cloudera.headlamp.HeadlampIndexManager$1.run(HeadlampIndexManager.java:430)
2014-10-07 15:00:06,333 ERROR com.cloudera.headlamp.HeadlampIndexManager: Index build failed for service hdfs1
Applies To
  • Cloudera Manager
  • HDFS Disk Usage Reports
  • Search Files and Manage Directories
Cause

Inconsistent Lucene Index

Instructions
To resolve this issue:
  • Upgrade to Cloudera Manager 5.9.0 and above. Cloudera Manager 5.9.0 has upgraded Lucene core to V3.4 and user should not encounter this issue any more.
  • Other Workaround if can not upgrade:
  1. Stop ReportsManager

Cloudera Manager > Cloudera Management Service > Reports Manager > Action > Stop

  1. Find Configuration Reports Manager Log Directory 

Cloudera Manager > Cloudera Management Service > Reports Manager > Configuration

  1. Move contents of Reports Manager Log Directory 

For example, mv /var/lib/cloudera-scm-headlamp/* /somewhere_else/

  1. Start Reports Manager

Cloudera Manager > Cloudera Management Service > Reports Manager > Action > Start

  1. Wait for a period of time for the Reports Manager to create the index.
  2. Reports Manager begins a new fetching for the HDFS fsimage and creating a new index. Allow time for this to complete (this varies between seconds to 30 minutes, depending on the size of the cluster's fsimage).
  3. View 'Disk Usage by ____' Reports.

Please log in or register to answer this question.

...