Kudu services crash with an error reading clock

0 votes
4 views
asked Oct 22, 2017 in Hadoop by anonymous
Summary
Symptoms
Kudu Master or TabletServer goes down with the following fatal error:
 
hybrid_clock.cc:227] Couldn't get the current time: Clock unsynchronized. Status: Service unavailable: Error reading clock. Clock considered unsynchronized
Applies To
  • All Kudu versions
Cause

If ntpd is started with the "-x" flag it means slewing is disabled. This means that it won't use the kernel's frequency adjustment support (via the adjtimex syscall) to manage the time. Instead, it just uses the 'adjtime' call in libc to adjust the clock offset. This has an underlying effect in the kernel of increasing or decreasing the frequency of the clock, but notable does not adjust the 'maxerror' variable inside the kernel. So, the kernel just adds 500us of maxerror once a second, and once that crosses a threshold of 16 sec (after about 8h 53m), the clock gets the "STA_UNSYNC" flag set and crashes Kudu.

Instructions
  • Restart ntpd without the "-x" flag

Please log in or register to answer this question.

...