hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Shelukhin <>
Subject HiveQA machines are low on disk space
Date Mon, 25 Jan 2016 21:37:34 GMT

Recently, there has occasionally been a stuck mini cluster test, usually
MiniTez, I think I’ve seen MiniSpark too.
Upon looking at the logs for one recent timeout, I don’t see any obvious
problem with Hive, but there’s this:

2016-01-25T09:23:15,658 WARN
r@6845113[]]: namenode.FSNamesystem ( -
NameNode low on available disk space. Entering safe mode.
2016-01-25T09:23:15,658 INFO
r@6845113[]]: hdfs.StateChange ( -
STATE* Safe mode is ON.
Resources are low on NN. Please add or free up more resources then turn
off safe mode manually. NOTE:  If you turn off safe mode before adding
resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin
-safemode leave" to turn safe mode off.

After that, there are many safe-mode related errors.

I think that could be related to recent changes to preserve more logs. Can
we perhaps scale down log preservation a little bit, or gzip them?
It could also be something else on HiveQA machine.

View raw message