hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Сергей Казаков <ss.f...@gmail.com>
Subject BlockPoolSliceScanner and GC
Date Wed, 06 May 2015 14:02:48 GMT
Hi everyone!


We've been investigating some unpredictably long GC pauses in Hadoop
Datanode (2.5.1, Oracle JRE8_25) and noticed strange behaviour in
DataBlockScanner.

DataBlockScanner initiates block verification tasks during DN startup and
every 504 hours after. It holds some structures in memory heap (HashMap,
TreeSet & LightWeightGSet) which, in our case, are around 330Mb total (over
1.5m blocks). Are these structures useful after block verification task
completes? Tenuring and promoting these objects to OldGen causes some
terrible STW pauses we would like to avoid.


Kind regards,
Sergey Kazakov

Mime
View raw message