hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Seyvet <nicolas.sey...@gmail.com>
Subject Re: How to prevent major compaction when doing bulk load provisioning?
Date Thu, 21 Mar 2013 19:06:29 GMT
@Ram: You are entirely correct, I made the exact same mistakes of mixing up
Large and minor compaction.  By looking closely, what I see is that at
around 200 HFiles per region it starts minor compacting files per group of
10 HFiles.  The "problem" seems that this minor compacting never stops even
when there are about 20 HFiles left.  It just keep on going and on taking
more and more time (I guess because the files to compact are getting
bigger).

Of course in parallel we keep on adding more and more data.

@J-D: "It seems to me that it would be better if you were able to do a
single load for all your files." Yes, I agree.. but that is not what we are
testing, our use case is to use 1min batch files.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message