hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bradford Stephens <bradfordsteph...@gmail.com>
Subject Re: HBase Failing on Large Loads
Date Fri, 12 Jun 2009 17:40:08 GMT
Yeah, I was thinking about that. I guess data in HBase needs to be
sorted once, so data locality is a null issue on load. So, we can have
a separate HBase cluster and it wouldn't impact load performance any
more than having our Hadoop data 'crunching' cluster on the same boxes
as HBase? We plan on doing most of our MR tasks *on* documents in
HBase, however.

I'm running 2 maps and 1 reducer per machine. All the Map tasks are
done by the time we start reducin' and loadin'.

Sorry if this had too many run-on sentences :)

On Fri, Jun 12, 2009 at 10:31 AM, Ryan Rawson<ryanobjc@gmail.com> wrote:
> yeah, sounds like it.  How many maps are you running per machine?
> You could try making the hbase and mr machines disjoint?
> On Jun 12, 2009 10:20 AM, "Bradford Stephens" <bradfordstephens@gmail.com>
> wrote:
> I don't think we're using ZK, I'm on HBase-0.19.4...am I wrong? :)
> I've already got the GC configured to do what you suggested... I'm not
> getting very long pauses from the log file. I really think the problem
> is resource starvation because I only have 2 total cores on each of
> those boxes, and it's running HBase on top of Hadoop DataNodes and
> TaskTrackers. Am I right in this thinking?
> On Thu, Jun 11, 2009 at 10:29 PM, Ryan Rawson<ryanobjc@gmail.com> wrote: >
> Since you are on a 2-4 c...

View raw message