hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-1044) Make hbase more less of a fainting lily when running beside a hogging tasktracker
Date Wed, 03 Dec 2008 19:07:44 GMT
Make hbase more less of a fainting lily when running beside a hogging tasktracker

                 Key: HBASE-1044
                 URL: https://issues.apache.org/jira/browse/HBASE-1044
             Project: Hadoop HBase
          Issue Type: Bug
            Reporter: stack

>From IRC (with some improving text added -- finishes on a good point made by jgray):

18:53 < St^Ack> The coupling in a MR cluster is looser than it is in hbase cluster
18:53 < St^Ack> TTs only need report in every ten minutes and a task can fail and be
18:54 < St^Ack> Whereas with hbase, it must report in at least every two minutes (IIRC)
and cannot 'redo' lost edit -- no chance of a redo
18:54 < St^Ack> So, your MR job can be rougher about getting to the finish line; messier.
18:55 < tim_s> yeah. 
18:55 < St^Ack> If MR is running on same nodes as those hosting hbase, then it can rob
resources from hbase in a way that damages hbase but not TT
18:56 < St^Ack> So, maybe we need to look at the hbase config; make it more tolerant
when its running beside a hogging TT job
18:57 < tim_s> hmm, that would be lovely
18:57 < St^Ack> Need to look at HDFS; see how 'fragile' it is too; hbase should be at
least that 'fragile'
18:57 < jgray> yeah, most issues we see come from resource issues on shared hdfs/tt/rs
18:57 < tim_s> so is it common to host hbase elsewhere?
18:57 < St^Ack> Let me make an issue on it because this is common failure case for hbase
(Setup hbase then run your old MR job as though nothing has changed -- then surprise when
the little hbase lady faints)
18:57 < jgray> tim_s: currently, no.  common practice is shared
18:58 < St^Ack> ... and its better if shared -- locality benefits
18:58 < tim_s> would that be a good idea though? cause I don't really need to have hadoop
local I guess.
18:58 < tim_s> ahh
18:58 < apurtell> we share also
18:59 < jgray> beyond locality, sharing makes sense as hdfs and hbase nodes have different
requirements... hdfs being heaviest on io (where hbase has no use), hbase heavy in memory,
TTs vary greatly but most often heavy in cpu/io

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message