hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Hammerbacher <ham...@cloudera.com>
Subject Re: Using HBase on other file systems
Date Thu, 13 May 2010 04:26:11 GMT
Some projects sacrifice stability and manageability for performance (see,
e.g., http://gluster.org/pipermail/gluster-users/2009-October/003193.html).

On Wed, May 12, 2010 at 11:15 AM, Edward Capriolo <edlinuxguru@gmail.com>wrote:

> On Wed, May 12, 2010 at 1:30 PM, Andrew Purtell <apurtell@apache.org>
> wrote:
>
> > Before recommending Gluster I suggest you set up a test cluster and then
> > randomly kill bricks.
> >
> > Also as pointed out in another mail, you'll want to colocate TaskTrackers
> > on Gluster bricks to get I/O locality, yet there is no way for Gluster to
> > export stripe locations back to Hadoop.
> >
> > It seems a poor choice.
> >
> >   - Andy
> >
> > > From: Edward Capriolo
> > > Subject: Re: Using HBase on other file systems
> > > To: "hbase-user@hadoop.apache.org" <hbase-user@hadoop.apache.org>
> > > Date: Wednesday, May 12, 2010, 6:38 AM
> > > On Tuesday, May 11, 2010, Jeff
> > > Hammerbacher <hammer@cloudera.com>
> > > wrote:
> > > > Hey Edward,
> > > >
> > > > I do think that if you compare GoogleFS to HDFS, GFS
> > > looks more full
> > > >> featured.
> > > >>
> > > >
> > > > What features are you missing? Multi-writer append was
> > > explicitly called out
> > > > by Sean Quinlan as a bad idea, and rolled back. From
> > > internal conversations
> > > > with Google engineers, erasure coding of blocks
> > > suffered a similar fate.
> > > > Native client access would certainly be nice, but FUSE
> > > gets you most of the
> > > > way there. Scalability/availability of the NN, RPC
> > > QoS, alternative block
> > > > placement strategies are second-order features which
> > > didn't exist in GFS
> > > > until later in its lifecycle of development as well.
> > > HDFS is following a
> > > > similar path and has JIRA tickets with active
> > > discussions. I'd love to hear
> > > > your feature requests, and I'll be sure to translate
> > > them into JIRA tickets.
> > > >
> > > > I do believe my logic is reasonable. HBase has a lot
> > > of code designed around
> > > >> HDFS.  We know these tickets that get cited all
> > > the time, for better random
> > > >> reads, or for sync() support. HBase gets the
> > > benefits of HDFS and has to
> > > >> deal with its drawbacks. Other key value stores
> > > handle storage directly.
> > > >>
> > > >
> > > > Sync() works and will be in the next release, and its
> > > absence was simply a
> > > > result of the youth of the system. Now that that
> > > limitation has been
> > > > removed, please point to another place in the code
> > > where using HDFS rather
> > > > than the local file system is forcing HBase to make
> > > compromises. Your
> > > > initial attempts on this front (caching, HFile,
> > > compactions) were, I hope,
> > > > debunked by my previous email. It's also worth noting
> > > that Cassandra does
> > > > all three, despite managing its own storage.
> > > >
> > > > I'm trying to learn from this exchange and always
> > > enjoy understanding new
> > > > systems. Here's what I have so far from your
> > > arguments:
> > > > 1) HBase inherits both the advantages and
> > > disadvantages of HDFS. I clearly
> > > > agree on the general point; I'm pressing you to name
> > > some specific
> > > > disadvantages, in hopes of helping prioritize our
> > > development of HDFS. So
> > > > far, you've named things which are either a) not
> > > actually disadvantages b)
> > > > no longer true. If you can come up with the
> > > disadvantages, we'll certainly
> > > > take them into account. I've certainly got a number of
> > > them on our roadmap.
> > > > 2) If you don't want to use HDFS, you won't want to
> > > use HBase. Also
> > > > certainly true, but I'm not sure there's not much to
> > > learn from this
> > > > assertion. I'd once again ask: why would you not want
> > > to use HDFS, and what
> > > > is your choice in its stead?
> > > >
> > > > Thanks,
> > > > Jeff
> > > >
> > >
> > > Jeff,
> > >
> > > Let me first mention that you have mentioned some thing as
> > > fixed, that
> > > are only fixed in trunk. I consider trunk futureware and I
> > > do not like
> > > to have tempral conversations. Even when trunk becomes
> > > current there
> > > is no guarentee that the entire problem is solved. After
> > > all appends
> > > were fixed in .19 or not , or again?
> > >
> > > I rescanned the gfs white paper to support my argument that
> > > hdfs is
> > > stripped down. Found
> > > Writes at offset ARE supported
> > > Checkpoints
> > > Application level checkpoints
> > > Snapshot
> > > Shadow read only master
> > >
> > > hdfs chose features it wanted and ignored others that is
> > > why I called
> > > it a pure map reduce implementation.
> > >
> > > My main point, is that hbase by nature needs high speed
> > > random read
> > > and random write. Hdfs by nature is bad at these things. If
> > > you can
> > > not keep a high cache hit rate via large block cache via
> > > ram hbase is
> > > going to slam hdfs doing large block reads for small parts
> > > of files.
> > >
> > > So you ask. Me what I would use instead. I do not think
> > > there is a
> > > viable alternative in the 100 tb and up range but I do
> > > think for
> > > people in the 20 tb range somethink like gluster that is
> > > very
> > > performance focused might deliver amazing results in some
> > > applications.
> > >
> >
> >
> >
> >
> >
> I did not recommend anything
>
> "people in the 20 tb range somethink like gluster that is very
> performance focused might deliver amazing results in some
> applications."
>
> I used words like "something. like. might."
>
> It may just be an interesting avenue of research.
>
> And since you mentioned
>
> "also as pointed out in another mail, you'll want to colocate TaskTrackers
> on Gluster bricks to get I/O locality, yet there is no way for Gluster to
> export stripe locations back to Hadoop."
>
> 1) I am sure if someone was so included they could find a way to export
> that
> information from Gluster.
>
> 2) I think you meant DataNode not TaskTracker. In any case, I remember
> reading on list that a RegionServer is not guarenteed to be colocated with
> a
> datanode, especially after a restart. Someone was going to open a ticket
> for
> it.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message