hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gibbon, Robert, VF-Group" <Robert.Gib...@vodafone.com>
Subject RE: Using HBase on other file systems
Date Fri, 14 May 2010 21:15:53 GMT
Hmm. What level of IOPs does Hbase need in order to support a reasonably responsive level of
service? How much latency in transfer times is acceptable before the nodes start to fail?
Do you use asynchronous IO queueing? Write-through caching? Prefetching?

On Fri, May 14, 2010 at 12:02 PM, Gibbon, Robert, VF-Group <
Robert.Gibbon@vodafone.com> wrote:

> My thinking is around separation of concerns - at an OU level not just at a
> system integration level. Walrus gives me a consistent, usable abstraction
> layer to transparently substitute the storage implementation - for example
> from symmetrix <--> isilon or anything in between. Walrus is storage
> subsystem agnostic, so it need not be configured for inconsistency like the
> Amazon service it emulates.
> Tight coupling for lock-in is a great commercial technique often seen with
> suppliers. But it is a bad one. Very bad.

However, reasonably tight coupling between a database (HBase) and its
storage layer (HDFS) is IMHO absolutely necessary to achieve a certain level
of correctness and performance. In HBase's case we use the Hadoop FileSystem
interface, so in theory it will work on anyone who has implemented said
interface, but I wouldn't run a production instance on anything but HDFS.

It's worth noting that most commercial databases operate on direct block
devices rather than on top of filesystems, so that they don't have to deal
with varying semantics/performance between ext3,ext4,xfs,ufs, myriad other
single-node filesystems that exist.


> -----Original Message-----
> From: Andrew Purtell [mailto:apurtell@apache.org]
> Sent: Thu 5/13/2010 11:54 PM
> To: hbase-user@hadoop.apache.org
> Subject: RE: Using HBase on other file systems
> You really want to run HBase backed by Eucalyptus' Walrus? What do you have
> behind that?
> > From: Gibbon, Robert, VF-Group
> > Subject: RE: Using HBase on other file systems
> [...]
> > NB. I checked out running HBase over Walrus (an AWS S3
> > clone): bork - you want me to file a Jira on that?

Todd Lipcon
Software Engineer, Cloudera

  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message