hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wayne <wav...@gmail.com>
Subject Re: Recommended Node Size Limits
Date Sat, 15 Jan 2011 13:45:41 GMT
Not everyone is looking for a distributed memcache. Many of us are looking
for a database that scales up and out, and for that there is only one
choice. HBase does auto partitioning with regions; this is the genius of the
original bigtable design. Regions are logical units small enough to be fast
to copy around/replicate, compact, and access with random disk I/O.
Cassandra has NOTHING like this. Cassandra partitions great to the node, but
then the node is one logical unit with some very very very big CF files.
What DBA can sleep at night with their data in 3x 350GB files? Our
definition of big data is big data ON the node and big data across the
nodes. Cassandra can not handle large nodes; 30 hour compaction is real on a
1TB node and is a serious problem with scale. HBase compresses with LZOP so
your data size is already much smaller, you can up the region size and
handle up to 100 "active" regions and a heck of lot more inactive regions
(perfect fit for time series data), and they all are compacted and accessed
individually. There is no common java limitation here...

I spent 6 months with Cassandra to get it to work perfectly, and then
totally abandoned it due to fundamental design flaws like this. This in
conjunction with having to ask ALL copies of data for a consistent read
which causes 3x the disk i/o on reads. Cassandra is not a good choice if
consistency is important or if scaling UP nodes is important. For those of
us looking to scale out/up a relational database (and we can not afford to
put 50TB in RAM) HBase is the only choice in the nosql space. Cassandra is a
great piece of software and the people behind it are top notch. Jonathan
Ellis has personally helped me with many problems and helped to get our
cluster to work flawlessly. Cassandra has many great uses and is a great
choice if you can keep most data in cache, but it is no relational database
replacement. I was bought into the hype and I hope others will be able to
get real assessments of differences to make the best decision for their

On Fri, Jan 14, 2011 at 10:50 PM, Edward Capriolo <edlinuxguru@gmail.com>wrote:

> On Fri, Jan 14, 2011 at 2:02 PM, Jonathan Gray <jgray@fb.com> wrote:
> > How about Integer.MAX_VALUE (or I believe 0 works) to completely disable
> splits?
> >
> > As far what we are running with today, we do have clusters with regions
> over 10GB and growing.  There has been a lot of work in the compaction logic
> to make these large regions more efficient with IO (by not compacting
> big/old files and such).
> >
> > JG
> >
> >> -----Original Message-----
> >> From: Ted Dunning [mailto:tdunning@maprtech.com]
> >> Sent: Friday, January 14, 2011 10:12 AM
> >> To: user@hbase.apache.org
> >> Subject: Re: Recommended Node Size Limits
> >>
> >> Way up = ??
> >>
> >> 1GB?
> >>
> >> 10GB?
> >>
> >> If 1GB, doesn't this mean that you are serving only 64GB of data per
> node?
> >>  That seems really, really small.
> >>
> >> On Fri, Jan 14, 2011 at 9:39 AM, Jonathan Gray <jgray@fb.com> wrote:
> >>
> >> > Then you can turn your split size way up, effectively preventing
> >> > further splits.  Again, this is for randomly distributed requests.
> >> >
> >
> I would think both systems (Cassandra || HBase) have large node
> pitfalls. If you want >500GB a node storage, low latency (<5ms), and
> random lookup you better have a lot of RAM!
> Current JVM's have diminishing returns past 24 GB of heap memory so in
> JVM caching for that volume is out. Both benefit from VFS cache, but
> both have cache churn from compacting.
> For applications that are mostly random read, the data to ram ratio is
> a biggest factor. I have been curious for a while on what peoples ram
> - data ration is.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message