hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Recommended Node Size Limits
Date Sat, 15 Jan 2011 14:16:19 GMT
On Sat, Jan 15, 2011 at 8:45 AM, Wayne <wav100@gmail.com> wrote:
> Not everyone is looking for a distributed memcache. Many of us are looking
> for a database that scales up and out, and for that there is only one
> choice. HBase does auto partitioning with regions; this is the genius of the
> original bigtable design. Regions are logical units small enough to be fast
> to copy around/replicate, compact, and access with random disk I/O.
> Cassandra has NOTHING like this. Cassandra partitions great to the node, but
> then the node is one logical unit with some very very very big CF files.
> What DBA can sleep at night with their data in 3x 350GB files? Our
> definition of big data is big data ON the node and big data across the
> nodes. Cassandra can not handle large nodes; 30 hour compaction is real on a
> 1TB node and is a serious problem with scale. HBase compresses with LZOP so
> your data size is already much smaller, you can up the region size and
> handle up to 100 "active" regions and a heck of lot more inactive regions
> (perfect fit for time series data), and they all are compacted and accessed
> individually. There is no common java limitation here...
>
> I spent 6 months with Cassandra to get it to work perfectly, and then
> totally abandoned it due to fundamental design flaws like this. This in
> conjunction with having to ask ALL copies of data for a consistent read
> which causes 3x the disk i/o on reads. Cassandra is not a good choice if
> consistency is important or if scaling UP nodes is important. For those of
> us looking to scale out/up a relational database (and we can not afford to
> put 50TB in RAM) HBase is the only choice in the nosql space. Cassandra is a
> great piece of software and the people behind it are top notch. Jonathan
> Ellis has personally helped me with many problems and helped to get our
> cluster to work flawlessly. Cassandra has many great uses and is a great
> choice if you can keep most data in cache, but it is no relational database
> replacement. I was bought into the hype and I hope others will be able to
> get real assessments of differences to make the best decision for their
> needs.
>
>
> On Fri, Jan 14, 2011 at 10:50 PM, Edward Capriolo <edlinuxguru@gmail.com>wrote:
>
>> On Fri, Jan 14, 2011 at 2:02 PM, Jonathan Gray <jgray@fb.com> wrote:
>> > How about Integer.MAX_VALUE (or I believe 0 works) to completely disable
>> splits?
>> >
>> > As far what we are running with today, we do have clusters with regions
>> over 10GB and growing.  There has been a lot of work in the compaction logic
>> to make these large regions more efficient with IO (by not compacting
>> big/old files and such).
>> >
>> > JG
>> >
>> >> -----Original Message-----
>> >> From: Ted Dunning [mailto:tdunning@maprtech.com]
>> >> Sent: Friday, January 14, 2011 10:12 AM
>> >> To: user@hbase.apache.org
>> >> Subject: Re: Recommended Node Size Limits
>> >>
>> >> Way up = ??
>> >>
>> >> 1GB?
>> >>
>> >> 10GB?
>> >>
>> >> If 1GB, doesn't this mean that you are serving only 64GB of data per
>> node?
>> >>  That seems really, really small.
>> >>
>> >> On Fri, Jan 14, 2011 at 9:39 AM, Jonathan Gray <jgray@fb.com> wrote:
>> >>
>> >> > Then you can turn your split size way up, effectively preventing
>> >> > further splits.  Again, this is for randomly distributed requests.
>> >> >
>> >
>>
>> I would think both systems (Cassandra || HBase) have large node
>> pitfalls. If you want >500GB a node storage, low latency (<5ms), and
>> random lookup you better have a lot of RAM!
>>
>> Current JVM's have diminishing returns past 24 GB of heap memory so in
>> JVM caching for that volume is out. Both benefit from VFS cache, but
>> both have cache churn from compacting.
>>
>> For applications that are mostly random read, the data to ram ratio is
>> a biggest factor. I have been curious for a while on what peoples ram
>> - data ration is.
>>
>


Defiantly the Cassandra compaction large disk volumes is less elegant
then the small region design. I think the best alternative their is to
lose your RAID-0/ RAID5 and just run one Cassandra instance per JBOD
disk. There is some overhead and management with that but hey then the
largest compaction are not much of an issue any more. (its something
like a virtualized solution)

K. It seemed from your post asking "can hbase handle 1k regions per
server" you were not sure if hbase could handle it either. However it
sounds like you have it figured it out.

Mime
View raw message