hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Kelkar <rohitkel...@gmail.com>
Subject advice needed on storing large objects on hdfs
Date Fri, 27 Jan 2012 08:42:52 GMT
I am using hbase to store java objects. The objects implement the
Writable interface. The size of objects to be stored in each row
ranges from a few kb to ~50 Mb. The strategy that I am planning to use
if object size < 5Mb
store it in hbase
store it on hdfs and insert its hdfs location in hbase

While storing the objects I am using
WritableUtils.toByteArray(myObject) method. Can I use the
WritableUtils.toByteArray(myObject).length to determine if the object
should go in hbase or hdfs? Is this an acceptable strategy? Is the 5
MB limit a safe enough threshold?

- Rohit Kelkar

View raw message