hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Chan" <cdj0...@gmail.com>
Subject Re: Re: How can I insert large image or video into HBase?
Date Thu, 24 Oct 2013 02:30:12 GMT
Big thanks to everybody  :)

Sorry that I am not good at English communication.But It's really good that I can receive
so many kind reply here.

Thanks to Michael for your kindly reply and recommending me Lars George and Nick Dimiduk's
book,I think that is <HBase, The Definitive Guide> and <HBase in Action>,

Roman,I really hoping for your solution on GItHub.

JM,thanks for your encouragement :)

to 谢良: 多谢回复,呵呵

Best Regards.

Jack Chan.

From: Jean-Marc Spaggiari
Date: 2013-10-23 22:46
To: Michael Segel
CC: user; cdj0579
Subject: Re: How can I insert large image or video into HBase?
To reply to Jack: There is no stupid question. What is stupid is to go in the wrong direction,
close its eyes, and run. Asking is always good. 

You got a very nicely detailed respons from Michael, so I don't think I have anything to add


2013/10/23 Michael Segel <michael_segel@hotmail.com>

Actually you didn't.

Its actually a good question and JM gave you the short answer. ;-)

First to clarify... depending on the size of the image or video you may want to actually store
the image/vid in a sequence file.
Then use HBase to index the file. (Its pretty much the same idea as what JM was saying but
that you have to remember that even Vid and image files may be too large to fit into HBase
yet too small to be separate files. )

This is one of the patterns that we've talked about at conferences. (Big Data TechCon) and
its covered in Boris Lublinsky, Kevin Smith and Alexey Y's new book.
(Its a Wiley publication.)

The longer answer...

In HBase your row cannot exceed the width of the region.  So you end up with very large regions,
with a single row and you have to cap the size of the video or image.
HBase doesn't really perform well and of course performance would suck when you tried to insert
or read the row.
(You can go through examples in either Lars George's book or Nick Dimiduk ?sp? book on what
happens during reads and writes.) Unless the row is in cache, reads and writes to HBase are
slower than reading or writing to HDFS.

The reason I like the question is that it forces people to think about how HBase works and
why it may not be the best option as a stand alone solution. ;-)

On Oct 23, 2013, at 1:38 AM, Jack Chan <cdj0579@gmail.com> wrote:

> Hi JM,
> Thanks for your reply,It seems I really asked a stupid question :)

> Jack Chan.
> From: Jean-Marc Spaggiari
> Date: 2013-10-23 12:12
> To: user; cdj0579
> Subject: Re: How can I insert large image or video into HBase?
> Put your file into HDFS and store only the name into HBase. HBase is not done do store
large files.
> JM
> 2013/10/23 Jack Chan <cdj0579@gmail.com>
> Hi All:
> This could be a stupid question.But here it goes....
> We knew that we can use "put" to insert some small files by converting it to bytes first.
> But for a large file,I think we would better stream it first.
> So,how can we insert the large file into HBase through Java code using the stream way?
> Thanks and regards
> Jack Chan.

The opinions expressed here are mine, while they may reflect a cognitive thought, that is
purely accidental.
Use at your own risk.
Michael Segel
michael_segel (AT) hotmail.com
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message