hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject RE: Binary Input Files
Date Thu, 10 Mar 2011 01:27:30 GMT


Maha,

I haven't tried streaming, but ingestion of Binary data in to HBase means doing exactly what
you suggest. (Write your own BinaryInputFormat and define your own record splits.)

HTH

-Mike

> From: maha@umail.ucsb.edu
> Subject: Binary Input Files
> Date: Wed, 9 Mar 2011 16:20:39 -0800
> To: common-user@hadoop.apache.org
> 
> Hello,
> 
>    	I find my question in the Archives http://www.mail-archive.com/core-user@hadoop.apache.org/msg01750.html
> 
>    which is how to use a my binary files with my specific buffer protocol to with the
InputFormat. 
> 
>   The answer is suggesting some base64 conversion, which I think eliminate the benefits
of using Binary files. 
> 
>        If I decided to write my own InputFormat that defines Splits based on my binary
protocol and a recordReader also on my binary protocol. 
> 
>    Will that interfere with the streaming stuff ? or it is doable ?
> 
> Thank you,
> Maha
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message