flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Alten-Lorenz <wget.n...@gmail.com>
Subject Re: Flume-ng 1.3.x HDFSSink - override Hadoop default size for only one of my sinks
Date Sat, 18 May 2013 10:50:32 GMT
Hi,

The HDFS sink writes back to HDFS, the blocksize is defined in your cluster (for new written
files). If you use the hdfs sink, you should have a hdfs-site.xml which defines the blocksize
(dfs.blocksize). So no, there is now way.

- Alex

On May 18, 2013, at 1:48 AM, Gary Malouf <malouf.gary@gmail.com> wrote:

> If it is not clear, I meant to type default block size.
> 
> 
> On Fri, May 17, 2013 at 7:46 PM, Gary Malouf <malouf.gary@gmail.com> wrote:
> Is there a way I can set the block size for files originating from a specific sink? 
My use case is that I have a number of different protobuf messages that each get written to
their own directories in HDFS.
> 

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF


Mime
View raw message