flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Alten-Lorenz <wget.n...@gmail.com>
Subject Re: Max events per RPC call from across sink to Avro source
Date Wed, 17 Apr 2013 05:44:50 GMT
> Only for Avro sink,  it should be batch-size.  But every where else it is  batchSize.


Hmm, indeed. I filed a jira in your behalf (https://issues.apache.org/jira/browse/FLUME-2006)
, we're happy to fix this small issue.

- Alex

On Apr 17, 2013, at 1:17 AM, Madhu Gmail <madhu.munagala@gmail.com> wrote:

> I found the issue with my configuration.
> 
> I had batchSize  instead of batch-size
> 
> It was the property name for batch size.
> 
> Only for Avro sink,  it should be batch-size.  But every where else it is  batchSize.
> 
> 
> 
> Madhu  Munagala
> (214)679-2872
> 
> On Apr 16, 2013, at 4:45 PM, Chris Neal <cwneal@gmail.com> wrote:
> 
>> I believe that comment is indicating the default value, not the max.  Perhaps your
config is not correct?  
>> 
>> 
>> On Tue, Apr 16, 2013 at 4:16 PM, Madhu Gmail <madhu.munagala@gmail.com> wrote:
>> Hello,
>> 
>> I have flume agent configuration with sink batch size as 900.
>> 
>> Also I have another  flume agent working as collector.
>> 
>> Even though I have 900 as batch-size on agent sink,  I can only see 100 event batch
when the sink gets the data from file channel and sends it to collector.
>> 
>> I also see below comment in the code where it says max is 100 per RPC call
>> 
>> <td><tt>batch-size</tt></td>
>> * <td>The maximum number of events to send per RPC.</td>
>> * <td>events (int)</td>
>> * <td>100</td>
>> 
>> Madhu  Munagala
>> (214)679-2872
>> 

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF


Mime
View raw message