flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Anchlia <mohitanch...@gmail.com>
Subject Re: Configuration question
Date Sat, 31 Mar 2012 16:40:16 GMT
Thanks a lot!

How do people handle continuous feed into HDFS? Does everything go under
one directory? Can it be split into multiple files?

On Sat, Mar 31, 2012 at 1:07 AM, Will McQueen <will@cloudera.com> wrote:

> Hi Mohit,
>
> The avro-client sends a file that you specify (run "flume help" to see see
> the options) in avro format/protocol to the host and port  you specify. For
> example, you could setup a Flume NG avro source (specifying an avro source
> in your config file with a logger sink), and then run your avro-client
> against the flume node, looking at the logger output to confirm that the
> events are coming through.
>
> Cheers,
> Will
>
>
> On Fri, Mar 30, 2012 at 4:27 PM, Mohit Anchlia <mohitanchlia@gmail.com>wrote:
>
>> I got the package compiled. Going through the getting started guide,
>> what's the difference between flume node and flume avro-client? I see one
>> line description but don't understand when to run one over other?
>>
>>
>> On Thu, Mar 29, 2012 at 1:42 PM, Will McQueen <will@cloudera.com> wrote:
>>
>>> Hi Mohit,
>>>
>>> You might want to start with the Flume NG Getting Started guide at:
>>>      https://cwiki.apache.org/FLUME/getting-started.html
>>>
>>> We also have an architecture doc at:
>>>      https://blogs.apache.org/flume/entry/flume_ng_architecture
>>>
>>> Cheers,
>>> Will
>>>
>>> Cheers,
>>> Will
>>>
>>>
>>> On Thu, Mar 29, 2012 at 1:36 PM, Mohit Anchlia <mohitanchlia@gmail.com>wrote:
>>>
>>>> I am new to Flume. I am looking at Flume NG and HDFS as sink for it. I
>>>> am just trying to do basic testing to understand how it works. Some basic
>>>> questions I have is that do I need to also install flume on the client
>>>> machine, for eg web servers?
>>>>
>>>> Also, if for some reason server slows down or network connection breaks
>>>> then does Flume client picks up the data from where it stopped writing?
>>>> Wondering how that works
>>>>
>>>
>>>
>>
>

Mime
View raw message