flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Novogrodsky <david.novogrod...@gmail.com>
Subject Can a Flume agent listen for a JSON/CSV file on multiple ports?
Date Wed, 11 Feb 2015 18:29:01 GMT

I have a general question regarding Flume.

We have some agents that write and send files from several host
machines.  The agent will be installed on several hundred machines.
The file will contain host information in a header in JSON fomat.  The
detail records, the rest of the file, will be in CSV format.  Each
host will be communicating through its own port.

1. Can a Flume agent listen on multiple ports?  We do not want a Flume
agent for each port.
2. Is there an out-of-the-box Flume agent that can handle JSON and CSV fformat?
3. We can change the format of the incoming file.  Would it be simpler
if each record was atomic and did not need header files??

David Novogrodsky

View raw message