flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Young Kim <y...@crunchyroll.com>
Subject Re: flume-ng syslogtcp issue
Date Thu, 12 Jul 2012 05:38:31 GMT
After some debugging, I found out that the syslogTcp is working as intended. I didn't realize
that debug no longer shows the resulting syslog messages on success. On that note, this may
be a bit off-topic, but is there documentation showing how file_roll should be used to write
to local disk? 

Thanks,
Young 

On Wednesday, July 11, 2012 at 11:12 AM, Young Kim wrote:

> Hey Alex, 
> 
> The IP is correct, but I've altered it to say localhost. As for the syslog tests, I've
ran it verbatim, but it doesn't register. However, when I alter /tmp/foo to include two new
lines, flume registers it as a partial event. 
> 
> It still seems odd to me that flume-ng isn't detecting the syslogtcp whereas flume-og
is doing it perfectly. Is it possible that there is a bug in the way flume-ng is implementing
it? 
> 
> Thanks,
> Young
> 
> 
> On Tuesday, July 10, 2012 at 11:22 PM, alo alt wrote:
> 
> > HI,
> > 
> > try these for syslog tests:
> > echo "<13>Jun 20 12:12:12 host foo[345]: a syslog message" > /tmp/foo
> > nc -v FLUME SYSLOG AGENT 5140 < /tmp/foo
> > 
> > > syslog-agent.sources.syslog.host = 172.17.1.92
> > 
> > Means, will only listen only on this interface, is the IP correct?
> > 
> > cheers,
> > Alex
> > 
> > On Jul 10, 2012, at 10:59 PM, Young Kim wrote:
> > 
> > > Hey guys, 
> > > 
> > > I've been trying to use flume-ng (build in trunk) to read syslogs through tcp
then write to a flat file, as seen in this configuration:
> > > 
> > > # Source, sink, and channel name below
> > > syslog-agent.channels = rmc
> > > syslog-agent.sources = syslog
> > > syslog-agent.sinks = flat
> > > 
> > > # Configure channels
> > > syslog-agent.channels.rmc.type = memory
> > > 
> > > # Configure sources
> > > syslog-agent.sources.syslog.type = syslogTcp
> > > syslog-agent.sources.syslog.port = 5140
> > > syslog-agent.sources.syslog.host = 172.17.1.92
> > > syslog-agent.sources.syslog.channels = rmc
> > > 
> > > # Configure sinks
> > > syslog-agent.sinks.flat.type = FILE_ROLL
> > > syslog-agent.sinks.flat.directory = /Users/ykim/Desktop/test
> > > syslog-agent.sinks.flat.rollInterval = 0
> > > syslog-agent.sinks.flat.channels = rmc
> > > 
> > > 
> > > After a bit of wrangling with rsyslog configurations, it appeared as if rsyslog
wasn't sending any packets. However, I decided to try out flume-og with a simple dump syslogTcp,
and it correctly reads the syslog. At this point, I'm a bit lost and not sure what else to
look at. Hopefully, someone on here can shed some light on what could be wrong. 
> > > 
> > > Thanks,
> > > Young Kim
> > > 
> > 
> > 
> > 
> > --
> > Alexander Alten-Lorenz
> > http://mapredit.blogspot.com
> > German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> > 
> > 
> > 
> 
> 


Mime
View raw message