cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deeter, Derek" <derek.dee...@digitalinsight.com>
Subject Newbie question - logsandra Flume Sink plugin.
Date Fri, 07 Jan 2011 02:22:45 GMT
Hi -

I'm not sure if this should be asked on the Cassandra or Flume list, so I'm trying both -


I am trying to do a proof of concept for logging into Cassandra.  We need to capture large
volumes of audit events in a non-lossy manner at the same time being able to quickly access
those events for various reports, so this Flume-Cassandra combination seems to be just the
ticket.  For this POC, I've set up a ring/cluster of 4 Cassandra nodes and they are all running
successfully it seems (version 0.6.8), and I can do the simple entry and access of data by
hand using the cli.

Next I am trying to use Flume to push log data into Cassandra - I've set up the logsandra
keyspace for logsandraSyslogSink and the extra column families for the simpleCassandraSink,
set up the plugins for Flume, updated the flume-site.xml to add the plugin references, and
set the $FLUME_CLASSPATH all as mentioned on the https://github.com/thobbs/flume-cassandra-plugin
page.  However, I always get an "Invalid sink" error when I try to configure a Flume node
to have Cassandra as a sink.  Neither of these commands work:

172.16.199.166 : collectorSource(35853) | simpleCassandraSink("KeySpace1","data","FlumeIndexes","172.16.200.130:7005");
172.16.199.166 : collectorSource(35853) | logsandraSyslogSink("172.16.200.130:7005");

Can anyone please tell me if this is the correct command, or what the format should be?  It
is not really mentioned in the instructions.

I do see the following when starting the flume nodes, but not on the flume master:
2011-01-06 17:12:06,586 [main] INFO conf.SinkFactoryImpl: Found sink builder simpleCassandraSink
in org.apache.cassandra.plugins.SimpleCassandraSink
2011-01-06 17:12:06,586 [main] WARN conf.SinkFactoryImpl: No sink decorators found in org.apache.cassandra.plugins.SimpleCassandraSink
2011-01-06 17:12:06,586 [main] INFO conf.SinkFactoryImpl: Found sink builder logsandraSyslogSink
in org.apache.cassandra.plugins.LogsandraSyslogSink
2011-01-06 17:12:06,586 [main] WARN conf.SinkFactoryImpl: No sink decorators found in org.apache.cassandra.plugins.LogsandraSyslogSink

So it looks like the plugins are being loaded - without going into the source I am at a loss
at the 'Invalid Sink' error.  Any information appreciated -

            Thanks in advance,
            -Derek


--
Derek Deeter, Sr. Software Engineer           Intuit Financial Services
				                      5601 Lindero Canyon Road.
derek.deeter@digitalinsight.com                Westlake, CA 91362



Mime
View raw message