flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaurav Khanna <khannap...@yahoo.com>
Subject Re: HBase and Flume
Date Thu, 02 Feb 2012 18:57:40 GMT
Got the information at:
https://github.com/cloudera/flume/tree/master/plugins/flume-plugin-hbasesink

Thanks
Gaurav



________________________________
 From: Gaurav Khanna <khannapost@yahoo.com>
To: "flume-user@incubator.apache.org" <flume-user@incubator.apache.org> 
Sent: Thursday, February 2, 2012 10:54 AM
Subject: HBase and Flume
 

Hi,
Am using flume Flume 0.9.4-cdh3u2 and wanted to use the HBase Sink. 

Entered this as the sink: hbase("testtable", "%{nanos}", "colfam1", "q1", "%{body}")

and got the following in the master logs (now I know that I am missing some configuration
steps - please can anybody point me to where I can get the Hbase related plugin and how to
configure that?):

2012-02-02 12:41:36,916 [exec-thread] INFO master.CommandManager: Executing command: config
[POC_Hadoop_Client_1.ssodev-bo.mtv1.vrsn.com, tail("/app/log-file-gen/out/server.log"), hbase("testtable",
"%{nanos}", "colfam1", "q1", "%{body}")]
2012-02-02 12:41:36,917 [exec-thread] ERROR master.CommandManager: Unexpected exception during
cmdid:0 config
 [POC_Hadoop_Client_1.ssodev-bo.mtv1.vrsn.com, tail("/app/log-file-gen/out/server.log"), hbase("testtable",
"%{nanos}", "colfam1", "q1", "%{body}")] : Attempted to write an invalid sink/source: Invalid
sink: hbase( "testtable", "%{nanos}", "colfam1", "q1", "%{body}" )
java.lang.IllegalArgumentException: Attempted to write an invalid sink/source: Invalid sink:
hbase( "testtable", "%{nanos}", "colfam1", "q1", "%{body}" )
        at com.cloudera.flume.master.ConfigManager.setConfig(ConfigManager.java:96)
        at com.cloudera.flume.master.flows.FlowConfigManager.setConfig(FlowConfigManager.java:285)
        at com.cloudera.flume.master.TranslatingConfigurationManager.setConfig(TranslatingConfigurationManager.java:101)
        at
 com.cloudera.flume.master.ConfigCommand$1.exec(ConfigCommand.java:87)
        at com.cloudera.flume.master.CommandManager.exec(CommandManager.java:266)
        at com.cloudera.flume.master.CommandManager.handleCommand(CommandManager.java:205)
        at com.cloudera.flume.master.CommandManager$ExecThread.run(CommandManager.java:236)
Caused by: com.cloudera.flume.conf.FlumeIdException: Invalid sink: hbase( "testtable", "%{nanos}",
"colfam1", "q1", "%{body}" )
        at com.cloudera.flume.conf.FlumeBuilder.buildEventSink(FlumeBuilder.java:503)
        at com.cloudera.flume.conf.FlumeBuilder.buildEventSink(FlumeBuilder.java:531)
        at
 com.cloudera.flume.conf.FlumeBuilder.buildSink(FlumeBuilder.java:302)
        at com.cloudera.flume.master.ConfigManager.setConfig(ConfigManager.java:93)
Mime
View raw message