chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thushara Wijeratna <>
Subject Using SocketTeeWriter without SeqFileWriter
Date Sun, 01 Nov 2009 00:11:59 GMT
I wanted to use SocketTeeWriter without going through the steps of
writing to HDFS.
It seems that PipelineStageWriter is designed to have any number of
PipeLineable writers, optionally followed by a SeqFileWriter.
So I changed my collector config :



After doing one minor change to SocketTeeWriter, I could get this to
work. The advantage is that now I do not need to set up HDFS.

Please let me know if this is something we should patch, I will submit
the patch.

SocketTeeWriter changes:

[~/hadoop-src/chukwa/trunk] svn diff
Index: src/java/org/apache/hadoop/chukwa/datacollection/writer/
--- src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.jav(revision
+++ src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.jav(working
@@ -225,7 +225,9 @@

   public CommitStatus add(List<Chunk> chunks) throws WriterException {
-    CommitStatus rv = next.add(chunks); //pass data through
+    CommitStatus rv = ChukwaWriter.COMMIT_OK;
+    if (next != null)
+	rv = next.add(chunks); //pass data through
     synchronized(tees) {
       Iterator<Tee> loop = tees.iterator();
       while(loop.hasNext()) {

View raw message