bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject Translate this to english :)
Date Wed, 28 Aug 2013 21:49:25 GMT
Hi flume  (cc'd bigtop also)....

I've been playing with the bigtop tests, and this snippet is particularly
difficult to
understand for a flume novice :

  private void compressionCommonTest(String id, String decompress, String
glob) {
    String node_config =
"node:text(\"events.txt\")|collectorSink(\"${hdfs_sink_dir}\",\"data\");";

    sh.exec("export FLUME_CONF_DIR=./${id}",
            "flume node_nowatch -s -1 -n node -c '${node_config}'");
    assertEquals("Flume failed to accept events",
                 0, sh.ret);

    sh.exec("hadoop fs -cat ${hdfs_sink_dir}/${glob} | ${decompress} | wc
-l");
    assertEquals("Wrong # of lines in output found at ${hdfs_sink_dir}",
                 "10000", sh.out[0]);
  }

Can someone explain the logic of it to me? In particular...

1) What is the difference between flume and flume ng - and does "flume_ng"
support these options (i.e. node_nowatch) ?

2) What is the "node_config |" syntax accomplishing?

The entire Flume bigtop smoke test can be found here:

https://github.com/apache/bigtop/tree/master/bigtop-tests/test-artifacts/flume/src/main/groovy/org/apache/bigtop/itest/flumesmoke

-- 
Jay Vyas
http://jayunit100.blogspot.com

Mime
View raw message