Mailing-List: contact user-help@bigtop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@bigtop.apache.org
Received-SPF: pass (athena.apache.org: domain of jayunit100@gmail.com
 designates 209.85.217.175 as permitted sender)
MIME-Version: 1.0
Date: Wed, 28 Aug 2013 17:49:25 -0400
Message-ID: 
 <CAAu13zEV9J95GZf8kiCei6aqRMcDLdOnzF5-2uhSQPj0BgBCVw@mail.gmail.com>
Subject: Translate this to english :)
From: Jay Vyas <jayunit100@gmail.com>
To: user@flume.apache.org, user@bigtop.apache.org
Content-Type: multipart/alternative; boundary=089e011766e585efde04e508f664

--089e011766e585efde04e508f664
Content-Type: text/plain; charset=ISO-8859-1

Hi flume  (cc'd bigtop also)....

I've been playing with the bigtop tests, and this snippet is particularly
difficult to
understand for a flume novice :

  private void compressionCommonTest(String id, String decompress, String
glob) {
    String node_config =
"node:text(\"events.txt\")|collectorSink(\"${hdfs_sink_dir}\",\"data\");";

    sh.exec("export FLUME_CONF_DIR=./${id}",
            "flume node_nowatch -s -1 -n node -c '${node_config}'");
    assertEquals("Flume failed to accept events",
                 0, sh.ret);

    sh.exec("hadoop fs -cat ${hdfs_sink_dir}/${glob} | ${decompress} | wc
-l");
    assertEquals("Wrong # of lines in output found at ${hdfs_sink_dir}",
                 "10000", sh.out[0]);
  }

Can someone explain the logic of it to me? In particular...

1) What is the difference between flume and flume ng - and does "flume_ng"
support these options (i.e. node_nowatch) ?

2) What is the "node_config |" syntax accomplishing?

The entire Flume bigtop smoke test can be found here:

https://github.com/apache/bigtop/tree/master/bigtop-tests/test-artifacts/flume/src/main/groovy/org/apache/bigtop/itest/flumesmoke

-- 
Jay Vyas
http://jayunit100.blogspot.com

--089e011766e585efde04e508f664
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi flume =A0(cc&#39;d bigtop also)....<div><br></div><div>=
I&#39;ve been playing with the bigtop tests, and this snippet is particular=
ly difficult to=A0</div><div>understand for a flume novice :=A0</div><div><=
br></div>
<div><div>=A0 private void compressionCommonTest(String id, String decompre=
ss, String glob) {</div><div>=A0 =A0 String node_config =3D &quot;node:text=
(\&quot;events.txt\&quot;)|collectorSink(\&quot;${hdfs_sink_dir}\&quot;,\&q=
uot;data\&quot;);&quot;;</div>
<div><br></div><div>=A0 =A0 sh.exec(&quot;export FLUME_CONF_DIR=3D./${id}&q=
uot;,</div><div>=A0 =A0 =A0 =A0 =A0 =A0 &quot;flume node_nowatch -s -1 -n n=
ode -c &#39;${node_config}&#39;&quot;);</div><div>=A0 =A0 assertEquals(&quo=
t;Flume failed to accept events&quot;,</div>
<div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A00, sh.ret);</div><div><br></div><di=
v>=A0 =A0 sh.exec(&quot;hadoop fs -cat ${hdfs_sink_dir}/${glob} | ${decompr=
ess} | wc -l&quot;);</div><div>=A0 =A0 assertEquals(&quot;Wrong # of lines =
in output found at ${hdfs_sink_dir}&quot;,</div>
<div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&quot;10000&quot;, sh.out[0]);</div=
><div>=A0 }</div><div><br></div><div style>Can someone explain the logic of=
 it to me? In particular...</div><div style><br></div><div style>1) What is=
 the difference between flume and flume ng - and does &quot;flume_ng&quot; =
support these options (i.e. node_nowatch) ?</div>
<div style><br></div><div style>2) What is the &quot;node_config |&quot; sy=
ntax accomplishing?=A0</div><div style><br></div><div style>The entire Flum=
e bigtop smoke test can be found here:=A0</div><div style><br></div><div st=
yle>
<a href=3D"https://github.com/apache/bigtop/tree/master/bigtop-tests/test-a=
rtifacts/flume/src/main/groovy/org/apache/bigtop/itest/flumesmoke">https://=
github.com/apache/bigtop/tree/master/bigtop-tests/test-artifacts/flume/src/=
main/groovy/org/apache/bigtop/itest/flumesmoke</a><br>
</div><div style><br></div>-- <br>Jay Vyas<br><a href=3D"http://jayunit100.=
blogspot.com" target=3D"_blank">http://jayunit100.blogspot.com</a>
</div></div>

--089e011766e585efde04e508f664--