Mailing-List: contact dev-help@pig.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@pig.apache.org
Received-SPF: pass (nike.apache.org: domain of liyun.zhang@intel.com
 designates 192.55.52.93 as permitted sender)
From: "Zhang, Liyun" <liyun.zhang@intel.com>
To: "pig-dev@hadoop.apache.org" <pig-dev@hadoop.apache.org>
Subject: some question about streaming_local.conf
Thread-Topic: some question about streaming_local.conf
Thread-Index: AdAg6JSXM0kdnL6OTV2X95Z2LFNEAw==
Date: Fri, 26 Dec 2014 08:47:37 +0000
Message-ID: 
 <AF9B21681D3EE94AA935901747BE4B8101DB0D55@shsmsx102.ccr.corp.intel.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: multipart/alternative;
	boundary="_000_AF9B21681D3EE94AA935901747BE4B8101DB0D55shsmsx102ccrcor_"
MIME-Version: 1.0

--_000_AF9B21681D3EE94AA935901747BE4B8101DB0D55shsmsx102ccrcor_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi all:
  I have 2 questiones about pig/test/e2e/pig/tests/streaming_local.conf:
1.
$cfg =3D {
        'driver' =3D> 'Pig',
        'nummachines' =3D> 5,

        'groups' =3D> [
                                {
                                # This group is for local mode testing
                                'name' =3D> 'StreamingLocal',
                'sortBenchmark' =3D> 1,
                'sortResults' =3D> 1,
                'floatpostprocess' =3D> 1,
                'delimiter' =3D> '   ',
                'tests' =3D> [
                                                {
                                        #Section 1.1: perl script, no param=
eters
                        'num' =3D> 1,
                        'execonly' =3D> 'local',  // this line
                        'pig' =3D> q#

all e2e test cases are only executed in local mode now. Can these e2e tests=
 run in other mode, like "mapreduce","tez","spark"?
when i replace 'execonly'=3D>'local' with 'execonly'=3D>'spark', all cases =
pass when POStream is implemented in spark mode.
I think we can remove 'execonly'=3D>'local' and can test these e2e tests in=
 other modes.

2. when using ship with streaming, it will send streaming binary and suppor=
ting files, if any, from the client node to the compute nodes..I found we u=
se "perl ./libexec/GroupBy.pl" in StreamingLocal_3.pig, this path is a rela=
tive path to current executed path. can we use "perl GroupBy.pl" because i =
think the file "./libexec/GroupBy.pl" has been shipped to compute nodes.
StreamingLocal_3.pig
/test/e2e/pigMD `perl ./libexec/GroupBy.pl '\t' 0` ship('./libexec/GroupBy.=
pl');
A =3D load './data/singlefile/studenttab10k';
B =3D group A by $0;
C =3D foreach B generate flatten(A);
D =3D stream C through CMD;
store D into './testout/root-1419582821-streaming_local.conf/StreamingLocal=
_3.out';


Best regards
Zhang,Liyun


--_000_AF9B21681D3EE94AA935901747BE4B8101DB0D55shsmsx102ccrcor_--