Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9FEEF19FB0 for ; Wed, 27 Apr 2016 16:50:04 +0000 (UTC) Received: (qmail 32263 invoked by uid 500); 27 Apr 2016 16:50:04 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 32166 invoked by uid 500); 27 Apr 2016 16:50:04 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 32155 invoked by uid 99); 27 Apr 2016 16:50:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Apr 2016 16:50:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id E5AD81A12AF for ; Wed, 27 Apr 2016 16:50:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.651 X-Spam-Level: ** X-Spam-Status: No, score=2.651 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_NEUTRAL=0.652] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id tUeLHvm0vI2I for ; Wed, 27 Apr 2016 16:49:59 +0000 (UTC) Received: from gproxy4-pub.mail.unifiedlayer.com (gproxy4-pub.mail.unifiedlayer.com [69.89.23.142]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 0BF2D5F298 for ; Wed, 27 Apr 2016 16:49:57 +0000 (UTC) Received: (qmail 16375 invoked by uid 0); 27 Apr 2016 16:49:49 -0000 Received: from unknown (HELO CMOut01) (10.0.90.82) by gproxy4.mail.unifiedlayer.com with SMTP; 27 Apr 2016 16:49:49 -0000 Received: from box220.bluehost.com ([69.89.27.220]) by CMOut01 with id nUpc1s00R4kwCAl01UpfxN; Wed, 27 Apr 2016 10:49:47 -0600 X-Authority-Analysis: v=2.1 cv=G/WPTbU5 c=1 sm=1 tr=0 a=JhnL/J+lANDRi3NK4GIIcg==:117 a=JhnL/J+lANDRi3NK4GIIcg==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=tfPvd5Tc7C0A:10 a=kxeWoCsFtUYA:10 a=kziv93cY1bsA:10 a=mV9VRH-2AAAA:8 a=pGLkceISAAAA:8 a=EPOiSyPHAAAA:8 a=Y3OxiLBZAAAA:8 a=fdnwabApUm-Iy94VLFcA:9 a=QEXdDO2ut3YA:10 a=0CTZCOJDxewA:10 a=QWGAg2j_AkqwC5LiW1kA:9 a=J8UEmvP__b3UFhWr:21 a=_W_S_7VecoQA:10 Received: from [24.7.149.223] (port=52820 helo=kens-mbp.hsd1.ca.comcast.net) by box220.bluehost.com with esmtpsa (TLSv1:ECDHE-RSA-AES256-SHA:256) (Exim 4.86_2) (envelope-from ) id 1avSef-0000bl-Cy for user@flink.apache.org; Wed, 27 Apr 2016 10:49:37 -0600 From: Ken Krugler Content-Type: multipart/alternative; boundary="Apple-Mail=_5D220316-C52A-46CF-AA1A-017B94FEE6E4" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: Command line arguments getting munged with CLI? Date: Wed, 27 Apr 2016 09:49:36 -0700 References: <0AF0E916-564A-4113-BCC3-00D558FACC37@transpac.com> To: user@flink.apache.org In-Reply-To: X-Mailer: Apple Mail (2.3112) X-Identified-User: {1610:box220.bluehost.com:bixolabs:scaleunlimited.com} {sentby:smtp auth 24.7.149.223 authed with kkrugler@scaleunlimited.com} --Apple-Mail=_5D220316-C52A-46CF-AA1A-017B94FEE6E4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Timur, Thanks, using =E2=80=98--=E2=80=99 seems to work. I=E2=80=99ve filed https://issues.apache.org/jira/browse/FLINK-3838 = to allow = =E2=80=98-=E2=80=99 as well. =E2=80=94 Ken > On Apr 27, 2016, at 12:20am, Timur Fayruzov = wrote: >=20 > Hi Ken, >=20 > I have built parameter parser in my jar to work with '--' instead of = '-' and it works fine (on 1.0.0 and on current master). After a cursory = look at parameter parser Flink uses = (http://commons.apache.org/proper/commons-cli/ = ) it seems that double vs = single dash could make a difference, so you could try it. >=20 >=20 > On Tue, Apr 26, 2016 at 11:45 AM, Ken Krugler = > = wrote: > Hi all, >=20 > I=E2=80=99m running this command, on the master in an EMR cluster: >=20 > ./bin/flink run -m yarn-cluster -yn 25 -yjm 1024 -ytm 4096 -c -planner flink -inputdir xxx >=20 > Everything seems to be starting up fine, up to: >=20 > All TaskManagers are connected > Using the parallelism provided by the remote cluster (25). To use = another parallelism, set it at the ./bin/flink client. >=20 > My main class then tries to parse the passed arguments, and fails. >=20 > The arguments being passed to my main() class aren=E2=80=99t what I = expect. I get... >=20 > -p > lanner > flink > inputdir > s3n://su-wikidump/wikidump-20151112/data/ <> >=20 > It looks like Flink is trying to process all of my arguments, so = -planner looks like =E2=80=9C-p=E2=80=9D, and -inputdir gets the =E2=80=98= -=E2=80=98 consumed before it realizes that there is no = =E2=80=98-i=E2=80=99 parameter that it knows about. >=20 > I was assuming that everything after the jar parameter would be sent = through as is. >=20 > Any ideas? >=20 > Thanks, >=20 > =E2=80=94 Ken >=20 > PS - this is with flink 1.0.2 -------------------------- Ken Krugler +1 530-210-6378 http://www.scaleunlimited.com custom big data solutions & training Hadoop, Cascading, Cassandra & Solr --Apple-Mail=_5D220316-C52A-46CF-AA1A-017B94FEE6E4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi Timur,

Thanks, using =E2=80=98--<parameter name>=E2=80=99 = seems to work.

I=E2=80=99ve filed https://issues.apache.org/jira/browse/FLINK-3838 to = allow =E2=80=98-<parameter name>=E2=80=99 as well.

=E2=80=94 Ken

On Apr 27, 2016, at 12:20am, Timur Fayruzov <timur.fairuzov@gmail.com> wrote:

Hi Ken,

I = have built parameter parser in my jar to work with '--' instead of '-' = and it works fine (on 1.0.0 and on current master). After a cursory look = at parameter parser Flink uses (http://commons.apache.org/proper/commons-cli/) it seems = that double vs single dash could make a difference, so you could try = it.


On Tue, = Apr 26, 2016 at 11:45 AM, Ken Krugler <kkrugler_lists@transpac.com> wrote:
Hi all,

I=E2=80=99m running this command, on = the master in an EMR cluster:

./bin/flink run -m yarn-cluster -yn 25 = -yjm 1024 -ytm 4096 -c <class name> <path to job = jar> -planner flink -inputdir xxx

Everything seems to be starting up = fine, up to:

All TaskManagers are connected
Using the parallelism provided by the remote cluster (25). To = use another parallelism, set it at the ./bin/flink = client.

My= main class then tries to parse the passed arguments, and = fails.

The = arguments being passed to my main() class aren=E2=80=99t what I expect. = I get...

= -p
= lanner
= flink
= inputdir
= s3n://su-wikidump/wikidump-20151112/data/

It looks like Flink is = trying to process all of my arguments, so -planner looks like =E2=80=9C-p=E2= =80=9D, and -inputdir gets the =E2=80=98-=E2=80=98 consumed before it = realizes that there is no =E2=80=98-i<anything>=E2=80=99 parameter = that it knows about.

I was assuming that everything after the jar parameter would = be sent through as is.

Any ideas?

Thanks,

=E2=80=94 Ken

PS - this is with flink = 1.0.2

--------------------------
Ken = Krugler
+1 530-210-6378
http://www.scaleunlimited.com
custom = big data solutions & training
Hadoop, = Cascading, Cassandra & = Solr
<= /span>



= --Apple-Mail=_5D220316-C52A-46CF-AA1A-017B94FEE6E4--