flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Behrouz Derakhshan <behrouz.derakhs...@gmail.com>
Subject Re: Advice on [FLINK-2021]: Rework examples to use new ParameterTool
Date Fri, 04 Sep 2015 12:46:20 GMT
Yes, I was referring mostly to blog posts and other websites and was
wondering if breaking them is an issue or not.
I have already created a subtask to add support for positional arguments (
FLINK-2621 <https://issues.apache.org/jira/browse/FLINK-2621>), so the
examples would be backward compatible.
The problem with that is, we have to detect from the arguments to the
program, if they are positional or key/value and parse them accordingly.
But if everyone is OK with completely switching to ParameterTool and
breaking the support for the old way of executing the examples, then my job
would be also a lot easier.



On Fri, Sep 4, 2015 at 2:34 PM, Robert Metzger <rmetzger@apache.org> wrote:

> If you are referring to this training material (
>
> https://github.com/dataArtisans/flink-training-exercises/blob/master/src/main/java/com/dataArtisans/flinkTraining/exercises/dataStreamJava/rideCleansing/RideCleansing.java
> ),
> some of the examples are actually already using the ParameterTool.
>
> The problem are probably websites / blogposts etc. that show how to use the
> Flink examples. But I think its fine to break these. All example jars
> contain the version number. If the way we pass arguments to the examples
> changes between 0.9 and 0.10, that should be fine.
>
> I think using the ParameterTool for the examples will improve the
> readability of the examples a lot. Right now, all examples have a
> (copy-pasted) parseParameters() method, which is doing very simplistic
> parameter parsing.
>
> The PT tool also allows to show the input parameters in the web interface.
>
> So I'm voting for doing a breaking change and using parameters such as
> "--input hdfs:/// --output hdfs:/// --iterations 15".
>
> On Fri, Sep 4, 2015 at 1:05 PM, Behrouz Derakhshan <
> behrouz.derakhshan@gmail.com> wrote:
>
> > Will do.
> >
> > Thanks,
> > Behrouz
> >
> > On Fri, Sep 4, 2015 at 11:29 AM, Maximilian Michels <mxm@apache.org>
> > wrote:
> >
> > > Hi Behrouz,
> > >
> > > I would create a new sub-task under the original issue that introduce
> > > the ParameterTool: https://issues.apache.org/jira/browse/FLINK-1525
> > >
> > > Cheers,
> > > Max
> > >
> > > On Fri, Sep 4, 2015 at 11:17 AM, Behrouz Derakhshan
> > > <behrouz.derakhshan@gmail.com> wrote:
> > > > Hi Max,
> > > >
> > > > What you said makes sense, for "ParameterTool doesn't seem to support
> > > > positional arguments :) but we could fix that." should we create a
> > > separate
> > > > ticket or should it also be part of FLINK-2021 ?
> > > >
> > > > BR,
> > > > Behrouz
> > > >
> > > >
> > > > On Fri, Sep 4, 2015 at 10:55 AM, Maximilian Michels <mxm@apache.org>
> > > wrote:
> > > >
> > > >> Hi Behrouz,
> > > >>
> > > >> Thanks for starting the discussion. If I understand your question
> > > >> correctly, you are asking if it breaks the training or other
> external
> > > >> material if we convert the Flink examples to make use of the
> > > >> ParameterTool?
> > > >>
> > > >> We could make the changes such that the examples will accept the
> same
> > > >> parameters but use the ParameterTool internally to verify the
> > > >> parameters and print usage information. I think most examples simply
> > > >> use positional arguments and we could keep it that way. The only
> > > >> problem is that the ParameterTool doesn't seem to support positional
> > > >> arguments :) but we could fix that.
> > > >>
> > > >> Cheers,
> > > >> Max
> > > >>
> > > >> On Thu, Sep 3, 2015 at 5:50 PM, Behrouz Derakhshan
> > > >> <behrouz.derakhshan@gmail.com> wrote:
> > > >> > Hi,
> > > >> >
> > > >> > I had at look at this ticket FLINK-2021
> > > >> > <https://issues.apache.org/jira/browse/FLINK-2021>, there
isn't
> > much
> > > to
> > > >> do
> > > >> > from a technical stand point and it kinda makes sense to use
the
> new
> > > >> > "ParameterTool", since it is being used in most of the other
part
> of
> > > the
> > > >> > code base.
> > > >> > The only question is do we really want to do it, since I'm
> guessing
> > > some
> > > >> of
> > > >> > the training materials, slides and articles are referencing these
> > > >> examples
> > > >> > and updating those might be a burden.
> > > >> >
> > > >> > Let me know what you guys think, either I can start working on
it
> or
> > > we
> > > >> can
> > > >> > just resolve it for good.
> > > >> >
> > > >> > Cheers,
> > > >> > Behrouz
> > > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message