beam-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Myrvold <amyrv...@google.com>
Subject Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1
Date Mon, 11 Jun 2018 19:37:06 GMT
+1 (non-binding)

tested some of the quickstarts

On Sun, Jun 10, 2018 at 1:39 AM Tim <timrobertson100@gmail.com> wrote:

> Tested by our team:
> - mvn inclusion
> - Avro, ES, Hadoop IF IO
> - Pipelines run on Spark (Cloudera 5.12.0 YARN cluster)
> - Reviewed release notes
>
> +1
>
> Thanks also to everyone who helped get over the gradle hurdle and in
> particular to JB.
>
> Tim
>
> > On 9 Jun 2018, at 05:56, Jean-Baptiste Onofré <jb@nanthrax.net> wrote:
> >
> > No problem Pablo.
> >
> > The vote period is a minimum, it can be extended as requested or if we
> > don't have the minimum of 3 binding votes.
> >
> > Regards
> > JB
> >
> >> On 09/06/2018 01:54, Pablo Estrada wrote:
> >> Hello all,
> >> I'd like to request an extension of the voting period until Monday
> >> evening (US time, so later in other geographical regions). This is
> >> because we were only now able to publish Dataflow Workers, and have not
> >> had the chance to run release validation tests on them. The extension
> >> will allow us to validate and vote by Monday.
> >> Is this acceptable to the community?
> >>
> >> Best
> >> -P.
> >>
> >> On Fri, Jun 8, 2018 at 6:20 AM Alexey Romanenko
> >> <aromanenko.dev@gmail.com <mailto:aromanenko.dev@gmail.com>> wrote:
> >>
> >>    Thank you JB for your work!
> >>
> >>    I tested running simple streaming (/KafkaIO/) and batch (/TextIO /
> >>    HDFS/) pipelines with SparkRunner on YARN cluster - it works fine.
> >>
> >>    WBR,
> >>    Alexey
> >>
> >>
> >>>    On 8 Jun 2018, at 10:00, Etienne Chauchot <echauchot@apache.org
> >>>    <mailto:echauchot@apache.org>> wrote:
> >>>
> >>>    I forgot to vote:
> >>>    +1 (non binding).
> >>>    What I tested:
> >>>    - no functional or performance regression comparing to v2.4
> >>>    - dependencies in the poms are ok
> >>>
> >>>    Etienne
> >>>>    Le vendredi 08 juin 2018 à 08:27 +0200, Romain Manni-Bucau a écrit
> :
> >>>>    +1 (non-binding), mainstream usage is not broken by the pom
> >>>>    changes and runtime has no known regression compared to the 2.4.0
> >>>>
> >>>>    (side note: kudo to JB for this build tool change release, I know
> >>>>    how it can hurt ;))
> >>>>
> >>>>    Romain Manni-Bucau
> >>>>    @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> >>>>    <https://rmannibucau.metawerx.net/> | Old Blog
> >>>>    <http://rmannibucau.wordpress.com/> | Github
> >>>>    <https://github.com/rmannibucau> | LinkedIn
> >>>>    <https://www.linkedin.com/in/rmannibucau> | Book
> >>>>    <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >>>>
> >>>>
> >>>>    Le jeu. 7 juin 2018 à 16:17, Jean-Baptiste Onofré
> >>>>    <jb@nanthrax.net <mailto:jb@nanthrax.net>> a écrit :
> >>>>>    Thanks for the details Etienne !
> >>>>>
> >>>>>    The good news is that the artifacts seem OK and the overall
> Nexmark
> >>>>>    results are consistent with the 2.4.0 release ones.
> >>>>>
> >>>>>    I'm starting a complete review using the beam-samples as well.
> >>>>>
> >>>>>    Regards
> >>>>>    JB
> >>>>>
> >>>>>>    On 07/06/2018 16:14, Etienne Chauchot wrote:
> >>>>>> Hi,
> >>>>>> I've just run the nexmark queries on v2.5.0-RC1 tag
> >>>>>> What we can notice:
> >>>>>> - query 3 (exercises CoGroupByKey, state and timer) shows
> >>>>>    different
> >>>>>> output with DR between batch and streaming and with the other
> >>>>>    runners =>
> >>>>>> I compared with v2.4 there were still these differences but
with
> >>>>>> different output size numbers
> >>>>>>
> >>>>>> - query 6 (exercises specialized combiner) shows different output
> >>>>>> between the runners => the correct output is 401. strange
that
> >>>>>    in batch
> >>>>>> mode some runners output les Sellers. I compared with v2.4
> >>>>>    same output
> >>>>>>
> >>>>>> - response time of query 7 (exercices Max transform, fanout
> >>>>>    and side
> >>>>>> input) is very slow on DR => I compared with v2.4 , comparable
> >>>>>    execution
> >>>>>> times
> >>>>>>
> >>>>>> I'm not comparing q10 because it is a write to GCS so it is
> >>>>>    very specific.
> >>>>>>
> >>>>>> => Basically no regression comparing to v2.4
> >>>>>>
> >>>>>> For the record here is the output (waiting for ongoing perfkit
> >>>>>    integration):
> >>>>>>
> >>>>>>
> >>>>>> 1. DR batch
> >>>>>>
> >>>>>> Performance:
> >>>>>>
> >>>>>>
> >>>>>    Conf  Runtime(sec)    (Baseline)  Events(/sec)    (Baseline)
>  Results    (Baseline)
> >>>>>>
> >>>>>>
> >>>>>    0000           5,8                     17283,1
>   100000
> >>>>>>
> >>>>>>
> >>>>>    0001           3,2                     31104,2
>    92000
> >>>>>>
> >>>>>>
> >>>>>    0002           1,2                     82918,7
>      351
> >>>>>>
> >>>>>>
> >>>>>    0003           2,2                     46210,7
>      458
> >>>>>>
> >>>>>>
> >>>>>    0004           1,2                      8503,4
>       40
> >>>>>>
> >>>>>>
> >>>>>    0005           4,0                     25220,7
>       12
> >>>>>>
> >>>>>>
> >>>>>    0006           0,9                     11148,3
>      401
> >>>>>>
> >>>>>>
> >>>>>    0007          13,2                      7580,9
>        1
> >>>>>>
> >>>>>>
> >>>>>    0008           1,5                     67340,1
>     6000
> >>>>>>
> >>>>>>
> >>>>>    0009           0,7                     14025,2
>      298
> >>>>>>
> >>>>>>
> >>>>>    0010          12,8                      7793,0
>        1
> >>>>>>
> >>>>>>
> >>>>>    0011           2,4                     42319,1
>     1919
> >>>>>>
> >>>>>>
> >>>>>    0012           1,6                     61462,8
>     1919
> >>>>>>
> >>>>>
> ==========================================================================================
> >>>>>>
> >>>>>> 2. DR streaming
> >>>>>>
> >>>>>> Performance:
> >>>>>>
> >>>>>>
> >>>>>    Conf  Runtime(sec)    (Baseline)  Events(/sec)    (Baseline)
>  Results    (Baseline)
> >>>>>>
> >>>>>>
> >>>>>    0000           6,5                     15285,8
>   100000
> >>>>>>
> >>>>>>
> >>>>>    0001           3,7                     27397,3
>    92000
> >>>>>>
> >>>>>>
> >>>>>    0002           1,4                     69108,5
>      351
> >>>>>>
> >>>>>>
> >>>>>    0003           3,2                     31181,8
>      447
> >>>>>>
> >>>>>>
> >>>>>    0004           1,2                      8361,2
>       40
> >>>>>>
> >>>>>>
> >>>>>    0005           5,3                     18903,6
>       12
> >>>>>>
> >>>>>>
> >>>>>    0006           0,9                     11111,1
>      401
> >>>>>>
> >>>>>>
> >>>>>    0007          82,5                      1212,2
>        1
> >>>>>>
> >>>>>>
> >>>>>    0008           2,0                     51072,5
>     6000
> >>>>>>
> >>>>>>
> >>>>>    0009           0,8                     12903,2
>      298
> >>>>>>
> >>>>>>
> >>>>>    0010          49,5                      2021,8
>        1
> >>>>>>
> >>>>>>
> >>>>>    0011           3,9                     25667,4
>     1919
> >>>>>>
> >>>>>>
> >>>>>    0012           2,4                     41067,8
>     1919
> >>>>>>
> >>>>>
> ==========================================================================================
> >>>>>>
> >>>>>> 3. Flink batch
> >>>>>> Performance:
> >>>>>>
> >>>>>>
> >>>>>    Conf  Runtime(sec)    (Baseline)  Events(/sec)    (Baseline)
>  Results    (Baseline)
> >>>>>>
> >>>>>>
> >>>>>    0000           1,0                     97656,3
>   100000
> >>>>>>
> >>>>>>
> >>>>>    0001           0,7                    141643,1
>    92000
> >>>>>>
> >>>>>>
> >>>>>    0002           0,4                    228310,5
>      351
> >>>>>>
> >>>>>>
> >>>>>    0003           1,6                     64020,5
>      580
> >>>>>>
> >>>>>>
> >>>>>    0004           0,7                     13831,3
>       40
> >>>>>>
> >>>>>>
> >>>>>    0005           1,4                     72939,5
>       12
> >>>>>>
> >>>>>>
> >>>>>    0006           0,5                     20491,8
>      103
> >>>>>>
> >>>>>>
> >>>>>    0007           1,3                     74239,0
>        1
> >>>>>>
> >>>>>>
> >>>>>    0008           0,8                    121506,7
>     6000
> >>>>>>
> >>>>>>
> >>>>>    0009           0,6                     17953,3
>      298
> >>>>>>
> >>>>>>
> >>>>>    0010           1,3                     74682,6
>        1
> >>>>>>
> >>>>>>
> >>>>>    0011           1,1                     92936,8
>     1919
> >>>>>>
> >>>>>>
> >>>>>    0012           0,8                    123001,2
>     1919
> >>>>>>
> >>>>>
> ==========================================================================================
> >>>>>>
> >>>>>> 4. Flink streaming
> >>>>>> Performance:
> >>>>>>
> >>>>>>
> >>>>>    Conf  Runtime(sec)    (Baseline)  Events(/sec)    (Baseline)
>  Results    (Baseline)
> >>>>>>
> >>>>>>
> >>>>>    0000           5,4                     18677,6
>   100000
> >>>>>>
> >>>>>>
> >>>>>    0001           2,8                     35511,4
>    92000
> >>>>>>
> >>>>>>
> >>>>>    0002           1,8                     54318,3
>      351
> >>>>>>
> >>>>>>
> >>>>>    0003           2,4                     41614,6
>      580
> >>>>>>
> >>>>>>
> >>>>>    0004           1,0                     10341,3
>       40
> >>>>>>
> >>>>>>
> >>>>>    0005           3,4                     29568,3
>       12
> >>>>>>
> >>>>>>
> >>>>>    0006           0,7                     13369,0
>      401
> >>>>>>
> >>>>>>
> >>>>>    0007           2,8                     36192,5
>        1
> >>>>>>
> >>>>>>
> >>>>>    0008           1,8                     54854,6
>     6000
> >>>>>>
> >>>>>>
> >>>>>    0009           0,7                     13369,0
>      298
> >>>>>>
> >>>>>>
> >>>>>    0010           3,4                     29841,8
>        2
> >>>>>>
> >>>>>>
> >>>>>    0011           5,0                     19932,2
>     1919
> >>>>>>
> >>>>>>
> >>>>>    0012           2,6                     38835,0
>     1919
> >>>>>>
> >>>>>
> ==========================================================================================
> >>>>>>
> >>>>>> 5. Spark batch
> >>>>>> Performance:
> >>>>>>
> >>>>>>
> >>>>>    Conf  Runtime(sec)    (Baseline)  Events(/sec)    (Baseline)
>  Results    (Baseline)
> >>>>>>
> >>>>>>
> >>>>>    0000           1,5                     65445,0
>   100000
> >>>>>>
> >>>>>>
> >>>>>    0001           1,3                     79491,3
>    92000
> >>>>>>
> >>>>>>
> >>>>>    0002           0,9                    112107,6
>      351
> >>>>>>
> >>>>>>
> >>>>>    0003           2,0                     48804,3
>      580
> >>>>>>
> >>>>>>
> >>>>>    0004           1,2                      8382,2
>       40
> >>>>>>
> >>>>>>
> >>>>>    0005           2,0                     50838,8
>       12
> >>>>>>
> >>>>>>
> >>>>>    0006           1,0                      9699,3
>      103
> >>>>>>
> >>>>>>
> >>>>>    0007           2,3                     43308,8
>        1
> >>>>>>
> >>>>>>
> >>>>>    0008           2,1                     46794,6
>     6000
> >>>>>>
> >>>>>>
> >>>>>    0009           1,1                      8976,7
>      298
> >>>>>>
> >>>>>>
> >>>>>    0010           1,6                     62111,8
>        1
> >>>>>>
> >>>>>>
> >>>>>    0011           2,1                     46598,3
>     1919
> >>>>>>
> >>>>>>
> >>>>>    0012           2,3                     43687,2
>     1919
> >>>>>>
> >>>>>
> ==========================================================================================
> >>>>>>
> >>>>>>> Le mercredi 06 juin 2018 à 10:50 +0200, Etienne Chauchot
a écrit :
> >>>>>>> Thanks JB for all your work ! I believe doing the first
> >>>>>    gradle release
> >>>>>>> must have been hard.
> >>>>>>> I'll run Nexmark on the release and keep you posted.
> >>>>>>>
> >>>>>>> Best
> >>>>>>> Etienne
> >>>>>>>
> >>>>>>>
> >>>>>>> Le mercredi 06 juin 2018 à 10:44 +0200, Jean-Baptiste Onofré
> >>>>>    a écrit :
> >>>>>>>> Hi everyone,
> >>>>>>>>
> >>>>>>>> Please review and vote on the release candidate #1 for
the
> >>>>>    version
> >>>>>>>> 2.5.0, as follows:
> >>>>>>>>
> >>>>>>>> [ ] +1, Approve the release
> >>>>>>>> [ ] -1, Do not approve the release (please provide specific
> >>>>>    comments)
> >>>>>>>>
> >>>>>>>> NB: this is the first release using Gradle, so don't
be too
> >>>>>    harsh ;) A
> >>>>>>>> PR about the release guide will follow thanks to this
release.
> >>>>>>>>
> >>>>>>>> The complete staging area is available for your review,
> >>>>>    which includes:
> >>>>>>>> * JIRA release notes [1],
> >>>>>>>> * the official Apache source release to be deployed
to
> >>>>>    dist.apache.org <http://dist.apache.org/>
> >>>>>>>> [2], which is signed with the key with fingerprint C8282E76
[3],
> >>>>>>>> * all artifacts to be deployed to the Maven Central
> >>>>>    Repository [4],
> >>>>>>>> * source code tag "v2.5.0-RC1" [5],
> >>>>>>>> * website pull request listing the release and publishing
> >>>>>    the API
> >>>>>>>> reference manual [6].
> >>>>>>>> * Java artifacts were built with Gradle 4.7 (wrapper)
and
> >>>>>    OpenJDK/Oracle
> >>>>>>>> JDK 1.8.0_172 (Oracle Corporation 25.172-b11).
> >>>>>>>> * Python artifacts are deployed along with the source
> >>>>>    release to the
> >>>>>>>> dist.apache.org <http://dist.apache.org/> [2].
> >>>>>>>>
> >>>>>>>> The vote will be open for at least 72 hours. It is adopted
> >>>>>    by majority
> >>>>>>>> approval, with at least 3 PMC affirmative votes.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> JB
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12342847
> >>>>>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.5.0/
> >>>>>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> >>>>>>>> [4]
> >>>>>
> https://repository.apache.org/content/repositories/orgapachebeam-1041/
> >>>>>>>> [5] https://github.com/apache/beam/tree/v2.5.0-RC1
> >>>>>>>> [6] https://github.com/apache/beam-site/pull/463
> >>>>>>>>
> >>>>>
> >>
> >> --
> >> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
>

Mime
View raw message