beam-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré ...@nanthrax.net>
Subject Re: Apache Beam, version 2.2.0
Date Tue, 05 Dec 2017 06:24:17 GMT
+1, and sorry again, I thought we got an consensus.

Regards
JB

On 12/05/2017 07:10 AM, Kenneth Knowles wrote:
> +1 to the poll and also to Reuven's point.
> 
> Those without a support contract would have been using JDK 7 without security 
> updates for years. IMO it seems harmful, as a netizen, to encourage its 
> use/existence.
> 
> If there's no noise from the prior thread, then I would assume no one on user@ 
> has any objection. Anyone else with customers should reach out to them.
> 
> Kenn
> 
> On Mon, Dec 4, 2017 at 9:49 PM, Reuven Lax <relax@google.com 
> <mailto:relax@google.com>> wrote:
> 
>     Technically it's a backwards-incompatible change, however if we are
>     convinced the risk is low we could do it.
> 
>     As mentioned on the original thread, it's not clear that all Beam users read
>     user@ - e.g. most Dataflow users definitely do not. I think we need to
>     separately reach out to users of each runner through runner-specific channels.
> 
>     Reuven
> 
>     On Mon, Dec 4, 2017 at 9:37 PM, Eugene Kirpichov <kirpichov@google.com
>     <mailto:kirpichov@google.com>> wrote:
> 
>         On the original thread
>         https://lists.apache.org/thread.html/2e1890c62d9f022f09b20e9f12f130fe9f1042e391979087f725d2e0@%3Cuser.beam.apache.org%3E
>         <https://lists.apache.org/thread.html/2e1890c62d9f022f09b20e9f12f130fe9f1042e391979087f725d2e0@%3Cuser.beam.apache.org%3E> ,
>         Robert and Ismaël were in favor of no major version change [Ismaël
>         said:/Also I am afraid that if we wait/
>         /until we have enough changes to switch Beam to a new major version the
>         switch to Java 8 will happen too late, probably after Java 8's end of
>         life. And I am not exaggerating, Java 8 is planned to EOL next march
>         2018!/]; JB and now Reuven are in favor of a major version change;
>         nobody so far argued against switching to Java8 in general.
> 
>         I'm personally in favor of no major version change (i.e. not waiting
>         until all other large changes for Beam 3.0 converge, which will likely
>         be many months), because:
>         - Reasons Ismaël cited; plus the reason that most people are likely
>         already using Java 8.
>         - Going Java8-only earlier will make other Beam 3.0 APIs better for
>         Java8 users, because we (Beam contributors) will have experience working
>         with them within the SDK in Java8 (e.g. writing tests with use of
>         lambdas and noticing whether it's clunky, or whether some other Beam
>         APIs need better Java8 support).
>         - Going Java8 will make it more reasonable to include (mostly or only)
>         Java8 snippets in Beam documentation, which will obviously look more
>         concise and attractive, addressing one of the common concerns of Beam
>         users that it has a heavyweight API compared to functional-style APIs of
>         Spark etc.
> 
>         I think resolving this via a poll of users would be reasonable. I'd
>         suggest e.g. the following phrasing:
> 
>         Apache Beam is considering dropping support for Java 7, and supporting
>         only Java 8 and above in a subsequent release. How would it impact your
>         usage of Beam?
>         - I am already using only Java 8+ for building my Beam code
>         - I am using Java 7 for building my Beam code, but I would have no
>         trouble switching to Java 8
>         - I am using Java 7 for building my Beam code, and dropping Java 7 would
>         be a blocker or hindrance to adopting the new release for me
> 
>         We could tweet this poll on Apache Beam twitter and publish on user@,
>         and, say, if we receive 5% or fewer votes for option 3 after keeping it
>         open for 2 weeks, then adopt Java 8 without a major version change.
> 
>         WDYT?
> 
>         On Mon, Dec 4, 2017 at 8:34 PM Jean-Baptiste Onofré <jb@nanthrax.net
>         <mailto:jb@nanthrax.net>> wrote:
> 
>             Good idea ! Definitely +1
> 
>             Regards
>             JB
> 
>             On 12/05/2017 05:25 AM, Reuven Lax wrote:
>              > We should bring this up on the Beam 3.0 thread. Since it's
>             technically a
>              > backwards-incompatible change, it might make a good item for Beam
>             3.0.
>              >
>              > Reuven
>              >
>              > On Mon, Dec 4, 2017 at 8:20 PM, Jean-Baptiste Onofré
>             <jb@nanthrax.net <mailto:jb@nanthrax.net>
>              > <mailto:jb@nanthrax.net <mailto:jb@nanthrax.net>>> wrote:
>              >
>              >     My apologizes, I thought we had a consensus already.
>              >
>              >     Regards
>              >     JB
>              >
>              >     On 12/04/2017 11:22 PM, Eugene Kirpichov wrote:
>              >
>              >         Thanks JB for sending the detailed notes about new stuff
>             in 2.2.0! A lot
>              >         of exciting things indeed.
>              >
>              >         Regarding Java 8: I thought our consensus was to have
the
>             release notes
>              >         say that we're *considering* going Java8-only, and use
>             that to get more
>              >         opinions from the user community - but I can't find the
>             emails that made
>              >         me think so.
>              >
>              >         +Ismaël Mejía <mailto:iemejia@gmail.com
>             <mailto:iemejia@gmail.com> <mailto:iemejia@gmail.com
>             <mailto:iemejia@gmail.com>>> - do
>              >         you think we should formally conclude the vote on the
>             thread [VOTE]
>              >         [DISCUSSION] Remove support for Java 7?
>              >         Or should we take more steps - e.g. perhaps tweet a link
>             to that thread
>              >         from the Beam twitter account, ask people to chime in,
>             and wait for say
>              >         2 weeks before declaring a conclusion?
>              >
>              >         Let's also have a process JIRA for going Java8. I've
>             filed one:
>              > https://issues.apache.org/jira/browse/BEAM-3285
>             <https://issues.apache.org/jira/browse/BEAM-3285>
>              >         <https://issues.apache.org/jira/browse/BEAM-3285
>             <https://issues.apache.org/jira/browse/BEAM-3285>>
>              >
>              >         On Mon, Dec 4, 2017 at 1:58 AM Jean-Baptiste Onofré
>             <jb@nanthrax.net <mailto:jb@nanthrax.net>
>              >         <mailto:jb@nanthrax.net <mailto:jb@nanthrax.net>>
>             <mailto:jb@nanthrax.net <mailto:jb@nanthrax.net>
>              >         <mailto:jb@nanthrax.net <mailto:jb@nanthrax.net>>>>
wrote:
>              >
>              >              Just an important note that we forgot to mention.
>              >
>              >              !! The 2.2.0 release will be the last one supporting
>             Spark 1.x and
>              >         Java 7 !!
>              >
>              >              Starting from Beam 2.3.0, the Spark runner will
work
>             only with
>              >         Spark 2.x and we
>              >              will focus only Java 8.
>              >
>              >              Regards
>              >              JB
>              >
>              >              On 12/04/2017 10:15 AM, Jean-Baptiste Onofré wrote:
>              >               > Thanks Reuven !
>              >               >
>              >               > I would like to emphasize on some highlights
in
>             2.2.0 release:
>              >               >
>              >               > - New IOs have been introduced:
>              >               >   * TikaIO leveraging Apache Tika, allowing
the
>             deal with a lot
>              >         of different
>              >               > data formats
>              >               >   * RedisIO to read and write key/value
pairs
>             from a Redis
>              >         server. This
>              >              IO will
>              >               > be soon extended to Redis PubSub.
>              >               >   * FileIO provides transforms for working
with
>             files (raw).
>              >         Especially, it
>              >               > provides matching file patterns and read
on
>             patterns. It can be
>              >         easily
>              >              extended
>              >               > for a specific format (like we do in AvroIO
or
>             TextIO now).
>              >               >   * SolrIO to interact with Apache Solr
(Lucene)
>              >               >
>              >               > - On the other hand, improvements have been
>             performed on
>              >         existing IOs:
>              >               >   * We started to introduce readAll pattern
in
>             IOs (AvroIO,
>              >         TextIO, JdbcIO,
>              >               > ...), allowing to pass "request" arguments
via an
>             input PCollection.
>              >               >   * ElasticsearchIO has an improved support
of
>             different
>              >         Elasticsearch
>              >              version
>              >               > (including Elasticsearch 5.x). It also now
>             supports SSL/TLS.
>              >               >   * HBaseIO is now able to do dynamic
work
>             rebalancing
>              >               >   * KinesisIO uses a more accurate watermark
>             (based on
>              >              approximateArrivalTimestamp)
>              >               >   * TextIO now supports custom delimiter
and like
>             AvroIO,
>              >         supports the
>              >              readAll
>              >               > pattern,
>              >               >   * Performance improvements on JdbcIO
when it
>             has to read lot
>              >         of rows
>              >               >   * Kafka write supports Exactly-Once
pattern
>             (introduce in
>              >         Kafka 0.11.x)
>              >               >
>              >               > - A new DSL has been introduced: the SQL
DSL !
>              >               >
>              >               > We are now focus on 2.3.0 release with new
>             improvements and
>              >         features !
>              >               >
>              >               > Stay tuned !
>              >               >
>              >               > JB on behalf of the Apache Beam community.
>              >               >
>              >               > On 12/02/2017 11:40 PM, Reuven Lax wrote:
>              >               >> The Apache Beam community is pleased
to announce the
>              >         availability of the
>              >               >> 2.2.0 release.
>              >               >>
>              >               >> This release adds support for generic
file
>             sources and sinks
>              >         (beyond TextIO
>              >               >> and AvroIO) using FileIO, including
support for
>             dynamic
>              >         filenames using
>              >               >> readAll; this allows streaming pipelines
to now
>             read from files by
>              >               >> continuously monitoring a directory
for new
>             filw. Many other
>              >         IOs are
>              >              improved,
>              >               >> notably including exactly-once support
for the
>             Kafka sink. Initial
>              >              support for
>              >               >> BEAM-SQL is also included in this release.
For a
>             more-complete
>              >         list of major
>              >               >> changes in the release, please refer
to the
>             release notes [2].
>              >               >>
>              >               >> The 2.2.0 release is now the recommended
>             version; we encourage
>              >         everyone to
>              >               >> upgrade from any earlier releases.
>              >               >>
>              >               >> We’d like to invite everyone to try
out Apache
>             Beam today and
>              >         consider
>              >               >> joining our vibrant community. We welcome
feedback,
>              >         contribution and
>              >               >> participation through our mailing lists,
issue
>             tracker, pull
>              >         requests, and
>              >               >> events.
>              >               >>
>              >               >> - Reuven Lax, on behalf of the Apache
Beam
>             community.
>              >               >>
>              >               >> [1]
>             https://beam.apache.org/get-started/downloads/
>             <https://beam.apache.org/get-started/downloads/>
>              >         <https://beam.apache.org/get-started/downloads/
>             <https://beam.apache.org/get-started/downloads/>>
>              >               >> [2]
>              >               >>
>              >
>             https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044
>             <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044>
>              >       
>               <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044
<https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044>>
>              >               >>
>              >               >
>              >
>              >              --
>              >              Jean-Baptiste Onofré
>              > jbonofre@apache.org <mailto:jbonofre@apache.org>
>             <mailto:jbonofre@apache.org <mailto:jbonofre@apache.org>>
>              >         <mailto:jbonofre@apache.org <mailto:jbonofre@apache.org>
>             <mailto:jbonofre@apache.org <mailto:jbonofre@apache.org>>>
>              > http://blog.nanthrax.net
>              >              Talend - http://www.talend.com
>              >
>              >
>              >     --
>              >     Jean-Baptiste Onofré
>              > jbonofre@apache.org <mailto:jbonofre@apache.org>
>             <mailto:jbonofre@apache.org <mailto:jbonofre@apache.org>>
>              > http://blog.nanthrax.net
>              >     Talend - http://www.talend.com
>              >
>              >
> 
>             --
>             Jean-Baptiste Onofré
>             jbonofre@apache.org <mailto:jbonofre@apache.org>
>             http://blog.nanthrax.net
>             Talend - http://www.talend.com
> 
> 
> 

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Mime
View raw message