ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Zinoviev <zaleslaw....@gmail.com>
Subject Re: Ignite-Spark integration meeting
Date Wed, 23 Oct 2019 15:46:50 GMT
Good point, now they are not in the Best state, we have a few ideas how to
fix, will discuss it later here in this thread.

If you have any objection write it, but my position - creation special
Spark Visa on TC, like for ML, maybe two for different Spark versions, and
all commits to Spark will require this Visa.


ср, 23 окт. 2019 г., 16:25 Ivan Pavlukhin <vololo100@gmail.com>:

> Actually, I meant running tests on TeamCity.
>
> ср, 23 окт. 2019 г. в 15:36, Alexey Zinoviev <zaleslaw.sin@gmail.com>:
> >
> > What kind of testing do you mean?  JUnit, integration via example or
> > something else?
> >
> > ср, 23 окт. 2019 г., 14:16 Ivan Pavlukhin <vololo100@gmail.com>:
> >
> > > Correction.
> > >
> > > How are you going to TEST integrations with different Spark versions?
> > >
> > > ср, 23 окт. 2019 г. в 15:16, Ivan Pavlukhin <vololo100@gmail.com>:
> > > >
> > > > Alexey,
> > > >
> > > > Thank you for sharing the summary!
> > > >
> > > > Small question. How are you going to integrations with different
> Spark
> > > versions?
> > > >
> > > > чт, 17 окт. 2019 г. в 22:30, Denis Magda <dmagda@apache.org>:
> > > > >
> > > > > >
> > > > > >  1. We are going to release Apache Ignite 2.8 with limited
> support of
> > > > > >    Spark 2.4.4 (known issues are listed here
> > > > > >    https://issues.apache.org/jira/browse/IGNITE-12054)
> > > > >
> > > > >
> > > > > What does limited support mean? Is it about regressions (something
> that
> > > > > worked before but will fail) or capabilities unsupported even for
> the
> > > > > presently released version of the integration?
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Thu, Oct 17, 2019 at 7:55 AM Alexey Zinoviev <
> > > zaleslaw.sin@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > The meeting notes and results of discussion are next:
> > > > > >
> > > > > >    1. We are going to release Apache Ignite 2.8 with limited
> support
> > > of
> > > > > >    Spark 2.4.4 (known issues are listed here
> > > > > >    https://issues.apache.org/jira/browse/IGNITE-12054)
> > > > > >    2. The AI 2.8 will support both, Spark 2.3 and Spark 2.4.4
> with
> > > known
> > > > > >    fixed bugs
> > > > > >    3. We will create the developer-profiles spark_2_3 and
> spark_2_4
> > > > > >    separately from current scala profile
> > > > > >    4. We are ready to move the Ignite-Spark integration to the
> > > separate
> > > > > >    module after release 2.8 with support of most actual Spark
> > > versions and
> > > > > >    with independent release cycle
> > > > > >    5. We will work together on the next release roadmap with
> Nikolay
> > > and
> > > > > >    all interested igniters to improve the current solution (add
> new
> > > > > > features
> > > > > >    as thin client support, working with caches via binary object
> and
> > > so on)
> > > > > >    6. Nikolay is ready to help with the review of tickets as
an
> > > author of
> > > > > >    this component
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > >
> > > > > >
> > > > > > пн, 14 окт. 2019 г. в 14:36, Alexey Zinoviev <
> zaleslaw.sin@gmail.com
> > > >:
> > > > > >
> > > > > > > Yes, let's use the Russian for short discussion, the meeting
> notes
> > > will
> > > > > > be
> > > > > > > presented in English
> > > > > > >
> > > > > > > пн, 14 окт. 2019 г. в 14:08, Nikolay Izhikov <
> nizhikov@apache.org
> > > >:
> > > > > > >
> > > > > > >> Hello, Alexey.
> > > > > > >>
> > > > > > >> I'm ready to take part in the meeting.
> > > > > > >> It will be in Russian, isn't it?
> > > > > > >>
> > > > > > >> В Пн, 14/10/2019 в 14:00 +0300, Alexey Zinoviev
пишет:
> > > > > > >> > Hi, Igniters, and especially Nikolay Izhikov
> > > > > > >> >
> > > > > > >> > I suggest to arrange the meeting via ASF-slack
in
> > > > > > >> #ignite-spark-integration
> > > > > > >> > channel at 17 October, in 16 MSK Time.
> > > > > > >> > The language of the meeting: Russian
> > > > > > >> >
> > > > > > >> > *The topics to discuss:*
> > > > > > >> >
> > > > > > >> >    - Review of next tickets
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >    1. [Spark][Bug] IgniteSpark integration forget
to close
> the
> > > > > > >> >       IgniteContext and stops the client node
in case if
> error
> > > during
> > > > > > >> >       PairFunction logic
> > > > > > >> >       <https://issues.apache.org/jira/browse/IGNITE-11724>
> > > > > > >> >       2. [Spark] Add initial support of Spark
2.4
> > > > > > >> >       <https://issues.apache.org/jira/browse/IGNITE-12247>
> > > > > > >> >       3. [Spark] Add joins to Spark Dataframe
examples
> > > > > > >> >       <https://issues.apache.org/jira/browse/IGNITE-9732>
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >    - Known issues of migration to 2.4 version
> > > > > > >> >    - Organization of multiple version support
for Apache
> Spark
> > > > > > >> >
> > > > > > >> > P.S. meeting notes will be shared with the community
> > > > > > >> > P.P.S Nikolay, if you have any objections about
the time or
> > > topics,
> > > > > > >> please
> > > > > > >> > let me know
> > > > > > >>
> > > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message