ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Pavlov <dpav...@apache.org>
Subject Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator
Date Tue, 18 Jun 2019 18:28:22 GMT
Hi Denis,

I like the first approach with applying Maxim's idea of creating a branch
named -igfs-Hadoop (not release, but current master state).
2nd) 3rd party repo can be Apache repo just like ignite-release. But it's
true it is time-consuming to move code.
3rd) Attic is for projects, I hope no one here wants to Ignite to be there
:) I'm not sure it is possible to move just one component there. But if it
is possible, we should anyway start from 2nd option and create standalone
repo ignite-igfs-hadoop (and it will become later attic-ignite-igfs-hadoop).

But if someone could stand up and say he/she wants to do migration from one
repo to another (option 2), I like it as well.

Sincerely

вт, 18 июн. 2019 г. в 21:05, Denis Magda <dmagda@apache.org>:

> Igniters,
>
> Thanks a lot for sharing your opinion. As I see, there is a consensus that
> IGFS and Hadoop Accelerator are to be discontinued and no longer supported
> by the community.
>
> As for the source code, if the community prefers moving the source code to
> another repository rather than removing it, then let's do it. I see 3
> solutions here:
>
>    - The simplest - just point out to the latest Ignite release branch that
>    has the source code. This should be Ignite 2.6.0. Remove from Ignite
> master.
>    - Decouple from the master and move to a 3rd party Github repo. More
>    complicated and time-consuming.
>    - See if we should move the component to Apache Attic (
>    http://attic.apache.org): the Attic is designed for projects to be
>    retired but not for the components. Thus, that might be not an option.
>
> Personally, I'm for the first approach. Does it sound reasonable?
>
> -
> Denis
>
>
> On Tue, Jun 18, 2019 at 7:39 AM Maxim Muzafarov <maxmuzaf@gmail.com>
> wrote:
>
> > Folks,
> >
> > +1 to reduce the number of supported features.
> >
> > Probably, the best solution will be removing IGFS from core module and
> > making it as an Ignite plugin (will require some efforts to do this).
> > I've also think we can move IGFS to the separate branch (from the
> > master one) if someone will decide merge to latest changes from the
> > master branch to build Ignite from scratch with IGFS feature.
> >
> > On Mon, 17 Jun 2019 at 22:42, Denis Magda <dmagda@apache.org> wrote:
> > >
> > > >
> > > > +1 from me to reduce supported feature list.
> > > > Guys, are we talking about Ignite 3.0?
> > >
> > >
> > > Nikolay, I would discontinue IGFS before 3.0. Let's do this in the next
> > > release? As for other features and integrations, 3.0 should be
> considered
> > > as a version.
> > >
> > >
> > > > +1 from me provided that we move sources to some supplementary
> > repository.
> > > > If someone later would like to maintain/fix this module, he/she
> should
> > be
> > > > able to access sources with current state of the master.
> > >
> > >
> > > Dmitry, are you suggesting to move the sources to Github and abandon
> them
> > > there? Sort of legacy code cemetery.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Jun 17, 2019 at 2:00 AM Nikolay Izhikov <nizhikov@apache.org>
> > wrote:
> > >
> > > > +1 from me to reduce supported feature list.
> > > >
> > > > Guys, are we talking about Ignite 3.0?
> > > >
> > > >
> > > > В Пн, 17/06/2019 в 11:57 +0300, Alexey Goncharuk пишет:
> > > > > Denis,
> > > > >
> > > > > I fully support this idea.
> > > > >
> > > > > First, looking back, I do not think it was a good design in the
> first
> > > > place
> > > > > to build IGFS on top of Ignite caches. Second, I have never seen
a
> > case
> > > > > where IGFS provided significant performance boost. Usually it's
> > either
> > > > all
> > > > > data already fits buffer cache, and IGFS caching is not needed; or
> > data
> > > > > does not fit buffer cache, and access pattern is close to full scan
> > and
> > > > > additional caching in IGFS does not make sense.
> > > > >
> > > > > пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <vololo100@gmail.com>:
> > > > >
> > > > > > Denis,
> > > > > >
> > > > > > I must say that aforementioned solutions for a Hadoop ecosystem
> > appear
> > > > > > from time to time in questions on a user mailing list. So, it
> seems
> > > > > > that there is a practical need for such solutions.
> > > > > >
> > > > > > But of course it does not mean that we should continue a support
> of
> > > > > > IGFS and Hadoop Accelerator. If both are not solutions that
fit
> > well
> > > > > > common use cases then we should discontinue it. If any of them
is
> > very
> > > > > > good for it's purposes but we do not have a capacity to support
> it
> > > > > > without sacrificing main Ignite goals then we still should
> > discontinue
> > > > > > it in my mind.
> > > > > >
> > > > > > P.S. Personally I am a fan of UNIX way. I like ideas of a single
> > > > > > responsibility and integrations. And I suppose there are other
> > Ignite
> > > > > > features which could be dropped.
> > > > > >
> > > > > > ср, 12 июн. 2019 г. в 21:04, Denis Magda <dmagda@apache.org>:
> > > > > >
> > > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I'd like us to move on and finish our conversation on the
IGFS
> > [1]
> > > > and
> > > > > > > Hadoop Accelerator [2] support.
> > > > > > >
> > > > > > > To my knowledge, there is no single committer who maintains
the
> > > > > > > integrations; they are no longer tested and, even more,
the
> > community
> > > > > > > stopped providing the binaries since Ignite 2.6.0 release
(look
> > for
> > > > > > > In-Memory Hadoop Accelerator table [3]).
> > > > > > >
> > > > > > > Why all of that happened? Because of a little value, something
> > > > succeeds
> > > > > > > while something fails. Does it mean that Ignite cannot
be used
> > for
> > > > Hadoop
> > > > > > > acceleration, in general? No, quite the opposite, it CAN
be
> used,
> > > > but a
> > > > > > > solution is different. Have Ignite with native persistence
> > deployed
> > > > close
> > > > > > > to your Hadoop cluster (replace GridGain with Ignite) [4].
> > > > > > >
> > > > > > > So, I propose we remove IGFS and In-Memory Hadoop Accelerator
> > from
> > > > our
> > > > > > > master repository and rework existing public documentation
> > showing
> > > > how to
> > > > > > > achieve the acceleration with Ignite.
> > > > > > >
> > > > > > > Any supporters or objections?
> > > > > > >
> > > > > > >
> > > > > > > [1]
> https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > > > > > > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > > > > > > [3] https://ignite.apache.org/download.cgi#binaries
> > > > > > > [4]
> > > > > > >
> > > > > >
> > > > > >
> > > >
> >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message