ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator
Date Mon, 17 Jun 2019 08:57:05 GMT
Denis,

I fully support this idea.

First, looking back, I do not think it was a good design in the first place
to build IGFS on top of Ignite caches. Second, I have never seen a case
where IGFS provided significant performance boost. Usually it's either all
data already fits buffer cache, and IGFS caching is not needed; or data
does not fit buffer cache, and access pattern is close to full scan and
additional caching in IGFS does not make sense.

пн, 17 июн. 2019 г. в 11:28, Павлухин Иван <vololo100@gmail.com>:

> Denis,
>
> I must say that aforementioned solutions for a Hadoop ecosystem appear
> from time to time in questions on a user mailing list. So, it seems
> that there is a practical need for such solutions.
>
> But of course it does not mean that we should continue a support of
> IGFS and Hadoop Accelerator. If both are not solutions that fit well
> common use cases then we should discontinue it. If any of them is very
> good for it's purposes but we do not have a capacity to support it
> without sacrificing main Ignite goals then we still should discontinue
> it in my mind.
>
> P.S. Personally I am a fan of UNIX way. I like ideas of a single
> responsibility and integrations. And I suppose there are other Ignite
> features which could be dropped.
>
> ср, 12 июн. 2019 г. в 21:04, Denis Magda <dmagda@apache.org>:
>
> >
> > Igniters,
> >
> > I'd like us to move on and finish our conversation on the IGFS [1] and
> > Hadoop Accelerator [2] support.
> >
> > To my knowledge, there is no single committer who maintains the
> > integrations; they are no longer tested and, even more, the community
> > stopped providing the binaries since Ignite 2.6.0 release (look for
> > In-Memory Hadoop Accelerator table [3]).
> >
> > Why all of that happened? Because of a little value, something succeeds
> > while something fails. Does it mean that Ignite cannot be used for Hadoop
> > acceleration, in general? No, quite the opposite, it CAN be used, but a
> > solution is different. Have Ignite with native persistence deployed close
> > to your Hadoop cluster (replace GridGain with Ignite) [4].
> >
> > So, I propose we remove IGFS and In-Memory Hadoop Accelerator from our
> > master repository and rework existing public documentation showing how to
> > achieve the acceleration with Ignite.
> >
> > Any supporters or objections?
> >
> >
> > [1] https://apacheignite-fs.readme.io/docs/in-memory-file-system
> > [2] https://apacheignite-fs.readme.io/docs/hadoop-accelerator
> > [3] https://ignite.apache.org/download.cgi#binaries
> > [4]
> >
> https://docs.gridgain.com/docs/bdb-getting-started#section-gridgain-data-lake-accelerator
> >
> > -
> > Denis
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message