ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Apache Ignite RPM packaging (Stage II)
Date Thu, 12 Apr 2018 23:39:24 GMT
I am not sure that an email thread of over 20 messages is a good medium to
discuss proposals. In Ignite, we create IEPs. Can you please summarize your
proposal there and send a link there? Please explain not only the change
itself, but the reason why we need it.

D.

On Thu, Apr 12, 2018 at 4:00 PM, Denis Magda <dmagda@apache.org> wrote:

> Petr,
>
> I wouldn't postpone this until 2.6 that will be out nor earlier than 3
> months from now.
>
> *Anton V.*, could review and sign off the changes? Not sure we have a
> better person in the community who can do that.
>
> --
> Denis
>
> On Thu, Apr 12, 2018 at 9:10 AM, Petr Ivanov <mr.weider@gmail.com> wrote:
>
> > If someone from PMCы or Committers still sees necessity about including
> > these tasks into Apache Ignite 2.5 release, this is the last chance to do
> > so.
> > Otherwise this task will be moved to at 2.6 release at least, or even
> > moved to backlog indefinitely.
> >
> >
> >
> > > On 9 Apr 2018, at 19:08, Petr Ivanov <mr.weider@gmail.com> wrote:
> > >
> > > To top new RPM architecture off, update to release process is
> introduced
> > — [1] [2].
> > >
> > > Both tasks (this one and IGNITE-7647) are ready for review and should
> be
> > merged simultaneously.
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > > [2] https://github.com/apache/ignite-release/pull/1
> > >
> > >
> > >
> > >
> > >> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <ilya.kasnacheev@gmail.com>
> > wrote:
> > >>
> > >> Hello!
> > >>
> > >> Let me share my idea of how this shoud work. Splitting package into
> > >> sub-packages should be dependency-driven.
> > >>
> > >> It means that all Ignite modules without dependencies or with small
> > >> dependencies (such as ignite-log4j) should be included in ignite-core.
> > It
> > >> doesn't make sense to make a zillion RPM packages.
> > >>
> > >> Critical things like ignite-spring and ignite-indexing should be in
> > >> ignite-core of course, even if they have dependencies. Ignite-core
> > should
> > >> be fully self-sufficient and feature-complete.
> > >>
> > >> However, e.g. .net API should probably be in a separate package,
> > because it
> > >> should depend on mono | net-core. We may also have ignite-devel
> package
> > >> which should include all modules which only make sense for developers
> > who
> > >> write code. Such as hibernate integration.
> > >>
> > >> I'm not sure about MR modules. The main question should be, does it
> have
> > >> dependencies? Can it run stand-alone without writing code?
> > >>
> > >> Hope this helps,
> > >>
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr.weider@gmail.com>:
> > >>
> > >>> Hi, Igniters!
> > >>>
> > >>>
> > >>> Here are some news on our RPM packages initiative.
> > >>>
> > >>> 1. I’ve finished preliminary developing of Stage II version of RPM
> > >>> packages [1]. Main “new feature” is — split design. Also I’ve
added
> > >>> package.sh script for automating package building process which will
> > help
> > >>> organise corresponding builds in TC as well as simplify process for
> > >>> developers who wishes to have custom packages.
> > >>> PR#3703 [2] is ready for review. Denis, in order to catch up with
> > Apache
> > >>> Ignite 2.5 release, I’d greatly appreciate your help in finding
> > reviewer.
> > >>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> > >>> repositories on Apache Bintray. Though they are already prepared for
> > >>> hosting RPM and DEB packages respectively, and there is a way of
> > linking
> > >>> them to apache.org/dist/ignite page, there is possible alternative
> in
> > >>> storing there only plain directory layout corresponding to each
> > repository
> > >>> type (RPM and DEB) and manage this layout (repodata, distributions,
> > >>> versions, etc.) by ourselves, having more control over repositories
> but
> > >>> lacking some simplicity of deploying new releases. WDYT? Should we
> try
> > >>> Cassandra approach? They are storing their DEB packages as I
> described
> > >>> above [5].
> > >>>
> > >>> Also — a question arose while I was working on this issue: which
OSes
> > (and
> > >>> which versions of each) are we going to support (if we are going) in
> > terms
> > >>> of step-by-step list? Currently RPM packages are tested only with
> > latest
> > >>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> > >>> distributives [6] some of which are more o less popular among OS
> > community
> > >>> (ALT, Fedora, openSUSE, etc.).
> > >>>
> > >>>
> > >>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> > >>> [2] https://github.com/apache/ignite/pull/3703
> > >>> [3] https://bintray.com/apache/ignite-rpm
> > >>> [4] https://bintray.com/apache/ignite-deb
> > >>> [5] https://bintray.com/apache/cassandra/debian#files/
> > >>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> > distributions
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr.weider@gmail.com>
wrote:
> > >>>>
> > >>>> I suppose that most everything if not all from libs/options will
go
> to
> > >>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > >>>> More precise lib selection (if something from optional would better
> to
> > >>> have in core package) will be discussed right after preliminary split
> > >>> architecture agreement.
> > >>>>
> > >>>>
> > >>>>
> > >>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dpavlov.spb@gmail.com>
> > wrote:
> > >>>>>
> > >>>>> I like idea of keeping simple system of modules, so +1 from
me.
> > >>>>>
> > >>>>> Where optional libs (e.g Direct IO plugin) would be included,
would
> > it
> > >>> be
> > >>>>> core or optional?
> > >>>>>
> > >>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dmagda@apache.org>:
> > >>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> How big would be a final core module?
> > >>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor
and create
> > it’s
> > >>> own
> > >>>>>>> package.
> > >>>>>>
> > >>>>>>
> > >>>>>> Guys, 30 vs 280M is a huuuuge difference.  I would agree
with Petr
> > and
> > >>>>>> propose the simplest modular system:
> > >>>>>>
> > >>>>>> - core module that includes basic Ignite capabilities including
> SQL,
> > >>>>>> compute grid, service grid, k/v
> > >>>>>> - optional module hosts the rest - ML, streamers integration
> (kafka,
> > >>>>>> flink), kubernetes, etc.
> > >>>>>>
> > >>>>>> What do you think?
> > >>>>>>
> > >>>>>> --
> > >>>>>> Denis
> > >>>>>>
> > >>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> mr.weider@gmail.com>
> > >>> wrote:
> > >>>>>>
> > >>>>>>> *DEB package
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr.weider@gmail.com>
> > wrote:
> > >>>>>>>>
> > >>>>>>>> Considering that DEV package for now is almost
platform
> > independent
> > >>>>>> (its
> > >>>>>>> a java application more or less), that package will
work almost
> on
> > any
> > >>>>>>> DEB-based linux, including but not limited to Ubuntu,
Debian,
> etc.
> > >>>>>>>> The only restriction is existence of systemctl
(systemd) service
> > >>>>>> manager
> > >>>>>>> — we are dependent on it.
> > >>>>>>>>
> > >>>>>>>> Thats why, for instance, our RPM repository is
called simply
> ‘rpm’
> > >>> and
> > >>>>>>> package has no arch or dist suffix — it will work
on CentOS,
> RHEL,
> > >>>>>> Fedora,
> > >>>>>>> etc. with presence of aforementioned systemd.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan
<
> > dsetrakyan@apache.org>
> > >>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Will Debian package work for Ubuntu?
> > >>>>>>>>>
> > >>>>>>>>> D.
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov
<
> > mr.weider@gmail.com>
> > >>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Not a problem, rather nuisance. Also, when
we will move to
> > official
> > >>>>>>>>>> repositories, there can be a problem from
OS community.
> > >>>>>>>>>>
> > >>>>>>>>>> Concerning DEB packages — I plan to use
RPM as base for DEB
> > package
> > >>>>>>> build
> > >>>>>>>>>> (package layout / install scripts) for
speeding up things and
> > >>>>>> excluding
> > >>>>>>>>>> possible duplication and desynchronisation,
so its a matter of
> > ’sit
> > >>>>>>> and do’
> > >>>>>>>>>> rather then some technical research. Thats
why I rose
> discussion
> > >>>>>> about
> > >>>>>>>>>> future package architecture, so that after
agreement I'm be
> > able to
> > >>>>>>> pack
> > >>>>>>>>>> both RPM and DEB identically.
> > >>>>>>>>>>
> > >>>>>>>>>> Yet, if you insist, I can create DEB package
according to
> > current
> > >>> RPM
> > >>>>>>>>>> layout in no time.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan
<
> > >>> dsetrakyan@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Peter,
> > >>>>>>>>>>>
> > >>>>>>>>>>> I don't think the package size of 280M
is going to be a
> > problem at
> > >>>>>>> all,
> > >>>>>>>>>> but
> > >>>>>>>>>>> what you are suggesting can be an improvement
down the road.
> > >>>>>>>>>>>
> > >>>>>>>>>>> In the mean time, I think our top priority
should be to
> provide
> > >>>>>>> packages
> > >>>>>>>>>>> for Debian and Ubuntu. Having only
RPMs is not nearly enough.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Agree?
> > >>>>>>>>>>>
> > >>>>>>>>>>> D.
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider
<
> mr.weider@gmail.com>
> > >>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Hi, Igniters!
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Release 2.4 is almost there, at
least binary part of it, so
> > I'd
> > >>>>>> like
> > >>>>>>> to
> > >>>>>>>>>>>> move
> > >>>>>>>>>>>> forward to further improve and
widen AI delivery through
> > >>> packages.
> > >>>>>>>>>>>> As of now, Apache Ignite ships
in RPM package weighing about
> > >>> 280M+
> > >>>>>>> and,
> > >>>>>>>>>> to
> > >>>>>>>>>>>> improve usability and significantly
reduce required download
> > >>>>>> sizes, I
> > >>>>>>>>>>>> purpose that in 2.5 release we
introduce splitted delivery
> as
> > >>>>>>> follows:
> > >>>>>>>>>>>> - CORE
> > >>>>>>>>>>>> - bin
> > >>>>>>>>>>>> - config
> > >>>>>>>>>>>> - libs (!optional)
> > >>>>>>>>>>>> - OPTIONAL LIBS
> > >>>>>>>>>>>> - BENCHMARKS
> > >>>>>>>>>>>> - DOCS (?)
> > >>>>>>>>>>>> - EXAMPLES
> > >>>>>>>>>>>> - .NET PLATFORM FILES
> > >>>>>>>>>>>> - C++ PLATFORM FILES
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> This architecture, as I assume,
will add flexibility (no
> > reason
> > >>> to
> > >>>>>>>>>> download
> > >>>>>>>>>>>> all 280M+ of binaries where you
are to run only core node
> > >>>>>>> functionality)
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>> maintainability (you are in full
control of what is
> installed
> > on
> > >>>>>> your
> > >>>>>>>>>>>> system).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> After successful architecture choice,
same scheme are
> planned
> > to
> > >>> be
> > >>>>>>>>>> used in
> > >>>>>>>>>>>> DEB packages as well.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> WDYT?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> --
> > >>>>>>>>>>>> Sent from: http://apache-ignite-
> developers.2346864.n4.nabble.
> > >>> com/
> > >>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>>
> > >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message