ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Kasnacheev <ilya.kasnach...@gmail.com>
Subject Re: Apache Ignite RPM packaging (Stage II)
Date Mon, 02 Apr 2018 15:22:23 GMT
Hello!

Let me share my idea of how this shoud work. Splitting package into
sub-packages should be dependency-driven.

It means that all Ignite modules without dependencies or with small
dependencies (such as ignite-log4j) should be included in ignite-core. It
doesn't make sense to make a zillion RPM packages.

Critical things like ignite-spring and ignite-indexing should be in
ignite-core of course, even if they have dependencies. Ignite-core should
be fully self-sufficient and feature-complete.

However, e.g. .net API should probably be in a separate package, because it
should depend on mono | net-core. We may also have ignite-devel package
which should include all modules which only make sense for developers who
write code. Such as hibernate integration.

I'm not sure about MR modules. The main question should be, does it have
dependencies? Can it run stand-alone without writing code?

Hope this helps,

-- 
Ilya Kasnacheev

2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr.weider@gmail.com>:

> Hi, Igniters!
>
>
> Here are some news on our RPM packages initiative.
>
> 1. I’ve finished preliminary developing of Stage II version of RPM
> packages [1]. Main “new feature” is — split design. Also I’ve added
> package.sh script for automating package building process which will help
> organise corresponding builds in TC as well as simplify process for
> developers who wishes to have custom packages.
> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> repositories on Apache Bintray. Though they are already prepared for
> hosting RPM and DEB packages respectively, and there is a way of linking
> them to apache.org/dist/ignite page, there is possible alternative in
> storing there only plain directory layout corresponding to each repository
> type (RPM and DEB) and manage this layout (repodata, distributions,
> versions, etc.) by ourselves, having more control over repositories but
> lacking some simplicity of deploying new releases. WDYT? Should we try
> Cassandra approach? They are storing their DEB packages as I described
> above [5].
>
> Also — a question arose while I was working on this issue: which OSes (and
> which versions of each) are we going to support (if we are going) in terms
> of step-by-step list? Currently RPM packages are tested only with latest
> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> distributives [6] some of which are more o less popular among OS community
> (ALT, Fedora, openSUSE, etc.).
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> [2] https://github.com/apache/ignite/pull/3703
> [3] https://bintray.com/apache/ignite-rpm
> [4] https://bintray.com/apache/ignite-deb
> [5] https://bintray.com/apache/cassandra/debian#files/
> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>
>
>
>
> > On 15 Mar 2018, at 22:15, Petr Ivanov <mr.weider@gmail.com> wrote:
> >
> > I suppose that most everything if not all from libs/options will go to
> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > More precise lib selection (if something from optional would better to
> have in core package) will be discussed right after preliminary split
> architecture agreement.
> >
> >
> >
> >> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dpavlov.spb@gmail.com> wrote:
> >>
> >> I like idea of keeping simple system of modules, so +1 from me.
> >>
> >> Where optional libs (e.g Direct IO plugin) would be included, would it
> be
> >> core or optional?
> >>
> >> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dmagda@apache.org>:
> >>
> >>>>
> >>>>>
> >>>>> How big would be a final core module?
> >>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s
> own
> >>>> package.
> >>>
> >>>
> >>> Guys, 30 vs 280M is a huuuuge difference.  I would agree with Petr and
> >>> propose the simplest modular system:
> >>>
> >>>  - core module that includes basic Ignite capabilities including SQL,
> >>>  compute grid, service grid, k/v
> >>>  - optional module hosts the rest - ML, streamers integration (kafka,
> >>>  flink), kubernetes, etc.
> >>>
> >>> What do you think?
> >>>
> >>> --
> >>> Denis
> >>>
> >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr.weider@gmail.com>
> wrote:
> >>>
> >>>> *DEB package
> >>>>
> >>>>
> >>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr.weider@gmail.com>
wrote:
> >>>>>
> >>>>> Considering that DEV package for now is almost platform independent
> >>> (its
> >>>> a java application more or less), that package will work almost on any
> >>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> >>>>> The only restriction is existence of systemctl (systemd) service
> >>> manager
> >>>> — we are dependent on it.
> >>>>>
> >>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
> and
> >>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
> >>> Fedora,
> >>>> etc. with presence of aforementioned systemd.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <dsetrakyan@apache.org>
> >>>> wrote:
> >>>>>>
> >>>>>> Will Debian package work for Ubuntu?
> >>>>>>
> >>>>>> D.
> >>>>>>
> >>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr.weider@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Not a problem, rather nuisance. Also, when we will move
to official
> >>>>>>> repositories, there can be a problem from OS community.
> >>>>>>>
> >>>>>>> Concerning DEB packages — I plan to use RPM as base for
DEB package
> >>>> build
> >>>>>>> (package layout / install scripts) for speeding up things
and
> >>> excluding
> >>>>>>> possible duplication and desynchronisation, so its a matter
of ’sit
> >>>> and do’
> >>>>>>> rather then some technical research. Thats why I rose discussion
> >>> about
> >>>>>>> future package architecture, so that after agreement I'm
be able to
> >>>> pack
> >>>>>>> both RPM and DEB identically.
> >>>>>>>
> >>>>>>> Yet, if you insist, I can create DEB package according to
current
> RPM
> >>>>>>> layout in no time.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Peter,
> >>>>>>>>
> >>>>>>>> I don't think the package size of 280M is going to be
a problem at
> >>>> all,
> >>>>>>> but
> >>>>>>>> what you are suggesting can be an improvement down the
road.
> >>>>>>>>
> >>>>>>>> In the mean time, I think our top priority should be
to provide
> >>>> packages
> >>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
enough.
> >>>>>>>>
> >>>>>>>> Agree?
> >>>>>>>>
> >>>>>>>> D.
> >>>>>>>>
> >>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr.weider@gmail.com>
> >>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi, Igniters!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Release 2.4 is almost there, at least binary part
of it, so I'd
> >>> like
> >>>> to
> >>>>>>>>> move
> >>>>>>>>> forward to further improve and widen AI delivery
through
> packages.
> >>>>>>>>> As of now, Apache Ignite ships in RPM package weighing
about
> 280M+
> >>>> and,
> >>>>>>> to
> >>>>>>>>> improve usability and significantly reduce required
download
> >>> sizes, I
> >>>>>>>>> purpose that in 2.5 release we introduce splitted
delivery as
> >>>> follows:
> >>>>>>>>> - CORE
> >>>>>>>>> - bin
> >>>>>>>>> - config
> >>>>>>>>> - libs (!optional)
> >>>>>>>>> - OPTIONAL LIBS
> >>>>>>>>> - BENCHMARKS
> >>>>>>>>> - DOCS (?)
> >>>>>>>>> - EXAMPLES
> >>>>>>>>> - .NET PLATFORM FILES
> >>>>>>>>> - C++ PLATFORM FILES
> >>>>>>>>>
> >>>>>>>>> This architecture, as I assume, will add flexibility
(no reason
> to
> >>>>>>> download
> >>>>>>>>> all 280M+ of binaries where you are to run only
core node
> >>>> functionality)
> >>>>>>>>> and
> >>>>>>>>> maintainability (you are in full control of what
is installed on
> >>> your
> >>>>>>>>> system).
> >>>>>>>>>
> >>>>>>>>> After successful architecture choice, same scheme
are planned to
> be
> >>>>>>> used in
> >>>>>>>>> DEB packages as well.
> >>>>>>>>>
> >>>>>>>>> WDYT?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
> com/
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message