ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Apache Ignite RPM packaging (Stage II)
Date Wed, 28 Mar 2018 15:02:08 GMT
>
> We can either
>     store files as is (as Cassandra does, see link below) in something
> that is called Generic Repository — this way we manage directory layouts
> ourselves and have more control over what is published and how
>         or
>     store files in prepared RPM / DEB repositories where we only upload
> packages and everything else repository does itself — this way we rely on
> Bintray’s implementations of corresponding repositories and have lower
> control over directory layout, but the overall process seems to be simpler.


Petr, he I would lean towards your experience and opinion. Personally, I
can't see any significant advantages or disadvantages of either approach.
This suggests me that we should go for the simplest (2nd) way.

--
Denis


On Tue, Mar 27, 2018 at 11:59 PM, Petr Ivanov <mr.weider@gmail.com> wrote:

>
>
> On 28 Mar 2018, at 00:18, Denis Magda <dmagda@gridgain.com> wrote:
>
> Petr, thanks for the update.
>
>
>> 1. I’ve finished preliminary developing of Stage II version of RPM
>> packages [1]. Main “new feature” is — split design. Also I’ve added
>> package.sh script for automating package building process which will help
>> organise corresponding builds in TC as well as simplify process for
>> developers who wishes to have custom packages.
>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
>
>
> *Anton*, would you be able to check up Petr's pull-request?
>
>
>> corresponding to each repository type (RPM and DEB) and manage this
>> layout (repodata, distributions, versions, etc.) by ourselves, having more
>> control over repositories but lacking some simplicity of deploying new
>> releases. WDYT? Should we try Cassandra approach? They are storing their
>> DEB packages as I described above [5].
>
>
> Petr, do bintray repos come for free for ASF project? Are there any
> limitations in regards total repository or package size?
>
>
> As I see, ASF has premium free account at Bintray with, I guess, virutally
> limitless posiiblilities (the only I stumbled across is limit to uploading
> files more that 250Mb through UI — upload is possible through API). Yet,
> I’ll ask INFRA specifically about this.
>
>
>
>  WDYT? Should we try Cassandra approach? They are storing their DEB
>> packages as I described above [5].
>
>
> I'm not sure I got the question. Could you show examples of both
> approaches?
>
>
> We can either
>     store files as is (as Cassandra does, see link below) in something
> that is called Generic Repository — this way we manage directory layouts
> ourselves and have more control over what is published and how
>         or
>     store files in prepared RPM / DEB repositories where we only upload
> packages and everything else repository does itself — this way we rely on
> Bintray’s implementations of corresponding repositories and have lower
> control over directory layout, but the overall process seems to be simpler.
>
>
>
> Also — a question arose while I was working on this issue: which OSes (and
>> which versions of each) are we going to support (if we are going) in terms
>> of step-by-step list? Currently RPM packages are tested only with latest
>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
>> distributives [6] some of which are more o less popular among OS community
>> (ALT, Fedora, openSUSE, etc.).
>
>
> I would test RHEL and Ubuntu for sure, of the last versions. As for the
> other distributions, I'd like to hear community opinion.
>
>
> There is minor to none necessity of testing packages on RHEL — CentOS is
> binary clone.
> Ubuntu is out of scope because it uses DEB packages and currently we are
> discussing RPM only.
>
>
>
> --
> Denis
>
>
> On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <mr.weider@gmail.com> wrote:
>
>> Hi, Igniters!
>>
>>
>> Here are some news on our RPM packages initiative.
>>
>> 1. I’ve finished preliminary developing of Stage II version of RPM
>> packages [1]. Main “new feature” is — split design. Also I’ve added
>> package.sh script for automating package building process which will help
>> organise corresponding builds in TC as well as simplify process for
>> developers who wishes to have custom packages.
>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
>> repositories on Apache Bintray. Though they are already prepared for
>> hosting RPM and DEB packages respectively, and there is a way of linking
>> them to apache.org/dist/ignite page, there is possible alternative in
>> storing there only plain directory layout corresponding to each repository
>> type (RPM and DEB) and manage this layout (repodata, distributions,
>> versions, etc.) by ourselves, having more control over repositories but
>> lacking some simplicity of deploying new releases. WDYT? Should we try
>> Cassandra approach? They are storing their DEB packages as I described
>> above [5].
>>
>> Also — a question arose while I was working on this issue: which OSes
>> (and which versions of each) are we going to support (if we are going) in
>> terms of step-by-step list? Currently RPM packages are tested only with
>> latest CentOS (and, respectively — RHEL), but there are a lot more
>> RPM-based distributives [6] some of which are more o less popular among OS
>> community (ALT, Fedora, openSUSE, etc.).
>>
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>> [2] https://github.com/apache/ignite/pull/3703
>> [3] https://bintray.com/apache/ignite-rpm
>> [4] https://bintray.com/apache/ignite-deb
>> [5] https://bintray.com/apache/cassandra/debian#files/
>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>>
>>
>>
>>
>> > On 15 Mar 2018, at 22:15, Petr Ivanov <mr.weider@gmail.com> wrote:
>> >
>> > I suppose that most everything if not all from libs/options will go to
>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>> > More precise lib selection (if something from optional would better to
>> have in core package) will be discussed right after preliminary split
>> architecture agreement.
>> >
>> >
>> >
>> >> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dpavlov.spb@gmail.com> wrote:
>> >>
>> >> I like idea of keeping simple system of modules, so +1 from me.
>> >>
>> >> Where optional libs (e.g Direct IO plugin) would be included, would it
>> be
>> >> core or optional?
>> >>
>> >> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dmagda@apache.org>:
>> >>
>> >>>>
>> >>>>>
>> >>>>> How big would be a final core module?
>> >>>> Around 30M. Can be shrinked to ~15M if separate Visor and create
>> it’s own
>> >>>> package.
>> >>>
>> >>>
>> >>> Guys, 30 vs 280M is a huuuuge difference.  I would agree with Petr and
>> >>> propose the simplest modular system:
>> >>>
>> >>>  - core module that includes basic Ignite capabilities including SQL,
>> >>>  compute grid, service grid, k/v
>> >>>  - optional module hosts the rest - ML, streamers integration (kafka,
>> >>>  flink), kubernetes, etc.
>> >>>
>> >>> What do you think?
>> >>>
>> >>> --
>> >>> Denis
>> >>>
>> >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr.weider@gmail.com>
>> wrote:
>> >>>
>> >>>> *DEB package
>> >>>>
>> >>>>
>> >>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr.weider@gmail.com>
wrote:
>> >>>>>
>> >>>>> Considering that DEV package for now is almost platform independent
>> >>> (its
>> >>>> a java application more or less), that package will work almost
on
>> any
>> >>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
>> >>>>> The only restriction is existence of systemctl (systemd) service
>> >>> manager
>> >>>> — we are dependent on it.
>> >>>>>
>> >>>>> Thats why, for instance, our RPM repository is called simply
‘rpm’
>> and
>> >>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
>> >>> Fedora,
>> >>>> etc. with presence of aforementioned systemd.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <dsetrakyan@apache.org
>> >
>> >>>> wrote:
>> >>>>>>
>> >>>>>> Will Debian package work for Ubuntu?
>> >>>>>>
>> >>>>>> D.
>> >>>>>>
>> >>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr.weider@gmail.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>>> Not a problem, rather nuisance. Also, when we will move
to
>> official
>> >>>>>>> repositories, there can be a problem from OS community.
>> >>>>>>>
>> >>>>>>> Concerning DEB packages — I plan to use RPM as base
for DEB
>> package
>> >>>> build
>> >>>>>>> (package layout / install scripts) for speeding up things
and
>> >>> excluding
>> >>>>>>> possible duplication and desynchronisation, so its a
matter of
>> ’sit
>> >>>> and do’
>> >>>>>>> rather then some technical research. Thats why I rose
discussion
>> >>> about
>> >>>>>>> future package architecture, so that after agreement
I'm be able
>> to
>> >>>> pack
>> >>>>>>> both RPM and DEB identically.
>> >>>>>>>
>> >>>>>>> Yet, if you insist, I can create DEB package according
to current
>> RPM
>> >>>>>>> layout in no time.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Peter,
>> >>>>>>>>
>> >>>>>>>> I don't think the package size of 280M is going
to be a problem
>> at
>> >>>> all,
>> >>>>>>> but
>> >>>>>>>> what you are suggesting can be an improvement down
the road.
>> >>>>>>>>
>> >>>>>>>> In the mean time, I think our top priority should
be to provide
>> >>>> packages
>> >>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
enough.
>> >>>>>>>>
>> >>>>>>>> Agree?
>> >>>>>>>>
>> >>>>>>>> D.
>> >>>>>>>>
>> >>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr.weider@gmail.com>
>> >>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi, Igniters!
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Release 2.4 is almost there, at least binary
part of it, so I'd
>> >>> like
>> >>>> to
>> >>>>>>>>> move
>> >>>>>>>>> forward to further improve and widen AI delivery
through
>> packages.
>> >>>>>>>>> As of now, Apache Ignite ships in RPM package
weighing about
>> 280M+
>> >>>> and,
>> >>>>>>> to
>> >>>>>>>>> improve usability and significantly reduce required
download
>> >>> sizes, I
>> >>>>>>>>> purpose that in 2.5 release we introduce splitted
delivery as
>> >>>> follows:
>> >>>>>>>>> - CORE
>> >>>>>>>>> - bin
>> >>>>>>>>> - config
>> >>>>>>>>> - libs (!optional)
>> >>>>>>>>> - OPTIONAL LIBS
>> >>>>>>>>> - BENCHMARKS
>> >>>>>>>>> - DOCS (?)
>> >>>>>>>>> - EXAMPLES
>> >>>>>>>>> - .NET PLATFORM FILES
>> >>>>>>>>> - C++ PLATFORM FILES
>> >>>>>>>>>
>> >>>>>>>>> This architecture, as I assume, will add flexibility
(no reason
>> to
>> >>>>>>> download
>> >>>>>>>>> all 280M+ of binaries where you are to run only
core node
>> >>>> functionality)
>> >>>>>>>>> and
>> >>>>>>>>> maintainability (you are in full control of
what is installed on
>> >>> your
>> >>>>>>>>> system).
>> >>>>>>>>>
>> >>>>>>>>> After successful architecture choice, same scheme
are planned
>> to be
>> >>>>>>> used in
>> >>>>>>>>> DEB packages as well.
>> >>>>>>>>>
>> >>>>>>>>> WDYT?
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> Sent from: http://apache-ignite-developer
>> s.2346864.n4.nabble.com/
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>
>> >
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message