ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Petr Ivanov <mr.wei...@gmail.com>
Subject Re: Apache Ignite RPM packaging (Stage II)
Date Thu, 12 Apr 2018 16:10:54 GMT
If someone from PMCы or Committers still sees necessity about including these tasks into Apache
Ignite 2.5 release, this is the last chance to do so.
Otherwise this task will be moved to at 2.6 release at least, or even moved to backlog indefinitely.



> On 9 Apr 2018, at 19:08, Petr Ivanov <mr.weider@gmail.com> wrote:
> 
> To top new RPM architecture off, update to release process is introduced — [1] [2].
> 
> Both tasks (this one and IGNITE-7647) are ready for review and should be merged simultaneously.
> 
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-8172
> [2] https://github.com/apache/ignite-release/pull/1
> 
> 
> 
> 
>> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <ilya.kasnacheev@gmail.com> wrote:
>> 
>> Hello!
>> 
>> Let me share my idea of how this shoud work. Splitting package into
>> sub-packages should be dependency-driven.
>> 
>> It means that all Ignite modules without dependencies or with small
>> dependencies (such as ignite-log4j) should be included in ignite-core. It
>> doesn't make sense to make a zillion RPM packages.
>> 
>> Critical things like ignite-spring and ignite-indexing should be in
>> ignite-core of course, even if they have dependencies. Ignite-core should
>> be fully self-sufficient and feature-complete.
>> 
>> However, e.g. .net API should probably be in a separate package, because it
>> should depend on mono | net-core. We may also have ignite-devel package
>> which should include all modules which only make sense for developers who
>> write code. Such as hibernate integration.
>> 
>> I'm not sure about MR modules. The main question should be, does it have
>> dependencies? Can it run stand-alone without writing code?
>> 
>> Hope this helps,
>> 
>> -- 
>> Ilya Kasnacheev
>> 
>> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr.weider@gmail.com>:
>> 
>>> Hi, Igniters!
>>> 
>>> 
>>> Here are some news on our RPM packages initiative.
>>> 
>>> 1. I’ve finished preliminary developing of Stage II version of RPM
>>> packages [1]. Main “new feature” is — split design. Also I’ve added
>>> package.sh script for automating package building process which will help
>>> organise corresponding builds in TC as well as simplify process for
>>> developers who wishes to have custom packages.
>>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>>> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
>>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
>>> repositories on Apache Bintray. Though they are already prepared for
>>> hosting RPM and DEB packages respectively, and there is a way of linking
>>> them to apache.org/dist/ignite page, there is possible alternative in
>>> storing there only plain directory layout corresponding to each repository
>>> type (RPM and DEB) and manage this layout (repodata, distributions,
>>> versions, etc.) by ourselves, having more control over repositories but
>>> lacking some simplicity of deploying new releases. WDYT? Should we try
>>> Cassandra approach? They are storing their DEB packages as I described
>>> above [5].
>>> 
>>> Also — a question arose while I was working on this issue: which OSes (and
>>> which versions of each) are we going to support (if we are going) in terms
>>> of step-by-step list? Currently RPM packages are tested only with latest
>>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
>>> distributives [6] some of which are more o less popular among OS community
>>> (ALT, Fedora, openSUSE, etc.).
>>> 
>>> 
>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>>> [2] https://github.com/apache/ignite/pull/3703
>>> [3] https://bintray.com/apache/ignite-rpm
>>> [4] https://bintray.com/apache/ignite-deb
>>> [5] https://bintray.com/apache/cassandra/debian#files/
>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>>> 
>>> 
>>> 
>>> 
>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr.weider@gmail.com> wrote:
>>>> 
>>>> I suppose that most everything if not all from libs/options will go to
>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>>>> More precise lib selection (if something from optional would better to
>>> have in core package) will be discussed right after preliminary split
>>> architecture agreement.
>>>> 
>>>> 
>>>> 
>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dpavlov.spb@gmail.com>
wrote:
>>>>> 
>>>>> I like idea of keeping simple system of modules, so +1 from me.
>>>>> 
>>>>> Where optional libs (e.g Direct IO plugin) would be included, would it
>>> be
>>>>> core or optional?
>>>>> 
>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dmagda@apache.org>:
>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> How big would be a final core module?
>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create
it’s
>>> own
>>>>>>> package.
>>>>>> 
>>>>>> 
>>>>>> Guys, 30 vs 280M is a huuuuge difference.  I would agree with Petr
and
>>>>>> propose the simplest modular system:
>>>>>> 
>>>>>> - core module that includes basic Ignite capabilities including SQL,
>>>>>> compute grid, service grid, k/v
>>>>>> - optional module hosts the rest - ML, streamers integration (kafka,
>>>>>> flink), kubernetes, etc.
>>>>>> 
>>>>>> What do you think?
>>>>>> 
>>>>>> --
>>>>>> Denis
>>>>>> 
>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr.weider@gmail.com>
>>> wrote:
>>>>>> 
>>>>>>> *DEB package
>>>>>>> 
>>>>>>> 
>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr.weider@gmail.com>
wrote:
>>>>>>>> 
>>>>>>>> Considering that DEV package for now is almost platform independent
>>>>>> (its
>>>>>>> a java application more or less), that package will work almost
on any
>>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian,
etc.
>>>>>>>> The only restriction is existence of systemctl (systemd)
service
>>>>>> manager
>>>>>>> — we are dependent on it.
>>>>>>>> 
>>>>>>>> Thats why, for instance, our RPM repository is called simply
‘rpm’
>>> and
>>>>>>> package has no arch or dist suffix — it will work on CentOS,
RHEL,
>>>>>> Fedora,
>>>>>>> etc. with presence of aforementioned systemd.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <dsetrakyan@apache.org>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Will Debian package work for Ubuntu?
>>>>>>>>> 
>>>>>>>>> D.
>>>>>>>>> 
>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr.weider@gmail.com>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Not a problem, rather nuisance. Also, when we will
move to official
>>>>>>>>>> repositories, there can be a problem from OS community.
>>>>>>>>>> 
>>>>>>>>>> Concerning DEB packages — I plan to use RPM as
base for DEB package
>>>>>>> build
>>>>>>>>>> (package layout / install scripts) for speeding up
things and
>>>>>> excluding
>>>>>>>>>> possible duplication and desynchronisation, so its
a matter of ’sit
>>>>>>> and do’
>>>>>>>>>> rather then some technical research. Thats why I
rose discussion
>>>>>> about
>>>>>>>>>> future package architecture, so that after agreement
I'm be able to
>>>>>>> pack
>>>>>>>>>> both RPM and DEB identically.
>>>>>>>>>> 
>>>>>>>>>> Yet, if you insist, I can create DEB package according
to current
>>> RPM
>>>>>>>>>> layout in no time.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
>>> dsetrakyan@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Peter,
>>>>>>>>>>> 
>>>>>>>>>>> I don't think the package size of 280M is going
to be a problem at
>>>>>>> all,
>>>>>>>>>> but
>>>>>>>>>>> what you are suggesting can be an improvement
down the road.
>>>>>>>>>>> 
>>>>>>>>>>> In the mean time, I think our top priority should
be to provide
>>>>>>> packages
>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not
nearly enough.
>>>>>>>>>>> 
>>>>>>>>>>> Agree?
>>>>>>>>>>> 
>>>>>>>>>>> D.
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr.weider@gmail.com>
>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Release 2.4 is almost there, at least binary
part of it, so I'd
>>>>>> like
>>>>>>> to
>>>>>>>>>>>> move
>>>>>>>>>>>> forward to further improve and widen AI delivery
through
>>> packages.
>>>>>>>>>>>> As of now, Apache Ignite ships in RPM package
weighing about
>>> 280M+
>>>>>>> and,
>>>>>>>>>> to
>>>>>>>>>>>> improve usability and significantly reduce
required download
>>>>>> sizes, I
>>>>>>>>>>>> purpose that in 2.5 release we introduce
splitted delivery as
>>>>>>> follows:
>>>>>>>>>>>> - CORE
>>>>>>>>>>>> - bin
>>>>>>>>>>>> - config
>>>>>>>>>>>> - libs (!optional)
>>>>>>>>>>>> - OPTIONAL LIBS
>>>>>>>>>>>> - BENCHMARKS
>>>>>>>>>>>> - DOCS (?)
>>>>>>>>>>>> - EXAMPLES
>>>>>>>>>>>> - .NET PLATFORM FILES
>>>>>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>>>>> 
>>>>>>>>>>>> This architecture, as I assume, will add
flexibility (no reason
>>> to
>>>>>>>>>> download
>>>>>>>>>>>> all 280M+ of binaries where you are to run
only core node
>>>>>>> functionality)
>>>>>>>>>>>> and
>>>>>>>>>>>> maintainability (you are in full control
of what is installed on
>>>>>> your
>>>>>>>>>>>> system).
>>>>>>>>>>>> 
>>>>>>>>>>>> After successful architecture choice, same
scheme are planned to
>>> be
>>>>>>>>>> used in
>>>>>>>>>>>> DEB packages as well.
>>>>>>>>>>>> 
>>>>>>>>>>>> WDYT?
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>>> com/
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>> 
>>> 
> 


Mime
View raw message