ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Petr Ivanov <mr.wei...@gmail.com>
Subject Re: Apache Ignite RPM packaging (Stage II)
Date Wed, 28 Mar 2018 09:13:38 GMT
I can start preparing DEB packages right after adding RPM build to nightly release build (as
an experiment / experience for future addition of packages build into release process) basing
on current RPM architecture.
I will create branch from IGNITE-7647, then.



> On 28 Mar 2018, at 10:06, Dmitriy Setrakyan <dsetrakyan@apache.org> wrote:
> 
> Thanks, Petr!
> 
> I would love to test the package installation, but I can only do it on
> Ubuntu. Do you know when will we be able to get the Debian instructions,
> similar to this:
> 
> https://ignite.apache.org/download.cgi#rpm-package
> 
> D.
> 
> On Wed, Mar 28, 2018 at 12:01 AM, Petr Ivanov <mr.weider@gmail.com> wrote:
> 
>> No, not yet.
>> 
>> 
>> Currently we are discussing RPM packages only.
>> I want to get all feedback and possible errors working on RPM packages, so
>> that when we have stable agreed architecture and etc. I can recreate it in
>> DEB packages without necessity to fix bugs in both RPM and DEB packages
>> simultaneously.
>> 
>> 
>> 
>>> On 28 Mar 2018, at 03:17, Dmitriy Setrakyan <dsetrakyan@apache.org>
>> wrote:
>>> 
>>> Petr,
>>> 
>>> I am confused. Do we already have Debian packages?
>>> 
>>> D.
>>> 
>>> On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <mr.weider@gmail.com>
>> wrote:
>>> 
>>>> Hi, Igniters!
>>>> 
>>>> 
>>>> Here are some news on our RPM packages initiative.
>>>> 
>>>> 1. I’ve finished preliminary developing of Stage II version of RPM
>>>> packages [1]. Main “new feature” is — split design. Also I’ve added
>>>> package.sh script for automating package building process which will
>> help
>>>> organise corresponding builds in TC as well as simplify process for
>>>> developers who wishes to have custom packages.
>>>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>>>> Ignite 2.5 release, I’d greatly appreciate your help in finding
>> reviewer.
>>>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
>>>> repositories on Apache Bintray. Though they are already prepared for
>>>> hosting RPM and DEB packages respectively, and there is a way of linking
>>>> them to apache.org/dist/ignite page, there is possible alternative in
>>>> storing there only plain directory layout corresponding to each
>> repository
>>>> type (RPM and DEB) and manage this layout (repodata, distributions,
>>>> versions, etc.) by ourselves, having more control over repositories but
>>>> lacking some simplicity of deploying new releases. WDYT? Should we try
>>>> Cassandra approach? They are storing their DEB packages as I described
>>>> above [5].
>>>> 
>>>> Also — a question arose while I was working on this issue: which OSes
>> (and
>>>> which versions of each) are we going to support (if we are going) in
>> terms
>>>> of step-by-step list? Currently RPM packages are tested only with latest
>>>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
>>>> distributives [6] some of which are more o less popular among OS
>> community
>>>> (ALT, Fedora, openSUSE, etc.).
>>>> 
>>>> 
>>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>>>> [2] https://github.com/apache/ignite/pull/3703
>>>> [3] https://bintray.com/apache/ignite-rpm
>>>> [4] https://bintray.com/apache/ignite-deb
>>>> [5] https://bintray.com/apache/cassandra/debian#files/
>>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
>> distributions
>>>> 
>>>> 
>>>> 
>>>> 
>>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr.weider@gmail.com> wrote:
>>>>> 
>>>>> I suppose that most everything if not all from libs/options will go to
>>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>>>>> More precise lib selection (if something from optional would better to
>>>> have in core package) will be discussed right after preliminary split
>>>> architecture agreement.
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dpavlov.spb@gmail.com>
>> wrote:
>>>>>> 
>>>>>> I like idea of keeping simple system of modules, so +1 from me.
>>>>>> 
>>>>>> Where optional libs (e.g Direct IO plugin) would be included, would
it
>>>> be
>>>>>> core or optional?
>>>>>> 
>>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dmagda@apache.org>:
>>>>>> 
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> How big would be a final core module?
>>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and
create
>> it’s
>>>> own
>>>>>>>> package.
>>>>>>> 
>>>>>>> 
>>>>>>> Guys, 30 vs 280M is a huuuuge difference.  I would agree with
Petr
>> and
>>>>>>> propose the simplest modular system:
>>>>>>> 
>>>>>>> - core module that includes basic Ignite capabilities including
SQL,
>>>>>>> compute grid, service grid, k/v
>>>>>>> - optional module hosts the rest - ML, streamers integration
(kafka,
>>>>>>> flink), kubernetes, etc.
>>>>>>> 
>>>>>>> What do you think?
>>>>>>> 
>>>>>>> --
>>>>>>> Denis
>>>>>>> 
>>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr.weider@gmail.com>
>>>> wrote:
>>>>>>> 
>>>>>>>> *DEB package
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr.weider@gmail.com>
wrote:
>>>>>>>>> 
>>>>>>>>> Considering that DEV package for now is almost platform
independent
>>>>>>> (its
>>>>>>>> a java application more or less), that package will work
almost on
>> any
>>>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian,
etc.
>>>>>>>>> The only restriction is existence of systemctl (systemd)
service
>>>>>>> manager
>>>>>>>> — we are dependent on it.
>>>>>>>>> 
>>>>>>>>> Thats why, for instance, our RPM repository is called
simply ‘rpm’
>>>> and
>>>>>>>> package has no arch or dist suffix — it will work on CentOS,
RHEL,
>>>>>>> Fedora,
>>>>>>>> etc. with presence of aforementioned systemd.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Will Debian package work for Ubuntu?
>>>>>>>>>> 
>>>>>>>>>> D.
>>>>>>>>>> 
>>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr.weider@gmail.com
>>> 
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Not a problem, rather nuisance. Also, when we
will move to
>> official
>>>>>>>>>>> repositories, there can be a problem from OS
community.
>>>>>>>>>>> 
>>>>>>>>>>> Concerning DEB packages — I plan to use RPM
as base for DEB
>> package
>>>>>>>> build
>>>>>>>>>>> (package layout / install scripts) for speeding
up things and
>>>>>>> excluding
>>>>>>>>>>> possible duplication and desynchronisation, so
its a matter of
>> ’sit
>>>>>>>> and do’
>>>>>>>>>>> rather then some technical research. Thats why
I rose discussion
>>>>>>> about
>>>>>>>>>>> future package architecture, so that after agreement
I'm be able
>> to
>>>>>>>> pack
>>>>>>>>>>> both RPM and DEB identically.
>>>>>>>>>>> 
>>>>>>>>>>> Yet, if you insist, I can create DEB package
according to current
>>>> RPM
>>>>>>>>>>> layout in no time.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan
<
>>>> dsetrakyan@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Peter,
>>>>>>>>>>>> 
>>>>>>>>>>>> I don't think the package size of 280M is
going to be a problem
>> at
>>>>>>>> all,
>>>>>>>>>>> but
>>>>>>>>>>>> what you are suggesting can be an improvement
down the road.
>>>>>>>>>>>> 
>>>>>>>>>>>> In the mean time, I think our top priority
should be to provide
>>>>>>>> packages
>>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is
not nearly enough.
>>>>>>>>>>>> 
>>>>>>>>>>>> Agree?
>>>>>>>>>>>> 
>>>>>>>>>>>> D.
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider
<mr.weider@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Release 2.4 is almost there, at least
binary part of it, so I'd
>>>>>>> like
>>>>>>>> to
>>>>>>>>>>>>> move
>>>>>>>>>>>>> forward to further improve and widen
AI delivery through
>>>> packages.
>>>>>>>>>>>>> As of now, Apache Ignite ships in RPM
package weighing about
>>>> 280M+
>>>>>>>> and,
>>>>>>>>>>> to
>>>>>>>>>>>>> improve usability and significantly reduce
required download
>>>>>>> sizes, I
>>>>>>>>>>>>> purpose that in 2.5 release we introduce
splitted delivery as
>>>>>>>> follows:
>>>>>>>>>>>>> - CORE
>>>>>>>>>>>>> - bin
>>>>>>>>>>>>> - config
>>>>>>>>>>>>> - libs (!optional)
>>>>>>>>>>>>> - OPTIONAL LIBS
>>>>>>>>>>>>> - BENCHMARKS
>>>>>>>>>>>>> - DOCS (?)
>>>>>>>>>>>>> - EXAMPLES
>>>>>>>>>>>>> - .NET PLATFORM FILES
>>>>>>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>>>>>> 
>>>>>>>>>>>>> This architecture, as I assume, will
add flexibility (no reason
>>>> to
>>>>>>>>>>> download
>>>>>>>>>>>>> all 280M+ of binaries where you are to
run only core node
>>>>>>>> functionality)
>>>>>>>>>>>>> and
>>>>>>>>>>>>> maintainability (you are in full control
of what is installed
>> on
>>>>>>> your
>>>>>>>>>>>>> system).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> After successful architecture choice,
same scheme are planned
>> to
>>>> be
>>>>>>>>>>> used in
>>>>>>>>>>>>> DEB packages as well.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>>>> com/
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>>> 
>>>> 
>> 
>> 


Mime
View raw message