ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@gridgain.com>
Subject Re: Making Ignite shmem library optional
Date Thu, 27 Oct 2016 19:08:41 GMT
Simply because it makes Ignite Spark integration unreliable as well.

—
Denis

> On Oct 27, 2016, at 12:02 PM, Dmitriy Setrakyan <dsetrakyan@apache.org> wrote:
> 
> Why not make it mandatory only with "ignite-spark" dependency?
> 
> On Thu, Oct 27, 2016 at 11:53 AM, Denis Magda <dmagda@gridgain.com> wrote:
> 
>> Igniters,
>> 
>> Presently, if someone starts up a cluster and has at least two nodes
>> running on a single Unix machine then those nodes will be communicating
>> over the shared memory (shmem) by default.
>> 
>> This approach sounds absolutely reasonable for me but the shmem library is
>> not ideal at the moment. I observed many situations when a cluster got
>> stuck in the production or during long running tests due to some unclear
>> issues in shmem internals. Even from Ignite community side we have the
>> following shmem related issues
>> https://issues.apache.org/jira/browse/IGNITE-1578 <
>> https://issues.apache.org/jira/browse/IGNITE-1578>
>> https://issues.apache.org/jira/browse/IGNITE-1294 <
>> https://issues.apache.org/jira/browse/IGNITE-1294>
>> 
>> My proposal would be to make this library optional starting from the
>> nearest Apache Ignite release. As far as I recall, it’s still mandatory
>> because it's supposed that it speeds up our Spark and Hadoop integrations.
>> But I would avoid using something that can accelerate a solution and fail
>> at some unpredictable time.
>> 
>> Any thoughts or controversial arguments?
>> 
>> —
>> Denis
>> 
>> 


Mime
View raw message