ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kamil Misuth <ki...@ethome.sk>
Subject Re: Ignite on FreeBSD 11 and OpenJDK
Date Tue, 18 Apr 2017 22:47:45 GMT
Replying, to my own question.

Indeed, it seems to be the case that for Ignite 2.0, everything will be 
off heap. Denis Magda just (23 hours ago) updated the relevant wiki 
page.

https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=67638199&selectedPageVersions=10&selectedPageVersions=8

Sounds cool to me :).

Kamil

On 2017-04-19 00:43, Kamil Misuth wrote:
> Sure, Andrey.
> 
> In mean time, a fact-checking question:
> 
> I haven't really gone through the new memory policy functionality in
> depth, but it does seem that for Ignite 2.0 your aim is to move data
> off heap implicitly for all caches (and possibly backup the off heap
> region by memory mapped file(s)). Is this what you meant by "many
> memory related fixes"?
> 
> Thanks.
> 
> Kamil
> 
> On 2017-04-13 13:08, Andrey Gura wrote:
>> Kamil,
>> 
>> thanks a lot for your help. As I told before I need some time for
>> problem analyse. After it I'll create JIRA ticket and share link to it
>> in this thread.
>> 
>> Thank you again!
>> 
>> On Wed, Apr 12, 2017 at 11:56 PM, Kamil Misuth <kimec@ethome.sk> 
>> wrote:
>>> Hi Andrey,
>>> 
>>> I've built ignite-3477-master (commit hash 5839f481b7) today.
>>> Apart from the fact that some other Configuration APIs changed,
>>> CacheMemoryMode disapeared. I guess this is related to
>>> http://apache-ignite-developers.2346864.n4.nabble.com/IGNITE-4758-introducing-cache-memory-policies-td14957.html
>>> 
>>> Anyway, I've got the following exception on the first try, since
>>> ignite-indexing 2.0.0 is compiled against H2 1.4.191 and Spring Boot
>>> 1.5.2.RELEASE managed H2 version is 1.4.193 and apparently 
>>> org.h2.result.Row
>>> changed from abstract class to an interface between the two versions.
>>> 
>>> Caused by: java.lang.IncompatibleClassChangeError: class
>>> org.apache.ignite.internal.processors.query.h2.opt.GridH2Row has 
>>> interface
>>> org.h2.result.Row as super class
>>> 
>>> After fixing the dependency issue, I've tried to create two node 
>>> cluster and
>>> JVM SIGSEGVed in DirectNioClientWorker just as before.
>>> 
>>> Attaching crash logs from both nodes (running on the same machine 
>>> FreeBSD
>>> 11) to this e-mail.
>>> 
>>> Kamil
>>> 
>>> 
>>> On 2017-04-12 00:49, Kamil Misuth wrote:
>>>> 
>>>> Sure thing.
>>>> 
>>>> I will check out ignite-3477-master as soon as I have some time 
>>>> tomorrow.
>>>> 
>>>> Kamil
>>>> 
>>>> 
>>>> On 04/11/2017 05:48 PM, Andrey Gura wrote:
>>>>> 
>>>>> Thanks for provided information. I need additional time for problem
>>>>> investigation.
>>>>> 
>>>>> You can also try code from ignite-3477-master branch. This branch
>>>>> contains many memory related fixes but it isn't stable yet.
>>>>> 
>>>>> On Mon, Apr 10, 2017 at 11:37 PM, kimec.ethome.sk <kimec@ethome.sk>
>>>>> wrote:
>>>>>> 
>>>>>> Hi Andrey,
>>>>>> 
>>>>>> sorry, I've got ahead of my self.
>>>>>> 
>>>>>> I am on FreeBSD 11.0-RELEASE-p1 amd64
>>>>>> With OpenJDK Runtime Environment 1.8.0_121-b13 Oracle Corporation
>>>>>> OpenJDK
>>>>>> 64-Bit Server VM 25.121-b13
>>>>>> hw.model: Intel(R) Core(TM) i7-4702MQ CPU @ 2.20GHz
>>>>>> hw.machine_arch: amd64
>>>>>> hw.ncpu: 8
>>>>>> hw.physmem: 8251813888
>>>>>> 
>>>>>> Core dump is 1 GB, so I guess that is no go. I am attaching crash

>>>>>> log to
>>>>>> this e-mail.
>>>>>> I have uploaded the project I've used during my testing here
>>>>>> https://github.com/kimec/ignite-spring-boot .
>>>>>> The sample works perfectly well with stock ignite-core on Linux 
>>>>>> OpenJDK
>>>>>> 8
>>>>>> xs64 CentOS 7 .
>>>>>> 
>>>>>> Kamil
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 2017-04-10 12:24, Andrey Gura wrote:
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> could you please share core dump file? If not, it would be 
>>>>>>> helpful to
>>>>>>> know what is CPU architecture on this server.
>>>>>>> 
>>>>>>> On Mon, Apr 10, 2017 at 2:53 AM, Kamil Misuth <kimec@ethome.sk>

>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Greetings,
>>>>>>>> 
>>>>>>>> OpenJDK (7 and 8) HotSpot JVM SIGSEGVs on FreeBSD 11 as soon
as 
>>>>>>>> node
>>>>>>>> joins a
>>>>>>>> topology and starts to communicate via DirectNioClientWorker.
>>>>>>>> The root cause is DirectByteBufferStreamImpl (both versions)

>>>>>>>> which
>>>>>>>> uses
>>>>>>>> GridUnsafe.getXXX/putXXX(Object object, offset, value) methods

>>>>>>>> to
>>>>>>>> manipulate
>>>>>>>> DirectByteBuffer, whereas it should really be using
>>>>>>>> GridUnsafe.getXXX/putXXX(address, value), since DirectByteBuffer

>>>>>>>> is
>>>>>>>> allocated on C heap (off java heap).
>>>>>>>> Notice that at least one instance of the same problem is
known 
>>>>>>>> to
>>>>>>>> exist
>>>>>>>> in
>>>>>>>> another project using Unsafe
>>>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-8325 .
>>>>>>>> The OpenJDK source of Unsafe is more or less clear on this
>>>>>>>> 
>>>>>>>> 
>>>>>>>> http://hg.openjdk.java.net/jdk8u/jdk8u60/jdk/file/935758609767/src/share/classes/sun/misc/Unsafe.java#l391
>>>>>>>> I have prepared a simple fix here
>>>>>>>> 
>>>>>>>> 
>>>>>>>> https://github.com/apache/ignite/compare/1.9.0-rc2...kimec:freebsd-support

>>>>>>>> .
>>>>>>>> However, I am not sure if the solution is right in regard
to 
>>>>>>>> overall
>>>>>>>> ignite
>>>>>>>> performance.
>>>>>>>> I've tried to compile ignite-core with tests and after applying

>>>>>>>> my
>>>>>>>> changes
>>>>>>>> was able to pass all the basic stuff until the performance
test 
>>>>>>>> stage
>>>>>>>> at
>>>>>>>> which point my machine run out of RAM and swap space (some
10 
>>>>>>>> GB)...
>>>>>>>> Not
>>>>>>>> sure if this is how the tests are supposed to be. After 
>>>>>>>> compiling with
>>>>>>>> -DskipTests I was able to create FreeBSD 11 - CentOS 7 two
node
>>>>>>>> cluster
>>>>>>>> and
>>>>>>>> everything seemed OK (the two nodes shared an IGFS instance

>>>>>>>> backed by
>>>>>>>> replicated caches).
>>>>>>>> Please note that OpenJDK on different systems as well as
Oracle 
>>>>>>>> JDK
>>>>>>>> (via
>>>>>>>> Linux compatility layer) on FreeBSD seem to be more forgiving

>>>>>>>> and does
>>>>>>>> not
>>>>>>>> SIGSEGV.
>>>>>>>> I've based my branch on 1.9.0-rc2 since tag 1.9.0 has already

>>>>>>>> POM with
>>>>>>>> version 2.0.
>>>>>>>> 
>>>>>>>> Kamil

Mime
View raw message