lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (LUCENE-7966) build mr-jar and use some java 9 methods if available
Date Tue, 02 Jan 2018 18:47:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308486#comment-16308486
] 

Uwe Schindler edited comment on LUCENE-7966 at 1/2/18 6:46 PM:
---------------------------------------------------------------

Sorry for silence on this. I wanted to improve the patch tomorrow.

[~mikemccand]:

bq. I retested w/ Uwe's patch:

By reading the Java mailing lists, I have an idea what could cause Mike's differences: Are
you sure that you used the same garbage collector in Java 8 and Java 9? By default Java 9
uses G1GC, which Java 8 uses the good old ParallelGC by default. On the mailinglist jdk-dev
there was a discussion that suddenly computing-intensive stuff was significantly slower with
Java 9. The reason was the Garbage Collector! So be sure to use the same one (give it explicit
on command line). G1GC adds additional barriers and because of them the number of free CPU
registers is lowered by 1. This causes some algorithms to behave worse as missing CPU registers
don't allow to do everything in the CPU ([~dweiss] has seen a sigicficant slowdown for some
of his code). More than 10% slowdown is possible! This also affects code that has no garbage
collection and barriers because of G1. It's just limited resources causing this. Interestingly
the person complaining was talking about LZ4 compression: http://openjdk.5641.n7.nabble.com/Reduced-performance-in-Java-9-0-1-vs-8u152-td322825.html

Could you compare Java 8 and Java 9 with same garbage collector? (a) G1GC on both and (b)
ParallelGC and/or CMS ?
Uwe


was (Author: thetaphi):
Sorry for silence on this. I wanted to improve the patch tomorrow.

[~mikemccand]:

bq. I retested w/ Uwe's patch:

By reading the Java mailing lists, I have an idea what could cause Mike's differences: Are
you sure that you used the same garbage collector in Java 8 and Java 9? By default Java 9
uses G1GC, which Java 8 uses the good old ParallelGC by default. On the mailinglist jdk-dev
there was a discussion that suddenly computing-intensive stuff was significantly slower with
Java 9. The reason was the Garbage Collector! So be sure to use the same one (give it explicit
on command line). G1GC adds additional barriers and because of them the number of free CPU
registers is lowered by 1. This causes some algorithms to behave worse as missing CPU registers
don't allow to do everything in the CPU ([~dweiss] has seen a sigicficant slowdown for some
of his code). More than 10% slowdown is possible! This also affects code that has no garbage
collection and barriers because of G1. It's just limited resources causing this.

Could you compare Java 8 and Java 9 with same garbage collector? (a) G1GC on both and (b)
ParallelGC and/or CMS ?
Uwe

> build mr-jar and use some java 9 methods if available
> -----------------------------------------------------
>
>                 Key: LUCENE-7966
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7966
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/other, general/build
>            Reporter: Robert Muir
>              Labels: Java9
>         Attachments: LUCENE-7966.patch, LUCENE-7966.patch, LUCENE-7966.patch, LUCENE-7966.patch,
LUCENE-7966.patch
>
>
> See background: http://openjdk.java.net/jeps/238
> It would be nice to use some of the newer array methods and range checking methods in
java 9 for example, without waiting for lucene 10 or something. If we build an MR-jar, we
can start migrating our code to use java 9 methods right now, it will use optimized methods
from java 9 when thats available, otherwise fall back to java 8 code.  
> This patch adds:
> {code}
> Objects.checkIndex(int,int)
> Objects.checkFromToIndex(int,int,int)
> Objects.checkFromIndexSize(int,int,int)
> Arrays.mismatch(byte[],int,int,byte[],int,int)
> Arrays.compareUnsigned(byte[],int,int,byte[],int,int)
> Arrays.equal(byte[],int,int,byte[],int,int)
> // did not add char/int/long/short/etc but of course its possible if needed
> {code}
> It sets these up in {{org.apache.lucene.future}} as 1-1 mappings to java methods. This
way, we can simply directly replace call sites with java 9 methods when java 9 is a minimum.
Simple 1-1 mappings mean also that we only have to worry about testing that our java 8 fallback
methods work.
> I found that many of the current byte array methods today are willy-nilly and very lenient
for example, passing invalid offsets at times and relying on compare methods not throwing
exceptions, etc. I fixed all the instances in core/codecs but have not looked at the problems
with AnalyzingSuggester. Also SimpleText still uses a silly method in ArrayUtil in similar
crazy way, have not removed that one yet.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message