lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer
Date Fri, 12 Aug 2016 17:45:20 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419194#comment-15419194
] 

Uwe Schindler commented on LUCENE-7409:
---------------------------------------

I committed a {{@Ignore}} with explanation to the test, but I commented it out. I'd like to
run it a while on Jenkins, especially Multi-CPU machines like the Apache or Policeman one.
Multi-CPU machines are morke likely to crush.

> Look into making mmapdirectory's unmap safer
> --------------------------------------------
>
>                 Key: LUCENE-7409
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7409
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Robert Muir
>            Assignee: Uwe Schindler
>         Attachments: StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in application code
but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current weak map,
e.g. make it a safer option.
> I made an ugly prototype here: https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the luceneutil benchmark
(1M):
> {noformat}
> Report after iter 19:
>                     Task    QPS base      StdDev   QPS patch      StdDev            
   Pct diff
>                   IntNRQ      105.23     (17.6%)      100.42     (10.1%)   -4.6% ( -27%
-   28%)
>                  Respell      128.35     (13.2%)      125.88      (7.4%)   -1.9% ( -19%
-   21%)
>                   Fuzzy1      110.14     (17.2%)      108.28     (13.2%)   -1.7% ( -27%
-   34%)
>                LowPhrase      337.02     (13.0%)      333.72      (9.3%)   -1.0% ( -20%
-   24%)
>                MedPhrase      146.44     (12.9%)      145.55      (8.0%)   -0.6% ( -19%
-   23%)
>              MedSpanNear       96.85     (13.1%)       96.57      (7.8%)   -0.3% ( -18%
-   23%)
>             HighSpanNear       95.85     (13.9%)       96.33      (8.2%)    0.5% ( -18%
-   26%)
>               HighPhrase      146.84     (13.6%)      148.40      (8.4%)    1.1% ( -18%
-   26%)
>                 HighTerm      295.15     (15.8%)      298.77      (9.5%)    1.2% ( -20%
-   31%)
>              LowSpanNear      268.80     (12.4%)      272.16      (7.9%)    1.2% ( -16%
-   24%)
>                 Wildcard      284.09     (11.7%)      290.91      (8.9%)    2.4% ( -16%
-   25%)
>                  Prefix3      212.50     (15.4%)      217.76     (10.0%)    2.5% ( -19%
-   32%)
>                OrHighLow      358.65     (15.0%)      368.93     (10.7%)    2.9% ( -19%
-   33%)
>               AndHighMed      799.65     (13.2%)      834.74      (7.8%)    4.4% ( -14%
-   29%)
>          MedSloppyPhrase      229.36     (15.9%)      239.95      (9.8%)    4.6% ( -18%
-   36%)
>                   Fuzzy2       69.58     (14.6%)       72.82     (14.5%)    4.7% ( -21%
-   39%)
>              AndHighHigh      426.98     (12.8%)      451.77      (7.3%)    5.8% ( -12%
-   29%)
>                  MedTerm     1361.11     (14.5%)     1450.90      (9.2%)    6.6% ( -14%
-   35%)
>                 PKLookup      266.61     (13.4%)      284.28      (8.4%)    6.6% ( -13%
-   32%)
>         HighSloppyPhrase      251.22     (16.9%)      268.32     (10.7%)    6.8% ( -17%
-   41%)
>                OrHighMed      235.92     (17.2%)      253.12     (12.8%)    7.3% ( -19%
-   45%)
>               OrHighHigh      186.79     (13.5%)      201.15      (9.7%)    7.7% ( -13%
-   35%)
>          LowSloppyPhrase      395.23     (15.9%)      425.93      (9.3%)    7.8% ( -15%
-   39%)
>               AndHighLow     1128.28     (14.9%)     1242.11      (8.2%)   10.1% ( -11%
-   38%)
>                  LowTerm     3024.62     (12.9%)     3367.65      (9.7%)   11.3% (  -9%
-   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we only need handles
for getters and everything inlines correctly, rather than needing a ton for every getXYZ()
method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message