lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Han Jiang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)
Date Wed, 20 Jun 2012 17:58:43 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397694#comment-13397694
] 

Han Jiang edited comment on LUCENE-3892 at 6/20/12 5:57 PM:
------------------------------------------------------------

OK, just reproduce your test. But Mike, are we using a same task file? Our relative speeds
for different queries are not the same. 
{noformat}
                Task    QPS Base StdDev Base     QPS For  StdDev For      Pct diff
              Phrase        5.07        0.45        3.76        0.19  -35% -  -14% (-44% -
 -18%)
          AndHighMed       28.32        2.34       22.67        0.67  -28% -  -10% (-38% -
  -9%)
            SpanNear        2.72        0.13        2.36        0.14  -22% -   -3% (-36% -
  -8%)
        SloppyPhrase        4.18        0.20        3.83        0.15  -16% -    0% (-33% -
  -6%)
             Respell       42.02        1.83       38.86        2.30  -16% -    2% (-18% -
   0%)
              Fuzzy1       44.96        1.58       42.85        1.69  -11% -    2% (-12% -
   0%)
              Fuzzy2       16.78        0.69       16.34        0.68  -10% -    5% (-12% -
   3%)
            PKLookup       89.11        2.15       87.33        2.19   -6% -    2% ( -2% -
   5%)
         AndHighHigh        7.61        0.44        7.69        0.21   -7% -   10% (-21% -
  10%)
            Wildcard       19.50        0.91       20.02        0.72   -5% -   11% (-21% -
   3%)
        TermBGroup1M       20.82        0.37       21.73        0.69    0% -    9% (  2% -
  10%)
         TermGroup1M       13.79        0.13       14.61        0.32    2% -    9% (  1% -
   9%)
              IntNRQ        4.11        0.56        4.56        0.56  -14% -   43% (-25% -
  33%)
      TermBGroup1M1P       21.45        0.75       24.00        0.51    5% -   18% ( -1% -
  22%)
           OrHighMed        5.08        0.49        5.73        0.15    0% -   28% (-16% -
  25%)
          OrHighHigh        4.22        0.39        4.78        0.13    1% -   28% (-15% -
  24%)
             Prefix3       30.91        1.63       35.65        2.02    3% -   28% (-14% -
  21%)
                Term       44.36        1.87       54.01        1.96   12% -   31% ( -1% -
  33%)
{noformat}
                
      was (Author: billy):
    OK, just reproduce your test. But Mike, are we using a same task file? Our relative speeds
for different queries are not the same. 
{quote}
                Task    QPS Base StdDev Base     QPS For  StdDev For      Pct diff
              Phrase        5.07        0.45        3.76        0.19  -35% -  -14% (-44% -
 -18%)
          AndHighMed       28.32        2.34       22.67        0.67  -28% -  -10% (-38% -
  -9%)
            SpanNear        2.72        0.13        2.36        0.14  -22% -   -3% (-36% -
  -8%)
        SloppyPhrase        4.18        0.20        3.83        0.15  -16% -    0% (-33% -
  -6%)
             Respell       42.02        1.83       38.86        2.30  -16% -    2% (-18% -
   0%)
              Fuzzy1       44.96        1.58       42.85        1.69  -11% -    2% (-12% -
   0%)
              Fuzzy2       16.78        0.69       16.34        0.68  -10% -    5% (-12% -
   3%)
            PKLookup       89.11        2.15       87.33        2.19   -6% -    2% ( -2% -
   5%)
         AndHighHigh        7.61        0.44        7.69        0.21   -7% -   10% (-21% -
  10%)
            Wildcard       19.50        0.91       20.02        0.72   -5% -   11% (-21% -
   3%)
        TermBGroup1M       20.82        0.37       21.73        0.69    0% -    9% (  2% -
  10%)
         TermGroup1M       13.79        0.13       14.61        0.32    2% -    9% (  1% -
   9%)
              IntNRQ        4.11        0.56        4.56        0.56  -14% -   43% (-25% -
  33%)
      TermBGroup1M1P       21.45        0.75       24.00        0.51    5% -   18% ( -1% -
  22%)
           OrHighMed        5.08        0.49        5.73        0.15    0% -   28% (-16% -
  25%)
          OrHighHigh        4.22        0.39        4.78        0.13    1% -   28% (-15% -
  24%)
             Prefix3       30.91        1.63       35.65        2.02    3% -   28% (-14% -
  21%)
                Term       44.36        1.87       54.01        1.96   12% -   31% ( -1% -
  33%)
{quote}
                  
> Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)
> -------------------------------------------------------------------------------------
>
>                 Key: LUCENE-3892
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3892
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>              Labels: gsoc2012, lucene-gsoc-12
>             Fix For: 4.1
>
>         Attachments: LUCENE-3892-direct-IntBuffer.patch, LUCENE-3892_for.patch, LUCENE-3892_for_unfold_method.patch,
LUCENE-3892_pfor.patch, LUCENE-3892_pfor.patch, LUCENE-3892_pfor.patch, LUCENE-3892_pfor_unfold_method.patch,
LUCENE-3892_settings.patch, LUCENE-3892_settings.patch
>
>
> On the flex branch we explored a number of possible intblock
> encodings, but for whatever reason never brought them to completion.
> There are still a number of issues opened with patches in different
> states.
> Initial results (based on prototype) were excellent (see
> http://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html
> ).
> I think this would make a good GSoC project.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message