lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adrien Grand (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-4322) Can we make oal.util.packed.BulkOperation* smaller?
Date Fri, 31 Aug 2012 13:35:07 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Adrien Grand updated LUCENE-4322:
---------------------------------

    Attachment: LUCENE-4322-2.patch

New iteration.

This patch makes the JAR 84kB smaller (now 2243513 bytes) by :
 1.removing the specializations for BulkOperationPackedSingleBlock,
 2. rolling back some loops,
 3. removing the assertions on the buffer sizes (that were rather useless).

I removed the specializations for BulkOperationPackedSingleBlock because the default version
is already rather fast (it has no conditions). For example, for bitsPerValue=4, BulkOperationPackedSingleBlock
is only ~20-30% slower than BulkOperationPacked4 and BulkOperationPackedSingleBlock4, while
BulkOperationPacked is ~300% slower.

Moreover I rolled back some loops (especially the decode(byte[],...) are now 8 times shorter).

I ran a benchmark between this patch and current trunk to make sure these changes don't hurt
the Block postings format performance ('local' means patch applied):
{noformat}
                Task   QPS trunkStdDev trunk   QPS localStdDev local      Pct diff
         LowSpanNear       28.22        0.81       27.11        1.64  -12% -    4%
         MedSpanNear        8.86        0.30        8.63        0.47  -11% -    6%
        HighSpanNear        7.59        0.24        7.42        0.42  -10% -    6%
             MedTerm      965.02       26.52      947.64       22.67   -6% -    3%
            HighTerm      265.84        9.09      262.46        5.01   -6% -    4%
          HighPhrase        9.43        1.17        9.41        1.23  -22% -   28%
            PKLookup      249.61       10.58      250.46        8.97   -7% -    8%
          OrHighHigh       58.76        2.22       59.09        3.13   -8% -   10%
           LowPhrase       76.16        2.73       76.82        3.24   -6% -    9%
           OrHighLow      140.83        5.06      142.33        7.41   -7% -   10%
           MedPhrase       60.21        2.81       60.92        3.28   -8% -   11%
           OrHighMed      136.16        4.81      138.12        7.21   -7% -   10%
     LowSloppyPhrase       81.25        2.17       82.80        2.82   -4% -    8%
             Respell       91.72        4.07       93.47        2.21   -4% -    9%
             Prefix3      251.63        8.73      256.48        6.61   -4% -    8%
         AndHighHigh      136.17        2.20      138.95        2.85   -1% -    5%
    HighSloppyPhrase       14.17        0.66       14.46        0.74   -7% -   12%
            Wildcard      185.86        5.38      190.23        3.38   -2% -    7%
              IntNRQ       63.10        6.93       64.65        2.58  -11% -   19%
     MedSloppyPhrase       25.48        1.05       26.27        1.17   -5% -   12%
              Fuzzy1      106.76        4.83      110.17        2.44   -3% -   10%
          AndHighLow     2650.35       53.66     2739.95       58.80    0% -    7%
              Fuzzy2       35.45        1.95       36.75        0.99   -4% -   12%
          AndHighMed      357.48        5.31      370.88        6.77    0% -    7%
             LowTerm     2211.45       89.59     2298.23      104.62   -4% -   13%
{noformat}

This looks good to me.

I think there are maybe a few things we could still do to reduce the JAR size but I'm worried
that there would be little gain compared to the increased complexity of these classes.

I just tried to remove all BulkOperation* specializations, and it only made the JAR 52kb smaller
compared to this patch, so maybe we should stop here? There's already been a lot of progress!
                
> Can we make oal.util.packed.BulkOperation* smaller?
> ---------------------------------------------------
>
>                 Key: LUCENE-4322
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4322
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Michael McCandless
>             Fix For: 5.0, 4.0
>
>         Attachments: LUCENE-4322-2.patch, LUCENE-4322.patch
>
>
> These source files add up to a lot of sources ... it caused problems when compiling under
Maven and InteliJ.
> I committed a change to make separates files, but in aggregate this is still a lot ...
> EG maybe we don't need to specialize encode?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message