lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <j...@apache.org>
Subject [jira] Updated: (LUCENE-2886) Adaptive Frame Of Reference
Date Thu, 10 Feb 2011 12:20:57 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robert Muir updated LUCENE-2886:
--------------------------------

    Attachment: LUCENE-2886.patch

I spent some more time on Simple64, took Mike's previous patch and added some minor improvements:

# Switched the decoding logic to the "Simple-8-4b" referred to in the paper. This is the same
encoding, but we process with ints instead of longs.
# Because our buffers are so tiny (for example 32 bytes), the overhead of NIO hurts rather
than helps, so I switched to native arrays.

The performance is looking much more reasonable. Here's my tests on windows, maybe i can convince
Mike to sanity-check it on linux.

64-bit SimpleFS
||Query||QPS BulkVInt||QPS Simple64VarInt4||Pct diff||||
|"united states"~3|3.79|3.67|{color:red}-3.3%{color}|
|doctitle:.*[Uu]nited.*|2.45|2.46|{color:green}0.3%{color}|
|spanNear([unit, state], 10, true)|22.13|22.64|{color:green}2.3%{color}|
|uni*|14.04|14.43|{color:green}2.7%{color}|
|united~0.75|6.83|7.04|{color:green}3.2%{color}|
|unit*|25.39|26.21|{color:green}3.2%{color}|
|doctimesecnum:[10000 TO 60000]|8.83|9.16|{color:green}3.6%{color}|
|united~0.6|4.29|4.47|{color:green}4.2%{color}|
|united states|9.35|9.74|{color:green}4.2%{color}|
|un*d|12.88|13.50|{color:green}4.8%{color}|
|"united states"|6.86|7.21|{color:green}5.1%{color}|
|unit~0.7|14.11|14.85|{color:green}5.3%{color}|
|unit~0.5|8.17|8.60|{color:green}5.3%{color}|
|u*d|5.70|6.05|{color:green}6.1%{color}|
|states|30.02|31.90|{color:green}6.3%{color}|
|spanFirst(unit, 5)|86.56|94.15|{color:green}8.8%{color}|
|+united +states|11.10|12.55|{color:green}13.1%{color}|
|+nebraska +states|46.72|57.90|{color:green}23.9%{color}|

32-bit SimpleFS
||Query||QPS BulkVInt||QPS Simple64VarInt4||Pct diff||||
|spanFirst(unit, 5)|95.67|91.02|{color:red}-4.9%{color}|
|"united states"|5.47|5.25|{color:red}-4.1%{color}|
|"united states"~3|3.37|3.32|{color:red}-1.6%{color}|
|unit*|20.45|20.33|{color:red}-0.6%{color}|
|uni*|11.10|11.06|{color:red}-0.3%{color}|
|doctimesecnum:[10000 TO 60000]|7.15|7.16|{color:green}0.0%{color}|
|doctitle:.*[Uu]nited.*|2.26|2.27|{color:green}0.4%{color}|
|unit~0.5|7.73|7.77|{color:green}0.5%{color}|
|un*d|10.80|10.87|{color:green}0.6%{color}|
|united~0.75|6.77|6.97|{color:green}2.8%{color}|
|unit~0.7|12.97|13.41|{color:green}3.4%{color}|
|united~0.6|4.10|4.26|{color:green}3.7%{color}|
|u*d|4.91|5.10|{color:green}4.0%{color}|
|spanNear([unit, state], 10, true)|20.50|21.72|{color:green}5.9%{color}|
|states|30.00|33.15|{color:green}10.5%{color}|
|+united +states|9.71|10.78|{color:green}11.1%{color}|
|united states|9.65|10.96|{color:green}13.6%{color}|
|+nebraska +states|43.93|54.38|{color:green}23.8%{color}|

64-bit MMap
||Query||QPS BulkVInt||QPS Simple64VarInt4||Pct diff||||
|"united states"|8.99|8.41|{color:red}-6.4%{color}|
|states|38.21|36.16|{color:red}-5.4%{color}|
|spanFirst(unit, 5)|118.11|112.19|{color:red}-5.0%{color}|
|doctimesecnum:[10000 TO 60000]|10.78|10.35|{color:red}-4.0%{color}|
|spanNear([unit, state], 10, true)|33.78|32.51|{color:red}-3.7%{color}|
|"united states"~3|4.68|4.54|{color:red}-3.0%{color}|
|unit*|30.00|29.26|{color:red}-2.4%{color}|
|uni*|17.48|17.06|{color:red}-2.4%{color}|
|united states|11.60|11.35|{color:red}-2.1%{color}|
|+united +states|13.95|14.08|{color:green}1.0%{color}|
|united~0.75|10.76|10.87|{color:green}1.1%{color}|
|united~0.6|7.75|7.88|{color:green}1.7%{color}|
|un*d|17.16|17.66|{color:green}2.9%{color}|
|doctitle:.*[Uu]nited.*|3.85|3.98|{color:green}3.3%{color}|
|unit~0.7|27.00|28.08|{color:green}4.0%{color}|
|unit~0.5|16.64|17.46|{color:green}4.9%{color}|
|u*d|8.68|9.31|{color:green}7.2%{color}|
|+nebraska +states|83.30|96.53|{color:green}15.9%{color}|



> Adaptive Frame Of Reference 
> ----------------------------
>
>                 Key: LUCENE-2886
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2886
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Codecs
>            Reporter: Renaud Delbru
>             Fix For: 4.0
>
>         Attachments: LUCENE-2886.patch, LUCENE-2886.patch, LUCENE-2886.patch, LUCENE-2886_simple64.patch,
LUCENE-2886_simple64_varint.patch, lucene-afor.tar.gz
>
>
> We could test the implementation of the Adaptive Frame Of Reference [1] on the lucene-4.0
branch.
> I am providing the source code of its implementation. Some work needs to be done, as
this implementation is working on the old lucene-1458 branch. 
> I will attach a tarball containing a running version (with tests) of the AFOR implementation,
as well as the implementations of PFOR and of Simple64 (simple family codec working on 64bits
word) that has been used in the experiments in [1].
> [1] http://www.deri.ie/fileadmin/documents/deri-tr-afor.pdf

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message