lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renaud Delbru <ren...@siren.solutions>
Subject [ANN] SIREn, a Lucene/Solr plugin for rich JSON data search
Date Wed, 23 Jul 2014 11:13:24 GMT
One of the coolest features of Lucene/Solr is its ability to index 
nested documents using a Blockjoin approach.

While this works well for small documents and document collections, it 
becomes unsustainable for larger ones: Blockjoin works by splitting the 
original document in many documents, one per nested record.

For example, a single USPTO patent (XML format converted to JSON) will 
end up being over 1500 documents in the index. This has massive 
implications on performance and scalability.

Introducing SIREn

SIREn is an open source plugin for Solr for indexing and searching rich 
nested JSON data.

SIREn uses a sophisticated "tree indexing" design which ensures that the 
index is not artificially inflated. This ensures that querying on many 
types of nested queries can be up to 3x faster. Further, depending on 
the data, memory requirements for faceting can be up to 10x higher. As 
such, SIREn allows you to use Solr for larger and more complex datasets, 
especially so for sophisticated analytics. (You can read our whitepaper 
to find out more [1])

SIREn is also truly schemaless - it even allows you to change the type 
of a property between documents without being restricted by a defined 
mapping. This can be very useful for data integration scenarios where 
data is described in different ways in different sources.

You only need a few minutes to download and try SIREn [2]. It comes with 
a detailed manual [3] and you have access to the code on GitHub [4].

We look forward to hear about your feedbacks.

[1] 
http://siren.solutions/siren/resources/whitepapers/comparing-siren-1-2-and-lucenes-blockjoin-performance-a-uspto-patent-search-scenario/
[2] http://siren.solutions/siren/downloads/
[3] http://siren.solutions/manual/preface.html
[4] https://github.com/sindicetech/siren
-- 
Renaud Delbru
CTO
SIREn Solutions

Mime
View raw message