lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <>
Subject [jira] [Updated] (SOLR-2700) transaction logging
Date Sat, 06 Aug 2011 16:16:32 GMT


Yonik Seeley updated SOLR-2700:

    Attachment: SOLR-2700.patch

Here's a draft patch.
There is a tlog.<number> file created for each commit.  The javabin format is used to
serialize SolrInputDocuments.
An in-memory map of pointers into the log is kept for documents not yet soft-committed, and
the realtime-get component checks that first before using SolrCore.getNewestSearcher().

Seems to work for getting documents not in the newest searcher so far.

Tons of stuff left to do
- the tlog files are currently in the CWD
- need to handle deletes
- need to handle flushes in a performant way
- need to implement optional fsync for durability on power-failure
- would be nice to make some of this multi-threaded for better performance
- need to implement durability (apply updates from logs on startup)
- need to implement some form of cleanup for transaction logs

> transaction logging
> -------------------
>                 Key: SOLR-2700
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>         Attachments: SOLR-2700.patch
> A transaction log is needed for durability of updates, for a more performant realtime-get,
and for replaying updates to recovering peers.

This message is automatically generated by JIRA.
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message