incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <>
Subject [jira] [Created] (BLUR-322) Add a transactional data update call in thrift
Date Sun, 22 Dec 2013 01:11:50 GMT
Aaron McCurry created BLUR-322:

             Summary: Add a transactional data update call in thrift
                 Key: BLUR-322
             Project: Apache Blur
          Issue Type: Improvement
          Components: Blur
    Affects Versions: 0.3.0, 0.2.2
            Reporter: Aaron McCurry
             Fix For: 0.3.0, 0.2.2

After spending the last few weeks using the most recent version of Blur off the apache-blur-0.2
branch I have found a lot of issues surrounding the WAL, the NRT Lucene implementation and
the IndexImporter.  The basic problems seem to be locking issues between these various components.

There is now a simpler version of the BlurIndex (BlurIndexSimpleWriter) that outperforms the
current default implementation in a lot of ways.  There are no blocking issues and there are
no index importing issues from MapReduce.

However the BlurIndexSimpleWriter is missing a WAL and it does not perform well at NRT updates.

So this issue to create new data ingestion methods in the Thrift API to allow for a more bulk
like updates.

The basic methods should be something like:

string startTransaction()
void replaceRow(Row)
void replaceRows(list<Row>)
void delete Row(string id)
void rollbackTransaction(string)
void commitTransaction(string)

This message was sent by Atlassian JIRA

View raw message