incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BLUR-290) NRT Updates using RAMDirectory & Swap
Date Wed, 08 Jan 2014 12:51:51 GMT

    [ https://issues.apache.org/jira/browse/BLUR-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13865380#comment-13865380
] 

Aaron McCurry commented on BLUR-290:
------------------------------------

You make some good points.

1. In the TransactionRecorder there was a sync around the commit call and yes that plus the
sync around the WAL and for whatever reason the various syncs in the NRT code were playing
havoc with one another.

2.  Well the slow part of the commit is the internal sync that is has plus the sync call to
all the files on disk.  That sync to all the files on disk is what I have made to be almost
nothing.

3. This is a great idea, if you extend the BlurIndex class and set the table property of "blur.shard.blurindex.class"
to equal your class you can try it out.  :-)

The basic reason I have done this is to make a rock solid implementation for the NRT piece
of Blur.  We can change and improve from here, we just needed a stable version to work from.

Aaron

> NRT Updates using RAMDirectory & Swap
> -------------------------------------
>
>                 Key: BLUR-290
>                 URL: https://issues.apache.org/jira/browse/BLUR-290
>             Project: Apache Blur
>          Issue Type: New Feature
>    Affects Versions: experimental-dev
>            Reporter: Ravikumar
>         Attachments: BlurFieldsConsumer.java, BlurFieldsConsumer.java, BlurFieldsConsumer.java,
BlurFlushingIndexWriter.java, BlurIndexTracker.java, BlurPostingsConsumer.java, BlurPostingsConsumer.java,
BlurPostingsFormat.java, BlurPostingsFormat.java, BlurRealTimeIndex.java, BlurRealTimeIndex.java,
BlurRealTimeIndexTest.java, BlurRealTimeIndexWriter.java, BlurRealTimeManager.java, BlurRealTimeManagerReopenThread.java,
BlurRowCodec.java, BlurRowCodec.java, BlurSegmentInfoFormat.java, BlurSegmentInfoWriter.java,
BlurTermsConsumer.java, BlurTermsConsumer.java, CompressingRowIndexReader.java, CompressingRowIndexWriter.java,
CompressingRowReader.java, CompressingRowReader.java, CompressingRowReader.java, CompressingRowWriter.java,
CompressingRowWriter.java, CompressingRowWriter.java, GrowableByteArrayDataOutput.java, PrimeDocCache.java,
RealTimeTransactionRecorder.java, RealTimeTransactionRecorder.java, RowCache.java, RowDocsCollector.java,
RowDocsCollector.java, RowReaderCache.java, RowReaderCache.java, SlabAllocator.java, SlabRAMDirectory.java,
SlabRAMFile.java, SlabRAMInputStream.java, SlabRAMOutputStream.java, SortingMultiReader.java,
SortingMultiReader.java, TestCompressingRowWriter.java, TestCompressingRowWriter.java
>
>
> We have been discussing about handling humungous rows in Blur (BLUR-220). Explore the
idea of using RAMDirectory at the front, backed by persistent-index.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message