incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravikumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BLUR-290) NRT Updates using RAMDirectory & Swap
Date Tue, 03 Dec 2013 07:34:35 GMT

    [ https://issues.apache.org/jira/browse/BLUR-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13837424#comment-13837424
] 

Ravikumar commented on BLUR-290:
--------------------------------

I thinking through another idea.

Lucene has NRTCachingDirectory, which basically wraps a Disk-Directory as a delegate. All
incoming writes are written in RAM segments.

Reads just work normally. This is really useful for frequent NRT re-opens and will perform
very well under high write load, thereby automatically addressing Blur-295 issue also ["Long
pause during NRT re-opens]

We also don't need a big solution I previously proposed and instead all of it can be elegantly
handled with NRTCachingDirectory, in a single IndexWriter itself. 

Please let me know your views on this. If fine, I shall re-work the patch towards this approach.

> NRT Updates using RAMDirectory & Swap
> -------------------------------------
>
>                 Key: BLUR-290
>                 URL: https://issues.apache.org/jira/browse/BLUR-290
>             Project: Apache Blur
>          Issue Type: New Feature
>    Affects Versions: experimental-dev
>            Reporter: Ravikumar
>         Attachments: BlurFieldsConsumer.java, BlurFieldsConsumer.java, BlurFieldsConsumer.java,
BlurFlushingIndexWriter.java, BlurIndexTracker.java, BlurPostingsConsumer.java, BlurPostingsConsumer.java,
BlurPostingsFormat.java, BlurPostingsFormat.java, BlurRealTimeIndex.java, BlurRealTimeIndex.java,
BlurRealTimeIndexTest.java, BlurRealTimeIndexWriter.java, BlurRealTimeManager.java, BlurRealTimeManagerReopenThread.java,
BlurRowCodec.java, BlurRowCodec.java, BlurSegmentInfoFormat.java, BlurSegmentInfoWriter.java,
BlurTermsConsumer.java, BlurTermsConsumer.java, CompressingRowIndexReader.java, CompressingRowIndexWriter.java,
CompressingRowReader.java, CompressingRowReader.java, CompressingRowReader.java, CompressingRowWriter.java,
CompressingRowWriter.java, CompressingRowWriter.java, GrowableByteArrayDataOutput.java, PrimeDocCache.java,
RealTimeTransactionRecorder.java, RealTimeTransactionRecorder.java, RowCache.java, RowDocsCollector.java,
RowDocsCollector.java, RowReaderCache.java, RowReaderCache.java, SlabAllocator.java, SlabRAMDirectory.java,
SlabRAMFile.java, SlabRAMInputStream.java, SlabRAMOutputStream.java, SortingMultiReader.java,
SortingMultiReader.java, TestCompressingRowWriter.java, TestCompressingRowWriter.java
>
>
> We have been discussing about handling humungous rows in Blur (BLUR-220). Explore the
idea of using RAMDirectory at the front, backed by persistent-index.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message