incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravikumar (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BLUR-290) NRT Updates using RAMDirectory & Swap
Date Tue, 19 Nov 2013 11:49:22 GMT

     [ https://issues.apache.org/jira/browse/BLUR-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ravikumar updated BLUR-290:
---------------------------

    Attachment: BlurRealTimeIndexTest.java
                RowReaderCache.java
                RowDocsCollector.java
                RealTimeTransactionRecorder.java
                BlurRealTimeIndex.java
                TestCompressingRowWriter.java
                CompressingRowWriter.java
                CompressingRowReader.java
                BlurPostingsConsumer.java
                BlurFieldsConsumer.java
                BlurSegmentInfoFormat.java
                BlurSegmentInfoWriter.java
                BlurPostingsFormat.java
                BlurTermsConsumer.java
                BlurRowCodec.java

I was just trying a proof-of-concept stuff. There are many nocommits and loose ends, so haven't
really gone to the perf aspect. But if you can eval, will be great. I have fixed some bugs
and attached few test-cases, in case you want to try it.

+1 for introducing table-types. A less adventurous table-type can even write unsorted segments
to disk and take a hit during searches, while doing the balancing act during merge.

It would be great if we can detect that NRT re-opens are going to suffer under load and hence
adjust accordingly. Shouldn't it be a policy that is applicable to all table-types instead
of it being a table-type itself?

PS: Sorry for the huge number of files. Is there place to check-in, may be an experimental-branch?

> NRT Updates using RAMDirectory & Swap
> -------------------------------------
>
>                 Key: BLUR-290
>                 URL: https://issues.apache.org/jira/browse/BLUR-290
>             Project: Apache Blur
>          Issue Type: New Feature
>    Affects Versions: experimental-dev
>            Reporter: Ravikumar
>         Attachments: BlurFieldsConsumer.java, BlurFieldsConsumer.java, BlurFieldsConsumer.java,
BlurFlushingIndexWriter.java, BlurIndexTracker.java, BlurPostingsConsumer.java, BlurPostingsConsumer.java,
BlurPostingsFormat.java, BlurPostingsFormat.java, BlurRealTimeIndex.java, BlurRealTimeIndex.java,
BlurRealTimeIndexTest.java, BlurRealTimeIndexWriter.java, BlurRealTimeManager.java, BlurRealTimeManagerReopenThread.java,
BlurRowCodec.java, BlurRowCodec.java, BlurSegmentInfoFormat.java, BlurSegmentInfoWriter.java,
BlurTermsConsumer.java, BlurTermsConsumer.java, CompressingRowIndexReader.java, CompressingRowIndexWriter.java,
CompressingRowReader.java, CompressingRowReader.java, CompressingRowReader.java, CompressingRowWriter.java,
CompressingRowWriter.java, CompressingRowWriter.java, GrowableByteArrayDataOutput.java, PrimeDocCache.java,
RealTimeTransactionRecorder.java, RealTimeTransactionRecorder.java, RowCache.java, RowDocsCollector.java,
RowDocsCollector.java, RowReaderCache.java, RowReaderCache.java, SlabAllocator.java, SlabRAMDirectory.java,
SlabRAMFile.java, SlabRAMInputStream.java, SlabRAMOutputStream.java, SortingMultiReader.java,
SortingMultiReader.java, TestCompressingRowWriter.java, TestCompressingRowWriter.java
>
>
> We have been discussing about handling humungous rows in Blur (BLUR-220). Explore the
idea of using RAMDirectory at the front, backed by persistent-index.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message