incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravikumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BLUR-290) NRT Updates using RAMDirectory & Swap
Date Tue, 26 Nov 2013 06:32:35 GMT

    [ https://issues.apache.org/jira/browse/BLUR-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13832356#comment-13832356
] 

Ravikumar commented on BLUR-290:
--------------------------------

Mirroring to github will be really useful. Please look at doing it, when time permits.

One hideous bug I caught in the code. Tried doing a transactional-commit between a read-only
IW and Disk-based IW. If something went wrong, I do a rollback on both.

IW.rollback() actually closes the IW. Talk about naming conventions & javadocs !!! Need
to look at alternative ways of addressing this issue.

There are also other things that "might" need to be customized, apart from BlurIndex.java

1. IndexManager.java
            -- Wishing to customize queries [like BlockJoin, Grouping etc...], fetching records
[DocValues instead of StoredFields etc...]

2. Thrift Operations
            -- Is it possible to abstract away basic operations in thrift, but allow customized
thrift-calls?
               Ex: Lets say an app wants to provide AutoSuggestion. It should simply define
an opCode, send it to shard-server.
                     Shard-Server examines the opCode [may be via a re-loadable prop-file??],
calls the app supplied class in pre-defined "lib"
                     folder, gets the "BlurResults" data-structure output and returns it client.
               Once we do this, making blur-contrib modules for different features of lucene
will become much more easier, isn't it?

     I don't know anything about thrift and totally new to it. I was thinking on lines of
hadoop RPC etc... and thought of asking you.


> NRT Updates using RAMDirectory & Swap
> -------------------------------------
>
>                 Key: BLUR-290
>                 URL: https://issues.apache.org/jira/browse/BLUR-290
>             Project: Apache Blur
>          Issue Type: New Feature
>    Affects Versions: experimental-dev
>            Reporter: Ravikumar
>         Attachments: BlurFieldsConsumer.java, BlurFieldsConsumer.java, BlurFieldsConsumer.java,
BlurFlushingIndexWriter.java, BlurIndexTracker.java, BlurPostingsConsumer.java, BlurPostingsConsumer.java,
BlurPostingsFormat.java, BlurPostingsFormat.java, BlurRealTimeIndex.java, BlurRealTimeIndex.java,
BlurRealTimeIndexTest.java, BlurRealTimeIndexWriter.java, BlurRealTimeManager.java, BlurRealTimeManagerReopenThread.java,
BlurRowCodec.java, BlurRowCodec.java, BlurSegmentInfoFormat.java, BlurSegmentInfoWriter.java,
BlurTermsConsumer.java, BlurTermsConsumer.java, CompressingRowIndexReader.java, CompressingRowIndexWriter.java,
CompressingRowReader.java, CompressingRowReader.java, CompressingRowReader.java, CompressingRowWriter.java,
CompressingRowWriter.java, CompressingRowWriter.java, GrowableByteArrayDataOutput.java, PrimeDocCache.java,
RealTimeTransactionRecorder.java, RealTimeTransactionRecorder.java, RowCache.java, RowDocsCollector.java,
RowDocsCollector.java, RowReaderCache.java, RowReaderCache.java, SlabAllocator.java, SlabRAMDirectory.java,
SlabRAMFile.java, SlabRAMInputStream.java, SlabRAMOutputStream.java, SortingMultiReader.java,
SortingMultiReader.java, TestCompressingRowWriter.java, TestCompressingRowWriter.java
>
>
> We have been discussing about handling humungous rows in Blur (BLUR-220). Explore the
idea of using RAMDirectory at the front, backed by persistent-index.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message