jackrabbit-oak-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chetan Mehrotra <chetan.mehro...@gmail.com>
Subject Re: Using DBCursor in MongoDocumentStore
Date Thu, 20 Mar 2014 09:18:15 GMT
So I would refactor the logic to use Iterables and the required
changes in MongoDocumentStore. For the RDB one I would just wrap
current impl. Should be able to push the change once Load 19 is cut.

Would that work for you Julian?
Chetan Mehrotra


On Thu, Mar 20, 2014 at 2:39 PM, Julian Reschke <julian.reschke@gmx.de> wrote:
> On 2014-03-20 10:02, Thomas Mueller wrote:
>>
>> Hi,
>>
>> For the database case, what we could do is return an iterator that
>> internally chunks, that is:
>>
>> 1) run the query "select * from datastore where id > ? limit 10000 order
>> by id"
>> 2) if the results set is empty, then iterator.hasNext is false
>> 3) else read this in memory and close the connection
>> 4) return the data from the in-memory buffer
>> 5) if the in-memory buffer is empty, start with 1) replacing ? with the
>> latest id
>>
>> This might also make sense for the MongoDB case. It's not very different
>> from what we do now, but the chunking logic would be on the DocumentStore
>> side, not on the NodeStore side. So chunking or not would be an
>> implementation detail of the DocumentStore.
>>
>> Regards,
>> Thomas
>
>
> SGTM.
>
> Best regards, Julian
>

Mime
View raw message