cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: Lucandra Ingestion
Date Mon, 25 Jan 2010 22:01:34 GMT
I'm not super familiar with Lucandra but my guess is you want to use
one indexwriter per thread, there's not much point in using a shared,
thread-unsafe object across multiple clients and serializing via
synchronized.

On Mon, Jan 25, 2010 at 3:59 PM, ML_Seda <sonnyheer@gmail.com> wrote:
>
>
> Jonathan Ellis-3 wrote:
>>
>> Are you using multiple threads?
>>
>
> I'm adding in threading now, and getting exceptions at times regarding a
> "broken pipe".
>
> I then added the following :
>        synchronized (this) {
>                indexWriter.addDocument(doc, analyzer);
>        }
>
> Which did get rid of the problem.  I'm currently using Phasers (jsr166) to
> register threads per file found in a given directory.  Although it still
> seems slow.
>
> Has anyone else ingested large # of files, and found ways to optimize
> ingestion?  If I apply a patch for batch operations (from the link in the
> post), will this work with the version of cassandra supported by lucandra?
>
> Thanks again.
>
> --
> View this message in context: http://n2.nabble.com/Lucandra-Ingestion-tp4415691p4457044.html
> Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.
>

Mime
View raw message