Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of lists@nabble.com designates
 216.139.236.158 as permitted sender)
Message-ID: <20305117.post@talk.nabble.com>
Date: Mon, 3 Nov 2008 07:47:31 -0800 (PST)
From: sunnyfr <johanna.34@gmail.com>
To: solr-user@lucene.apache.org
Subject: Re: XML vs mysql import with DataImportHandler
In-Reply-To: <69de18140806100557l51dfd4a0uff1985ac4b831e81@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
References: <ae5f73f20806100541p1397e588h9804c07ebcea38de@mail.gmail.com>
 <69de18140806100557l51dfd4a0uff1985ac4b831e81@mail.gmail.com>


Hi Shalin,
*
I would like to know if you just used batchsize = -1.
When I do that I use all Mysql's memory and it's a problem for the database
and other process on it like update insert ... 
It will keep my database busy for 10hours, it's too much, is there a way to
manage it differently ? 

Thanks a lot,


Shalin Shekhar Mangar wrote:
> 
> Hi Rusli,
> 
> Without going into specifics, I can tell you that we're using
> DataImportHandler successfully in production with many millions of
> documents
> being imported from databases. Try it out and let us know if you face any
> problems. DataImportHandler will be part of Solr 1.3 (the next release)
> and
> may be committed quite soon after which you may not need to fiddle with
> patches.
> 
> Disclaimer: Since I'm one of the developers, I may be biased :)
> 
> On Tue, Jun 10, 2008 at 6:11 PM, Rusli Ruslakall
> <rusliruslakall@gmail.com>
> wrote:
> 
>> I wanted to see what people think about using DataImportHandler and
>> importing directly from mysql rather than with XML.
>>
>> I have already programmed mysql -> XML in PHP and configured Solr to
>> import it when I realized I could use DataImportHandler and skip the
>> XML stage. Are there any negatives of using DataImportHandler? I would
>> have to use the patch that complicates the process I guess when its
>> kind of tempting to try to use Solr as much vanilla as possible.
>>
>> What is the general method when using database information?
>>
> 
> 
> 
> -- 
> Regards,
> Shalin Shekhar Mangar.
> 
> 

-- 
View this message in context: http://www.nabble.com/XML-vs-mysql-import-with-DataImportHandler-tp17754471p20305117.html
Sent from the Solr - User mailing list archive at Nabble.com.