Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of noble.paul@gmail.com
 designates 74.125.78.27 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=jBSSMmRb/SFohWFVw3Q5RoAtJZtgPPN3NS3/CUvro2hoFlGbHEApdSJprt8u4n8EsP
         Y548pl3arJBc9sGV3XT5X3ZTT+GmPT+8NPcQcJAMHEqv2WdAEHoWlzZJ02NprlNHbhEk
         oBBaPQ2Sp35fBjO412vIg4UyMbfHeVRPbCg7I=
Message-ID: <5e76b0ad0811032026g437a2658pc622e8f2e8f3a2e7@mail.gmail.com>
Date: Tue, 4 Nov 2008 09:56:00 +0530
From: "=?UTF-8?B?Tm9ibGUgUGF1bCDgtKg=?=
 =?UTF-8?B?4LWL4LSs4LS/4LSz4LWN4oCNIOCkqOCli+CkrOCljeCks+CljQ==?="
 <noble.paul@gmail.com>
To: solr-user@lucene.apache.org
Subject: Re: XML vs mysql import with DataImportHandler
In-Reply-To: <20305117.post@talk.nabble.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <ae5f73f20806100541p1397e588h9804c07ebcea38de@mail.gmail.com>
	 <69de18140806100557l51dfd4a0uff1985ac4b831e81@mail.gmail.com>
	 <20305117.post@talk.nabble.com>

The attribute name is batchSize="-1" (it is case sensitive) . Tjis
ensures that Mysql driver fetcches row by row
http://wiki.apache.org/solr/DataImportHandlerFaq

On Mon, Nov 3, 2008 at 9:17 PM, sunnyfr <johanna.34@gmail.com> wrote:
>
> Hi Shalin,
> *
> I would like to know if you just used batchsize = -1.
> When I do that I use all Mysql's memory and it's a problem for the database
> and other process on it like update insert ...
> It will keep my database busy for 10hours, it's too much, is there a way to
> manage it differently ?
>
> Thanks a lot,
>
>
>
> Shalin Shekhar Mangar wrote:
>>
>> Hi Rusli,
>>
>> Without going into specifics, I can tell you that we're using
>> DataImportHandler successfully in production with many millions of
>> documents
>> being imported from databases. Try it out and let us know if you face any
>> problems. DataImportHandler will be part of Solr 1.3 (the next release)
>> and
>> may be committed quite soon after which you may not need to fiddle with
>> patches.
>>
>> Disclaimer: Since I'm one of the developers, I may be biased :)
>>
>> On Tue, Jun 10, 2008 at 6:11 PM, Rusli Ruslakall
>> <rusliruslakall@gmail.com>
>> wrote:
>>
>>> I wanted to see what people think about using DataImportHandler and
>>> importing directly from mysql rather than with XML.
>>>
>>> I have already programmed mysql -> XML in PHP and configured Solr to
>>> import it when I realized I could use DataImportHandler and skip the
>>> XML stage. Are there any negatives of using DataImportHandler? I would
>>> have to use the patch that complicates the process I guess when its
>>> kind of tempting to try to use Solr as much vanilla as possible.
>>>
>>> What is the general method when using database information?
>>>
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>>
>
> --
> View this message in context: http://www.nabble.com/XML-vs-mysql-import-with-DataImportHandler-tp17754471p20305117.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>


-- 
--Noble Paul