lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Noble Paul നോബിള്‍ नोब्ळ्" <noble.p...@gmail.com>
Subject Re: Large Data Set Suggestions
Date Fri, 07 Nov 2008 04:38:40 GMT
Hi Lance,
This is one area we left open in DIH. What is the best way to handle
this. On error it should give up or continue with the next?



On Fri, Nov 7, 2008 at 12:44 AM, Lance Norskog <goksron@gmail.com> wrote:
> You can also do streaming XML upload for the XML-based indexing. This can
> feed, say, 100k records in one XML file from a separate machine.
>
> All of these options ignore the case where there is an error in your input
> records v.s. the schema.  DIH gives up on an error. Streaming XML gives up
> on an error.
>
> Lance
>
> -----Original Message-----
> From: Steven Anderson [mailto:sanderson@vsticorp.com]
> Sent: Thursday, November 06, 2008 5:57 AM
> To: solr-user@lucene.apache.org
> Subject: RE: Large Data Set Suggestions
>
>> In that case you may put the file in a mounted NFS directory or you
>> can serve it out with an apache server.
>
> That's one option although someone else on the list mentioned that
> performance was 10x slower in their NFS experience.
>
> Another option is to serve up the files via Apache and pull them via DIH
> HTTP.
>
> Thankfully, there are lots of options, but we need to determine which one
> will perform best.
>
> Thanks,
>
> A. Steven Anderson
> 410-418-9908 VSTI
> 443-790-4269 cell
>
>
>
>



-- 
--Noble Paul

Mime
View raw message