lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Serba <ase...@gmail.com>
Subject Re: About Solr DataImportHandler
Date Tue, 07 Sep 2010 08:07:16 GMT
> i found that Solr import the datas to memory first,then write them to index dir.
That's not really true. DataImportHandler streams the result from
database query and adding documents into index. So it shouldn't load
all database data into memory. Disabling autoCommit, warming queries
and spellcheckers usually decreases required amount of memory during
indexing process.

Please share your hardware details, jvm options, solrconfig and schema
configuration, etc.



2010/9/7 郭芸 <mickey.guoyun@gmail.com>:
> Dear all:
> I use Solr DataImportHandler's JdbcDataSource to import the Sqlsever 2005's
> datas to Solr,but My table is versy big,about 300G.and i found that Solr
> import the datas to memory first,then write them to index dir.So if the
> datas are too big,there will trigger an OutOfMemoryException.
> I want to solve this problem,and how can ti do it?anybody can help me?Thank
> you.
>
> 2010-09-07
> ________________________________
> 郭芸

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message