lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Khludnev <mkhlud...@griddynamics.com>
Subject Re: What is the best approach to send lots of XML Messages to Solr to build index?
Date Sun, 15 Jun 2014 10:14:15 GMT
Hello Floyd,

Did you consider to disable tlog?
Does a file consist of many docs?
Do you have SolrCloud? Do you use just sh/curl or have a java program?
DIH is not really performant so far. Submitting roughly ten huge files in
parallel is a way to perform good. Once again, nuke tlog.


On Sun, Jun 15, 2014 at 12:44 PM, Floyd Wu <floyd.wu@gmail.com> wrote:

> Hi,
> I have many XML Message file formatted like this
> https://wiki.apache.org/solr/UpdateXmlMessages
>
> These files are generated by my index builder daily.
> Currently I am sending these file through http post to Solr but sometimes I
> hit OOM exception or pending too many tlog.
>
> Do you have better way to "import" these files to Solr to build index?
>
> Thanks for the suggestion
>
> Floyd
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
 <mkhludnev@griddynamics.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message