lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Khludnev <mkhlud...@griddynamics.com>
Subject Re: Multi-threaded post.jar?
Date Mon, 04 Feb 2013 18:19:28 GMT
Hello,

Threads were cut off from the DIH at 4.0.
My proposal for server-side concurrency is
https://issues.apache.org/jira/browse/SOLR-3585

Regards


On Sun, Feb 3, 2013 at 9:42 PM, Upayavira <uv@odoko.co.uk> wrote:

> I haven't tried DIH, although if it does support multithreading, I might
> be inclined to.
>
> Upayavira
>
> On Sun, Feb 3, 2013, at 05:17 PM, Alexandre Rafalovitch wrote:
> > What times do you get with DIH? It has native support for that format
> > too.
> > On 3 Feb 2013 11:20, "Upayavira" <uv@odoko.co.uk> wrote:
> >
> > > I have a scenario in which I need to post 500,000 documents to Solr as
> a
> > > test. I have these documents in XML files already formatted in Solr's
> > > xml format.
> > >
> > > Posting to Solr using post.jar it takes 1m55s. With a bit of bash
> > > jiggery-pokery, I was able to get this down to 1m08s by running four
> > > concurrent post.jar instances, which strikes me as a significant
> > > improvement.
> > >
> > > I'm considering adding multithreaded capabilities to post.jar, but
> > > before I go to that effort, I wanted to see if anyone else would
> > > consider it a useful feature. Given that the SimplePostTool is becoming
> > > far from simple, I wanted to see whether the feature is likely to be
> > > accepted before I put in the effort. Also, I would need to consider
> > > which parts of the tool to add that to. Currently I only want it for
> > > posting XML docs, but there's also crawling capabilities in it too.
> > >
> > > Thoughts?
> > >
> > > Upayavira
> > >
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
 <mkhludnev@griddynamics.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message