lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zheng Lin Edwin Yeo <edwinye...@gmail.com>
Subject Re: Does more shards in core improve performance?
Date Wed, 23 Sep 2015 15:33:17 GMT
I leave it to the default settings for now, which should be balancing 50-50
across both shards.

Regards,
Edwin

On 23 September 2015 at 22:49, Alessandro Benedetti <
benedetti.alex85@gmail.com> wrote:

> Using a second machines , you will dispose of fresh memory, disk and CPUs.
> So assuming you succeeded in saturating the first machine indexing power,
> of course it is normal that you improve your indexing time giving an
> additional node serving the indexing process.
> Are you balancing 50:50 or with some compositeId strategy(which anyway will
> try to balance if possible) ?
>
> Cheers
>
> 2015-09-23 15:33 GMT+01:00 Zheng Lin Edwin Yeo <edwinyeozl@gmail.com>:
>
> > I've tried to run different shards from different machine, and there is a
> > slight improvement in the performance (about 3 mins faster for 1GB worth
> of
> > data, from 22 mins to 19 mins).
> >
> > Is this a normal scenario? Both of my machine are running on Intel i7
> core.
> >
> >
> > Regards,
> > Edwin
> >
> >
> > On 21 September 2015 at 16:24, Zheng Lin Edwin Yeo <edwinyeozl@gmail.com
> >
> > wrote:
> >
> > > I'm not sure if that is because currently my machine is a normal PC and
> > > not a server, but my CPU specification for each of the core is Intel(R)
> > > Core(TM) i7-4910MQ CPU @ 2.90GHz.
> > >
> > > It should probably be better when the real server which has a much
> better
> > > specification comes, and I should be able to do the indexing in a
> lesser
> > > time using the knowledge that I've learnt here.
> > >
> > >
> > > Regards,
> > > Edwin
> > >
> > >
> > >
> > > On 21 September 2015 at 16:00, Toke Eskildsen <te@statsbiblioteket.dk>
> > > wrote:
> > >
> > >> On Mon, 2015-09-21 at 10:13 +0800, Zheng Lin Edwin Yeo wrote:
> > >> > I didn't find any increase in indexing throughput by adding shards
> in
> > >> the
> > >> > same machine.
> > >> >
> > >> > However, I've managed to feed the index to Solr from more than one
> > >> thread
> > >> > at a time. It can take up to 3 threads without affecting the
> indexing
> > >> > speed. Anything more than that, the CPU will hit 100%, and the
> > indexing
> > >> > speed in all the threads will be reduced.
> > >>
> > >> It is a bit surprising that the limit is 3 Threads on an 8 core
> machine,
> > >> but I am happy to hear that your findings fit the overall theory.
> > >>
> > >>
> > >> Thank you for the verification,
> > >> Toke Eskildsen, State and University Library, Denmark
> > >>
> > >>
> > >>
> > >
> >
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message