lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koen De Groote <koen.degro...@limecraft.com>
Subject Re: Solr process takes several minutes before accepting commands after restart
Date Thu, 21 Nov 2019 20:21:02 GMT
Erick:

No suggesters. There is 1 spellchecker for

<str name="queryAnalyzerFieldType">text_general</str>

But no buildOnCommit or buildOnStartup setting mentioned anywhere.

That being said, the point in time at which this occurs, the database is
guaranteed to be empty, as the data folders had previously been deleted and
recreated empty. Then the docker container is restarted and this behavior
is observed.

Long shot, but even if Solr is getting data from zookeeper telling of file
locations and checking for the existence of these files... that should be
pretty fast, I'd think.

This is really disturbing. I know what to expect when recovering now, but
someone doing this on a live environment that has to be up again ASAP is
probably going to be sweating bullets.


On Thu, Nov 21, 2019 at 2:45 PM Erick Erickson <erickerickson@gmail.com>
wrote:

> Koen:
>
> Do you have any spellcheckers or suggesters defined with buildOnCommit or
> buildOnStartup set to “true”? Depending on the implementation, this may
> have to read the stored data for the field used in the
> suggester/spellchecker from _every_ document in your collection, which can
> take many minutes. Even if your implementation in your config is file-based
> it can still take a while.
>
> Shot in the dark….
>
> Erick
>
> > On Nov 21, 2019, at 4:03 AM, Koen De Groote <koen.degroote@limecraft.com>
> wrote:
> >
> > The logs files showed a startup, printing of all the config options that
> > had been set, 1 or 2 commands that got executed and then nothing.
> >
> > Sending the curl did not get shown in the logs files until after that
> > period where Solr became unresponsive.
> >
> > Service mesh, I don't think so? It's in a docker container, but that
> > shouldn't be a problem, it usually never is.
> >
> >
> > On Wed, Nov 20, 2019 at 10:42 AM Jörn Franke <jornfranke@gmail.com>
> wrote:
> >
> >> Have you checked the log files of Solr?
> >>
> >>
> >> Do you have a service mesh in-between? Could it be something at the
> >> network layer/container orchestration  that is blocking requests for
> some
> >> minutes?
> >>
> >>> Am 20.11.2019 um 10:32 schrieb Koen De Groote <
> >> koen.degroote@limecraft.com>:
> >>>
> >>> Hello
> >>>
> >>> I was testing some backup/restore scenarios.
> >>>
> >>> 1 of them is Solr7.6 in a docker container(7.6.0-slim), set up as
> >>> SolrCloud, with zookeeper.
> >>>
> >>> The steps are as follows:
> >>>
> >>> 1. Manually delete the data folder.
> >>> 2. Restart the container. The process is now in error mode, complaining
> >>> that it cannot find the cores.
> >>> 3. Fix the install, meaning create new data folders, which are empty at
> >>> this point.
> >>> 4. Restart the container again, to pick up the empty folders and not be
> >> in
> >>> error anymore.
> >>> 5. Perform the restore
> >>> 6. Check if everything is available again
> >>>
> >>> The problem is between step 4 and 5. After step 4, it takes several
> >> minutes
> >>> before solr actually responds to curl commands.
> >>>
> >>> Once responsive, the restore happened just fine. But it's very
> stressful
> >> in
> >>> a situation where you have to restore a production environment and the
> >>> process just doesn't respond for 5-10 minutes.
> >>>
> >>> We're talking about 20GB of data here, so not very much, but not little
> >>> either.
> >>>
> >>> Is it normal that it takes so long before solr responds? If not, what
> >>> should I look at in order to find the cause?
> >>>
> >>> I have asked this before recently, though the wording was confusing.
> This
> >>> should be clearer.
> >>>
> >>> Kind regards,
> >>> Koen De Groote
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message