lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Bernstein (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-10524) Explore in-memory partitioning for processing Overseer queue messages
Date Mon, 08 May 2017 20:59:04 GMT

    [ https://issues.apache.org/jira/browse/SOLR-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001514#comment-16001514
] 

Joel Bernstein edited comment on SOLR-10524 at 5/8/17 8:58 PM:
---------------------------------------------------------------

I'm seeing the errors below when running the StreamExpressionTest. I suspect it's related
to this ticket. I've been adding tests the past couple of days but only started seeing this
today:
 Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239409 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239410 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239411 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239412 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239413 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239413 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239414 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239415 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop



was (Author: joel.bernstein):
I'm seeing the errors below in the StreamingExpressionTest. I suspect it's related to this
ticket:
 Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239409 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239410 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239411 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239412 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239413 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239413 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239414 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> 239415 ERROR (OverseerStateUpdate-97928916256817164-127.0.0.1:51485_solr-n_0000000000)
[n:127.0.0.1:51485_solr    ] o.a.s.c.Overseer Exception in Overseer main queue loop


> Explore in-memory partitioning for processing Overseer queue messages
> ---------------------------------------------------------------------
>
>                 Key: SOLR-10524
>                 URL: https://issues.apache.org/jira/browse/SOLR-10524
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Erick Erickson
>         Attachments: SOLR-10524-NPE-fix.patch, SOLR-10524.patch, SOLR-10524.patch, SOLR-10524.patch,
SOLR-10524.patch
>
>
> There are several JIRAs (I'll link in a second) about trying to be more efficient about
processing overseer messages as the overseer can become a bottleneck, especially with very
large numbers of replicas in a cluster. One of the approaches mentioned near the end of SOLR-5872
(15-Mar) was to "read large no:of items say 10000. put them into in memory buckets and feed
them into overseer....".
> This JIRA is to break out that part of the discussion as it might be an easy win whereas
"eliminating the Overseer queue" would be quite an undertaking.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message