lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Woytowitz, Matthew" <Matthew.Woytow...@ManTech.com>
Subject RE: Help Writing a Distributed Component
Date Thu, 21 May 2009 22:02:20 GMT
I found a couple more things for you.

Your modifyRequest method won't get called, because in the default
/autosuggest the termsComp is the only component.

  /** Called after another component adds a request */
  public void modifyRequest(ResponseBuilder rb, SearchComponent who,
ShardRequest sreq) {
  }

<searchComponent name="termsComp"
class="org.apache.solr.handler.component.TermsComponent"/>
  <requestHandler name="/autoSuggest"
class="org.apache.solr.handler.component.SearchHandler">
    <arr name="components">
    <str>termsComp</str>
    </arr>
</requestHandler>

You should move the logic from modifyRequest (since there isn't going to
be another component creating a shard request for you to modify) to
distributeProcess

I lifted this from QueryComponent.

  @Override
  public int distributedProcess(ResponseBuilder rb) throws IOException {
    if (rb.stage == ResponseBuilder.STAGE_EXECUTE_QUERY) {
      TermsInfo ti = rb._termsInfo;
      if (ti == null) {
        ti = rb._termsInfo = new TermsInfo();
        ti.init(rb.req.getParams());
      }
      createMainQuery(rb);
    }
    if (rb.stage < ResponseBuilder.STAGE_EXECUTE_QUERY){ 
      return ResponseBuilder.STAGE_EXECUTE_QUERY;
    }else {
      return ResponseBuilder.STAGE_DONE;
    }
  }

  private void createMainQuery(ResponseBuilder rb) {
    ShardRequest sreq = new ShardRequest();
    sreq.purpose = ShardRequest.PURPOSE_GET_TERMS;

    sreq.params = new ModifiableSolrParams(rb.req.getParams());
    // TODO: base on current params or original params?

    // don't pass through any shards param
    sreq.params.remove(ShardParams.SHARDS);


	//Add what ever else you need.

    rb.addRequest(this, sreq);
  }



Make sure you add shards.qt to your original request.  I set it to
shards.qt=/autosuggest.  If you don't the shard request will be send to
/select instead of /autoSuggest.

Matt Woytowitz

-----Original Message-----
From: Matt Weber [mailto:matt@mattweber.org] 
Sent: Thursday, May 21, 2009 5:03 PM
To: solr-dev@lucene.apache.org
Subject: Re: Help Writing a Distributed Component

Thanks Matt that worked!  My patch still doesn't work correctly, but  
at least I can actually get it calling the distributed methods now.   
Do you mind explaining how you run solr though a debugger?  That would  
sure make things easier for me!  Thanks again, I really appreciate  
your help.

Thanks,

Matt Weber
eSr Technologies
http://www.esr-technologies.com




On May 21, 2009, at 1:28 PM, Woytowitz, Matthew wrote:

> Matt,
>
> I applied you patch to the trunk and tossed it in the debugger.  I'm  
> not
> sure how your solrconfig.xml looks, but if your using the out of the  
> box
> autosuggest shards don't work.
>
> The reason is that the default search handler has the QueryComponent  
> in
> it's list and the autosuggest doesn't.
>
> Try stealing these lines from the end QueryComponent.prepare
>
>    // TODO: temporary... this should go in a different component.
>    String shards = params.get(ShardParams.SHARDS);
>    if (shards != null) {
>      List<String> lst = StrUtils.splitSmart(shards, ",", true);
>      rb.shards = lst.toArray(new String[lst.size()]);
>    }
>
> Matt Woytowitz
>
>
> -----Original Message-----
> From: Matt Weber [mailto:matt@mattweber.org]
> Sent: Wednesday, May 20, 2009 12:07 PM
> To: solr-dev@lucene.apache.org
> Subject: Re: Help Writing a Distributed Component
>
> Hey Grant,
>
> I have opened a new ticket with the current version of my patch at
> https://issues.apache.org/jira/browse/SOLR-1177
> .  What do you mean by configuring my shards?  I implemented the
> distributed methods (prepare, process, modifyRequest, etc) as is done
> in the other distributed components, then I just specify a shards
> parameter in the query.  I looked though the other distributed
> components and did not see them doing any special shard configuration
> other than modifying the query that is sent to the shards.  I must be
> missing something, but I can't seem to figure out due to my lack of
> knowledge on distributed components and the solr codebase in general.
> Thanks for your help!
>
> For my testing, I have 2 instances of Solr running in tomcat called
> solr1 and solr2.  My test query looks like:
>
>
http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost 
> :
> 8080/solr2&terms=true&terms.fl=spell&terms.prefix=f&terms.lower=f
>
> Thanks,
>
> Matt Weber
> eSr Technologies
> http://www.esr-technologies.com
>
>
> On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
>
>> Matt,
>>
>> Perhaps you can post your patch even in it's current state on a JIRA
>> issue and then we can work through it?  Also, how are you
>> configuring your shards?
>>
>> I started that Wiki page as a way to make sure I understood what was
>> going on in distributed.  I'm not 100% sure it is correct, so it
>> would be good if others review/edit as well.
>>
>> -Grant
>>
>> On May 20, 2009, at 1:36 AM, Matt Weber wrote:
>>
>>> Hi all,
>>>
>>> I am working on a patch to get TermsComponent distributed and have
>>> run into a problem.  I have overridden the prepare, process,
>>> modifyRequest, handleResponses, and finishStage methods as
>>> described at
> http://wiki.apache.org/solr/WritingDistributedSearchComponents
>>> .  My problem is that only the prepare and process methods are
>>> called just as if it was non-distributed.  It looks like the shards
>>> parameter is not being honored for this handler.  For some reason
>>> rb.shards is always null.  I looked though all the other
>>> distributed components code (facets, stats, highlighting, etc) and
>>> did not notice them doing anything special that my handler is not.
>>> Is there some setting I need to enable for the shards parameter to
>>> be honored?
>>
>


Mime
View raw message