lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Weber <m...@mattweber.org>
Subject Re: Help Writing a Distributed Component
Date Thu, 21 May 2009 23:41:43 GMT
Thank you Matt, this seems to be working as expected.  I am going to  
clean it up a bit, write some unit tests, and we should be good to  
go.  Thanks again!

Grant, yes that is exactly what this implies.  I had no idea this was  
the cause because the other distributed components (facets, stats,  
debug, etc) are all default components that follow the query  
component.  Because they follow the query component, they don't do  
this manual parsing of the shards and I never knew this is why they  
worked and my code did not.  Anyways, I think abstracting this logic  
as you suggested is the way to go because it will define a standard  
way to write a distributed component.

Thanks,

Matt Weber
eSr Technologies
http://www.esr-technologies.com




On May 21, 2009, at 3:54 PM, Grant Ingersoll wrote:

>
> On May 21, 2009, at 4:28 PM, Woytowitz, Matthew wrote:
>
>> Matt,
>>
>> I applied you patch to the trunk and tossed it in the debugger.   
>> I'm not
>> sure how your solrconfig.xml looks, but if your using the out of  
>> the box
>> autosuggest shards don't work.
>>
>> The reason is that the default search handler has the  
>> QueryComponent in
>> it's list and the autosuggest doesn't.
>>
>
> Does this imply that the QC must be present for most components to  
> be distro ready?  Perhaps we need to abstract some of that up, so  
> that other components can better take advantage w/o requiring the QC  
> or having to repeat the same code.
>
>
>> Try stealing these lines from the end QueryComponent.prepare
>>
>>   // TODO: temporary... this should go in a different component.
>>   String shards = params.get(ShardParams.SHARDS);
>>   if (shards != null) {
>>     List<String> lst = StrUtils.splitSmart(shards, ",", true);
>>     rb.shards = lst.toArray(new String[lst.size()]);
>>   }
>>
>> Matt Woytowitz
>>
>>
>> -----Original Message-----
>> From: Matt Weber [mailto:matt@mattweber.org]
>> Sent: Wednesday, May 20, 2009 12:07 PM
>> To: solr-dev@lucene.apache.org
>> Subject: Re: Help Writing a Distributed Component
>>
>> Hey Grant,
>>
>> I have opened a new ticket with the current version of my patch at
>> https://issues.apache.org/jira/browse/SOLR-1177
>> .  What do you mean by configuring my shards?  I implemented the
>> distributed methods (prepare, process, modifyRequest, etc) as is done
>> in the other distributed components, then I just specify a shards
>> parameter in the query.  I looked though the other distributed
>> components and did not see them doing any special shard configuration
>> other than modifying the query that is sent to the shards.  I must be
>> missing something, but I can't seem to figure out due to my lack of
>> knowledge on distributed components and the solr codebase in general.
>> Thanks for your help!
>>
>> For my testing, I have 2 instances of Solr running in tomcat called
>> solr1 and solr2.  My test query looks like:
>>
>> http://localhost:8080/solr1/terms?shards=localhost:8080/solr1,localhost 
>> :
>> 8080/solr2&terms=true&terms.fl=spell&terms.prefix=f&terms.lower=f
>>
>> Thanks,
>>
>> Matt Weber
>> eSr Technologies
>> http://www.esr-technologies.com
>>
>>
>> On May 20, 2009, at 7:24 AM, Grant Ingersoll wrote:
>>
>>> Matt,
>>>
>>> Perhaps you can post your patch even in it's current state on a JIRA
>>> issue and then we can work through it?  Also, how are you
>>> configuring your shards?
>>>
>>> I started that Wiki page as a way to make sure I understood what was
>>> going on in distributed.  I'm not 100% sure it is correct, so it
>>> would be good if others review/edit as well.
>>>
>>> -Grant
>>>
>>> On May 20, 2009, at 1:36 AM, Matt Weber wrote:
>>>
>>>> Hi all,
>>>>
>>>> I am working on a patch to get TermsComponent distributed and have
>>>> run into a problem.  I have overridden the prepare, process,
>>>> modifyRequest, handleResponses, and finishStage methods as
>>>> described at
>> http://wiki.apache.org/solr/WritingDistributedSearchComponents
>>>> .  My problem is that only the prepare and process methods are
>>>> called just as if it was non-distributed.  It looks like the shards
>>>> parameter is not being honored for this handler.  For some reason
>>>> rb.shards is always null.  I looked though all the other
>>>> distributed components code (facets, stats, highlighting, etc) and
>>>> did not notice them doing anything special that my handler is not.
>>>> Is there some setting I need to enable for the shards parameter to
>>>> be honored?
>>>
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
> using Solr/Lucene:
> http://www.lucidimagination.com/search
>


Mime
View raw message