lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eks Dev (JIRA)" <>
Subject [jira] Commented: (SOLR-1044) Use Hadoop RPC for inter Solr communication
Date Mon, 02 Mar 2009 21:36:56 GMT


Eks Dev commented on SOLR-1044:

I do not know much about Solr needs there, but we are using one of prehistoric versions of
hadoop RPC (no NIO version)  as everything else proved to eat far to much time (in 800+ rq/sec
environment every millisecond counts). Creating new Sockets is not working there as OSs start
having problems to keep up with this rate (especially with java , slower Socket release due
to gc() latency).  

We are anyhow contemplating to give etch (or thrift) a try. Etch looks like really good peace
of work, with great flexibility. Someone tried it? 

> Use Hadoop RPC for inter Solr communication
> -------------------------------------------
>                 Key: SOLR-1044
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>          Components: search
>            Reporter: Noble Paul
> Solr uses http for distributed search . We can make it a whole lot faster if we use an
RPC mechanism which is more lightweight/efficient. 
> Hadoop RPC looks like a good candidate for this.  
> The implementation should just have one protocol. It should follow the Solr's idiom of
making remote calls . A uri + params +[optional stream(s)] . The response can be a stream
of bytes.
> To make this work we must make the SolrServer implementation pluggable in distributed
search. Users should be able to choose between the current CommonshttpSolrServer, or a HadoopRpcSolrServer

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message