Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of markrmiller@gmail.com designates
 64.233.184.228 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding;
        b=ReFt3Qft5KBfkVTpJ4xxxvskRF+LvAcQB+PpHRhxd3cVBPawwqhnbZPfr8I0u7FXWRhlq2yObKnR+kpYbFnt8HRI/XPMWcc4lXNPM2N3lq1kCkiT4WCjxsgLnekYA9acIwIYV+ocsAoXQ2cQ0J8oN7B4N05bpwXRmIZQNpMRDL8=
Message-ID: <44C96957.3070701@gmail.com>
Date: Thu, 27 Jul 2006 21:33:11 -0400
From: Mark Miller <markrmiller@gmail.com>
User-Agent: Thunderbird 1.5.0.4 (Windows/20060516)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Distributed Search
References: <20060728005131.84322.qmail@web50304.mail.yahoo.com>
In-Reply-To: <20060728005131.84322.qmail@web50304.mail.yahoo.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Otis Gospodnetic wrote:
> I think we have an RMI example in Lucene in Action.
> You could also look at how Nutch does it.  I think the code is in org.apache.nutch.ipc package.
> I'm not sure why cross-platform requirement rules out Solr, I would think it would exactly the opposite.
> As for 10m limit, it depends.  It depends on the actual size of the index (indexed fields), complexity of queries, required query latency, the hardware you throw at it, etc.  So you can't really say 10m is the limit.  You might have gotten that number from some of the older Nutch docs/presentations, which means they are a few years old now and are Nutch-specific.
>
> Clustering and failover and "easily" don't really go together, in my experience, and this is not limited to Luceneland. :(
> I'd love to be wrong about this, but it seems clustering/failover/HA stuff + Lucene always ends up being a custom and propriatory job.
>
> Otis
>
> ----- Original Message ----
> From: Mark Miller <markrmiller@gmail.com>
> To: java-user@lucene.apache.org
> Sent: Thursday, July 27, 2006 5:45:52 PM
> Subject: Distributed Search
>
> I know there has been a lot of discussion on distributed search...I am 
> looking for a cross platform solution, which seems to kill solr's 
> approach...Everyone seems to have implemented this, but only as 
> proprietary code...it would seem that just using the RMI searcher would 
> allow a simple solution? Is this the case? Could you easily provide 
> clustering and fail over using a variety of indexes and searching them 
> all with RMI searcher? Is it all really that complicated? I have read 
> that Lucene tops out at about 10m docs for a single server...I want to 
> hit 100m. I have a beautiful app that allows realtime updating/searching 
> (updates are rare but should be instant)...and I just want it to scale 
> up to 100m docs or so . Is that going to be an really advanced project 
> no matter how I slice it? I have done a lot of custom work with the 
> lucene stuff so it would seem difficult to adapt it to Nutch (but what 
> do I know Nutch) ... I have seen a lot of talk but not much on a simple 
> RMI searcher solution...any idea?
>
>
> - Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>   
Thanks for the info Otis. I thought I read that solr requires an OS that 
supports hard links and thought that Windows only supports soft links. 
Perhaps I am wrong.

Thanks,

- mark

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org