lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anshum <ansh...@gmail.com>
Subject Re: Scale out design patterns
Date Fri, 21 Jan 2011 06:34:44 GMT
Hi Ganesh,
I'd suggest, if you have a particular dimension/field on which you could
shard your data such that the query/data breakup gets predictable, that
would be a good way to scale out e.g. if you have users which are equally
active/searched then you may want to split their data on a simple mod of
some numeric (auto increment) userid.
This works well under normal cases unless your partitioning is not
predictable.

--
Anshum Gupta
http://ai-cafe.blogspot.com


On Fri, Jan 21, 2011 at 10:52 AM, Ganesh <emailgane@yahoo.co.in> wrote:

> Hello all,
>
> Could you any one guide me what all the various ways we could scale out?
>
> 1. Index:  Add data to the nodes in round-robin.
>   Search: Query all the nodes and cluster the results using carrot2.
>
> 2.Horizontal partitioning and No shared architecture,
>   Index:   Split the data based on userid and index few set of users data
> in each node.
>   Search: Have a mapper kind of application which could tell which userid
> is mapped to node, redirect the search traffic to corresponding node.
>
> Which one is best? Did you guys tried any of these approach. Please share
> your thoughts.
>
> Regards
> Ganesh
> Send free SMS to your Friends on Mobile from your Yahoo! Messenger.
> Download Now! http://messenger.yahoo.com/download.php
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message