incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: shards
Date Sun, 20 Jan 2013 01:18:21 GMT
Thanks!


On Sat, Jan 19, 2013 at 8:06 PM, Tim Williams <williamstw@gmail.com> wrote:

> On Sat, Jan 19, 2013 at 3:50 PM, Aaron McCurry <amccurry@gmail.com> wrote:
> > It can. While its not implemented yet, I want to make a pluggable
> strategy.  Along with controlling the layout it would also handle shard
> count changes, and if they are allowed.
> >
> > So a couple of strategies that I have been thinking about.
> >
> > -Hash based where it would hash on a pre-configured field.  Field would
> not be allowed to be null and the number of shards would be fixed.  Also
> the shard placement provided by the user would be ignored.
> > -User based where the user has total control over the placement of the
> document by providing it during indexing.  If a shard index is provided in
> an update and the current table does not continue that shard, then a new
> one would be created and added to the table.
> >
> > As for now we are now somewhere in between.  The number of shards are
> fixed and it's up to the user to provide the shard index.  I think (need to
> look at the code) if the user provides a -1 then it randomly chooses a
> shard for the document. It's could be dangerous for updates. We should
> create a jira issue to discuss further and provide a better implementation.
>
> Cool, I added a JIRA(BLUR-55)[1] to discuss it.  Yeah, maybe require
> an 'id' field and do a typical mod by shard count as a happier
> default.
>
> --tim
>
> [1] - https://issues.apache.org/jira/browse/BLUR-55
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message