incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: shards
Date Sat, 19 Jan 2013 20:50:48 GMT
It can. While its not implemented yet, I want to make a pluggable strategy.  Along with controlling
the layout it would also handle shard count changes, and if they are allowed.

So a couple of strategies that I have been thinking about. 

-Hash based where it would hash on a pre-configured field.  Field would not be allowed to
be null and the number of shards would be fixed.  Also the shard placement provided by the
user would be ignored. 
-User based where the user has total control over the placement of the document by providing
it during indexing.  If a shard index is provided in an update and the current table does
not continue that shard, then a new one would be created and added to the table. 

As for now we are now somewhere in between.  The number of shards are fixed and it's up to
the user to provide the shard index.  I think (need to look at the code) if the user provides
a -1 then it randomly chooses a shard for the document. It's could be dangerous for updates.
We should create a jira issue to discuss further and provide a better implementation. 

Aaron

Sent from my iPad

On Jan 19, 2013, at 1:46 PM, Tim Williams <williamstw@gmail.com> wrote:

> regarding the 0.2 stuffs, if you do mutations through the thrift api,
> it looks like the client "owns" the sharding strategy - is that
> understanding right?
> 
> thanks,
> --tim

Mime
View raw message