incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BLUR-55) Pluggable sharding strategy
Date Sun, 20 Jan 2013 19:44:12 GMT

    [ https://issues.apache.org/jira/browse/BLUR-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558353#comment-13558353
] 

Aaron McCurry commented on BLUR-55:
-----------------------------------

I created a wiki page to describe some thoughts about the layout strategy.

http://wiki.apache.org/blur/LayoutStrategy
                
> Pluggable sharding strategy
> ---------------------------
>
>                 Key: BLUR-55
>                 URL: https://issues.apache.org/jira/browse/BLUR-55
>             Project: Apache Blur
>          Issue Type: Bug
>    Affects Versions: 0.2.0
>            Reporter: Tim Williams
>
> The 0.2-dev code currently is driven from the client.  We should make the sharding strategy
pluggable so that someone who needs something more than a typical modulo on the docid over
the shard count can do it. 
> From Aaron's response[1]:
> "So a couple of strategies that I have been thinking about.
> -Hash based where it would hash on a pre-configured field.  Field would not be allowed
to be null and the number of shards would be fixed.  Also the shard placement provided by
the user would be ignored.
> -User based where the user has total control over the placement of the document by providing
it during indexing.  If a shard index is provided in an update and the current table does
not continue that shard, then a new one would be created and added to the table.
> As for now we are now somewhere in between.  The number of shards are fixed and it's
up to the user to provide the shard index.  I think (need to look at the code) if the user
provides a -1 then it randomly chooses a shard for the document. It's could be dangerous for
updates. We should create a jira issue to discuss further and provide a better implementation."
> [1] - http://mail-archives.apache.org/mod_mbox/incubator-blur-dev/201301.mbox/%3CC671051A-11E8-4721-AC95-D902250E3EA9%40gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message