geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Udo Kohlmeyer <ukohlme...@pivotal.io>
Subject Re: [DISCUSS] easy string based partitioning
Date Mon, 05 Jun 2017 20:27:32 GMT
My concern with this approach is that we provide an implementation that 
might be a white elephant and only used by a 1% user base. In addition 
to that, it does really restrict the user on his keys.

Whereas something a little more configurable, like a SPEL 
implementation, would provide a lot more flexibility. Which means that 
one could easily change the partition resolver expression upon region 
creation/configuration.

--Udo


On 6/5/17 08:46, Jens Deppe wrote:
> I like the idea of keeping the configuration 'conventional' and thus not
> requiring any server configuration. As soon as you need to define a regex
> on the server you might as well be writing PartitionResolver code.
>
> As an aside I also think that using regexes to parse keys would also
> introduce a measurable performance hit.
>
> --Jens
>
> On Mon, Jun 5, 2017 at 8:21 AM, Udo Kohlmeyer <ukohlmeyer@pivotal.io> wrote:
>
>> Whilst I like the idea to make something (and yes,
>> DelimitedStringPartitionResolver is the simplest), I feel that we might
>> be able to do one better.
>>
>> */Could/* one possibly have an /*SPEL*/ <https://docs.spring.io/spring
>> -framework/docs/current/spring-framework-reference/html/expressions.html>-based
>> PartitionResolver for this? At least this way, we don't have to rely on
>> delimiters or a strong knowledge of REGEX.
>>
>> IMO, this would provide the greatest bang for buck implementation.
>>
>> --Udo
>>
>>
>>
>>
>> On 6/2/17 19:15, Jacob Barrett wrote:
>>
>>> If you implement as regular expression the user doesn't have to reformat
>>> their key to a specific format (akin to making them implement a class). I
>>> would concat the matching groups for generate the routing key.
>>>
>>> Consider RegEx: .*\bcorrelation=(\d+).*\bmaybe-something-else=(\w)
>>> With Keys:
>>> A: my,key;with:any-chars;unique=12345;correlation=678/and,maybe
>>> -something-else=a
>>> B: my,key;unique=876324;correlation=678;and,maybe-something-else=a,foo
>>> C: somthing;different=988975;correlation=678;then,maybe-something-else=ba
>>>
>>> Keys A and B would have routing key '678a'. Key C would have routing key
>>> '678b'.
>>>
>>> -Jake
>>>
>>>
>>>
>>> Consider
>>>
>>> On Fri, Jun 2, 2017 at 4:02 PM Darrel Schneider <dschneider@pivotal.io>
>>> wrote:
>>>
>>> Geode partitioned regions usually partition the data based on the key's
>>>> hashcode.
>>>> You can do your own partitioning by implementing the PartitionResolver
>>>> interface and configuring it on the partitioned region.
>>>>
>>>> In some use cases needing to deploy your class that implements
>>>> PartitionResolver can be problematic so we would like to find a way to
>>>> offer partitioning based on a portion of the key (instead of the default
>>>> which uses the entire key) that does not require you to implement your
>>>> own
>>>> PartitionResolver and does not require you to deploy your own code to do
>>>> the custom partitioning.
>>>>
>>>> Another group of users that do not want to implement PartitionResolver
>>>> are
>>>> non-java clients. So the solution is required to be usable by non-java
>>>> geode clients without needing to reimplement the client to support a new
>>>> feature.
>>>>
>>>> Another constraint on the solution is for it to be both easy to use and
>>>> easy to implement.
>>>>
>>>> The proposed solution is to provide a class named:
>>>>       org.apache.geode.cache.StringPrefixPartitionResolver
>>>> This class will implement PartitionResolver and have a default
>>>> constructor.
>>>> To use it you need to configure a partitioned region's PartitionResolver
>>>> using the already existing mechanism for this (api, gfsh, or xml).
>>>> The StringPrefixPartitionResolver will require all keys on its region to
>>>> be
>>>> of type String.
>>>> It also requires that the string key contains at least one ':' character.
>>>> The substring of the key that precedes the first ':' is the prefix that
>>>> will be returned from "getRoutingObject".
>>>>
>>>> An example of doing this in gfsh is:
>>>>       create region --name=r1 --type=PARTITION
>>>> --partition-resolver=org.apache.geode.cache.StringPrefixPart
>>>> itionResolver
>>>>
>>>> Note that attempting to use a key that is not a String or does not
>>>> contain
>>>> a ':' will throw an exception. This is to help developers realize they
>>>> made
>>>> a mistake.
>>>>
>>>> Note that the delimiter is always a ':'. It would be easy to made the
>>>> delimiter configurable when using apis or xml but currently gfsh does not
>>>> provide a way to pass parameters to the --partition-resolver create
>>>> region
>>>> option.
>>>>
>>>> The only public api change this proposal makes is the new
>>>> StringPrefixPartitionResolver class.
>>>>
>>>>


Mime
View raw message