geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Darrel Schneider <dschnei...@pivotal.io>
Subject Re: [DISCUSS] easy string based partitioning
Date Mon, 05 Jun 2017 19:06:03 GMT
I pictured the top level parent region (your customer region) as not having
the StringPrefixPartitionResolver. Instead it would just use string keys
that have no prefix and use the default resolver.
It would be each co-located child region (your order region) that would
have the StringPrefixPartitionResolver and would format its keys to be
"parentKey:childKey". Does that make sense? Will it work?


On Mon, Jun 5, 2017 at 10:06 AM, Barry Oglesby <boglesby@pivotal.io> wrote:

> The top-level region may not have / need a delimiter. If you have a
> customer region and a colocated orders region, the customer region key
> could be the customerId, and the orders region key could be the
> customerId:orderId. The colocation would be on the customerId. In the
> customer region, it doesn't need a delimiter. Is it ok that this idea would
> require one? Maybe the top-level region shouldn't require a delimiter? If
> the StringPrefixPartitionResolver is using key.split(":"), the customer
> region would return the entire key.
>
>
> Thanks,
> Barry Oglesby
>
>
> On Mon, Jun 5, 2017 at 8:46 AM, Jens Deppe <jdeppe@pivotal.io> wrote:
>
> > I like the idea of keeping the configuration 'conventional' and thus not
> > requiring any server configuration. As soon as you need to define a regex
> > on the server you might as well be writing PartitionResolver code.
> >
> > As an aside I also think that using regexes to parse keys would also
> > introduce a measurable performance hit.
> >
> > --Jens
> >
> > On Mon, Jun 5, 2017 at 8:21 AM, Udo Kohlmeyer <ukohlmeyer@pivotal.io>
> > wrote:
> >
> > > Whilst I like the idea to make something (and yes,
> > > DelimitedStringPartitionResolver is the simplest), I feel that we
> might
> > > be able to do one better.
> > >
> > > */Could/* one possibly have an /*SPEL*/ <https://docs.spring.io/spring
> > > -framework/docs/current/spring-framework-reference/
> > html/expressions.html>-based
> > > PartitionResolver for this? At least this way, we don't have to rely on
> > > delimiters or a strong knowledge of REGEX.
> > >
> > > IMO, this would provide the greatest bang for buck implementation.
> > >
> > > --Udo
> > >
> > >
> > >
> > >
> > > On 6/2/17 19:15, Jacob Barrett wrote:
> > >
> > >> If you implement as regular expression the user doesn't have to
> reformat
> > >> their key to a specific format (akin to making them implement a
> class).
> > I
> > >> would concat the matching groups for generate the routing key.
> > >>
> > >> Consider RegEx: .*\bcorrelation=(\d+).*\bmaybe-something-else=(\w)
> > >> With Keys:
> > >> A: my,key;with:any-chars;unique=12345;correlation=678/and,maybe
> > >> -something-else=a
> > >> B: my,key;unique=876324;correlation=678;and,maybe-
> something-else=a,foo
> > >> C: somthing;different=988975;correlation=678;then,maybe-
> > something-else=ba
> > >>
> > >> Keys A and B would have routing key '678a'. Key C would have routing
> key
> > >> '678b'.
> > >>
> > >> -Jake
> > >>
> > >>
> > >>
> > >> Consider
> > >>
> > >> On Fri, Jun 2, 2017 at 4:02 PM Darrel Schneider <
> dschneider@pivotal.io>
> > >> wrote:
> > >>
> > >> Geode partitioned regions usually partition the data based on the
> key's
> > >>> hashcode.
> > >>> You can do your own partitioning by implementing the
> PartitionResolver
> > >>> interface and configuring it on the partitioned region.
> > >>>
> > >>> In some use cases needing to deploy your class that implements
> > >>> PartitionResolver can be problematic so we would like to find a way
> to
> > >>> offer partitioning based on a portion of the key (instead of the
> > default
> > >>> which uses the entire key) that does not require you to implement
> your
> > >>> own
> > >>> PartitionResolver and does not require you to deploy your own code
to
> > do
> > >>> the custom partitioning.
> > >>>
> > >>> Another group of users that do not want to implement
> PartitionResolver
> > >>> are
> > >>> non-java clients. So the solution is required to be usable by
> non-java
> > >>> geode clients without needing to reimplement the client to support
a
> > new
> > >>> feature.
> > >>>
> > >>> Another constraint on the solution is for it to be both easy to use
> and
> > >>> easy to implement.
> > >>>
> > >>> The proposed solution is to provide a class named:
> > >>>      org.apache.geode.cache.StringPrefixPartitionResolver
> > >>> This class will implement PartitionResolver and have a default
> > >>> constructor.
> > >>> To use it you need to configure a partitioned region's
> > PartitionResolver
> > >>> using the already existing mechanism for this (api, gfsh, or xml).
> > >>> The StringPrefixPartitionResolver will require all keys on its region
> > to
> > >>> be
> > >>> of type String.
> > >>> It also requires that the string key contains at least one ':'
> > character.
> > >>> The substring of the key that precedes the first ':' is the prefix
> that
> > >>> will be returned from "getRoutingObject".
> > >>>
> > >>> An example of doing this in gfsh is:
> > >>>      create region --name=r1 --type=PARTITION
> > >>> --partition-resolver=org.apache.geode.cache.StringPrefixPart
> > >>> itionResolver
> > >>>
> > >>> Note that attempting to use a key that is not a String or does not
> > >>> contain
> > >>> a ':' will throw an exception. This is to help developers realize
> they
> > >>> made
> > >>> a mistake.
> > >>>
> > >>> Note that the delimiter is always a ':'. It would be easy to made the
> > >>> delimiter configurable when using apis or xml but currently gfsh does
> > not
> > >>> provide a way to pass parameters to the --partition-resolver create
> > >>> region
> > >>> option.
> > >>>
> > >>> The only public api change this proposal makes is the new
> > >>> StringPrefixPartitionResolver class.
> > >>>
> > >>>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message