Mailing-List: contact dev-help@geode.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@geode.apache.org
MIME-Version: 1.0
In-Reply-To: <CAJCfvb5_iMKu9hyvtg6-vJ2=N32pyexmi1uwRG0JV1Hpuv_rog@mail.gmail.com>
References: <CAJCfvb59E5mvujjur4rO1-GQbUH4-CSzQ=Xs5is83tkHnevOQg@mail.gmail.com>
 <CAA1wVxo1e3K1iCGKGVYS_oO3hZfGxW6o_Br=HSRsM=0q7gixsA@mail.gmail.com>
 <3777c347-f832-5422-a7a2-a279e63fefed@pivotal.io> <CANztrJONMpDhNzKKjDNhVfZfdGvufeyDus-i9VVwsCPxLNJtTg@mail.gmail.com>
 <CAPHj4oQbrQ=+OcS_UvTwTC4cC+XFuKxrgbiyCSQP470eDipReQ@mail.gmail.com> <CAJCfvb5_iMKu9hyvtg6-vJ2=N32pyexmi1uwRG0JV1Hpuv_rog@mail.gmail.com>
From: Michael Stolz <mstolz@pivotal.io>
Date: Mon, 5 Jun 2017 16:28:27 -0400
Message-ID: <CANgOuWs25EgVv00PYRFugCS9cdDn=o9yrH0czL-5GNVJH9eGWg@mail.gmail.com>
Subject: Re: [DISCUSS] easy string based partitioning
To: dev@geode.apache.org
Content-Type: multipart/alternative; boundary="001a1140bf2c95144305513c5845"
archived-at: Mon, 05 Jun 2017 20:28:36 -0000

--001a1140bf2c95144305513c5845
Content-Type: text/plain; charset="UTF-8"

Yep that will work.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771

On Jun 5, 2017 12:06 PM, "Darrel Schneider" <dschneider@pivotal.io> wrote:

> I pictured the top level parent region (your customer region) as not having
> the StringPrefixPartitionResolver. Instead it would just use string keys
> that have no prefix and use the default resolver.
> It would be each co-located child region (your order region) that would
> have the StringPrefixPartitionResolver and would format its keys to be
> "parentKey:childKey". Does that make sense? Will it work?
>
>
> On Mon, Jun 5, 2017 at 10:06 AM, Barry Oglesby <boglesby@pivotal.io>
> wrote:
>
> > The top-level region may not have / need a delimiter. If you have a
> > customer region and a colocated orders region, the customer region key
> > could be the customerId, and the orders region key could be the
> > customerId:orderId. The colocation would be on the customerId. In the
> > customer region, it doesn't need a delimiter. Is it ok that this idea
> would
> > require one? Maybe the top-level region shouldn't require a delimiter? If
> > the StringPrefixPartitionResolver is using key.split(":"), the customer
> > region would return the entire key.
> >
> >
> > Thanks,
> > Barry Oglesby
> >
> >
> > On Mon, Jun 5, 2017 at 8:46 AM, Jens Deppe <jdeppe@pivotal.io> wrote:
> >
> > > I like the idea of keeping the configuration 'conventional' and thus
> not
> > > requiring any server configuration. As soon as you need to define a
> regex
> > > on the server you might as well be writing PartitionResolver code.
> > >
> > > As an aside I also think that using regexes to parse keys would also
> > > introduce a measurable performance hit.
> > >
> > > --Jens
> > >
> > > On Mon, Jun 5, 2017 at 8:21 AM, Udo Kohlmeyer <ukohlmeyer@pivotal.io>
> > > wrote:
> > >
> > > > Whilst I like the idea to make something (and yes,
> > > > DelimitedStringPartitionResolver is the simplest), I feel that we
> > might
> > > > be able to do one better.
> > > >
> > > > */Could/* one possibly have an /*SPEL*/ <
> https://docs.spring.io/spring
> > > > -framework/docs/current/spring-framework-reference/
> > > html/expressions.html>-based
> > > > PartitionResolver for this? At least this way, we don't have to rely
> on
> > > > delimiters or a strong knowledge of REGEX.
> > > >
> > > > IMO, this would provide the greatest bang for buck implementation.
> > > >
> > > > --Udo
> > > >
> > > >
> > > >
> > > >
> > > > On 6/2/17 19:15, Jacob Barrett wrote:
> > > >
> > > >> If you implement as regular expression the user doesn't have to
> > reformat
> > > >> their key to a specific format (akin to making them implement a
> > class).
> > > I
> > > >> would concat the matching groups for generate the routing key.
> > > >>
> > > >> Consider RegEx: .*\bcorrelation=(\d+).*\bmaybe-something-else=(\w)
> > > >> With Keys:
> > > >> A: my,key;with:any-chars;unique=12345;correlation=678/and,maybe
> > > >> -something-else=a
> > > >> B: my,key;unique=876324;correlation=678;and,maybe-
> > something-else=a,foo
> > > >> C: somthing;different=988975;correlation=678;then,maybe-
> > > something-else=ba
> > > >>
> > > >> Keys A and B would have routing key '678a'. Key C would have routing
> > key
> > > >> '678b'.
> > > >>
> > > >> -Jake
> > > >>
> > > >>
> > > >>
> > > >> Consider
> > > >>
> > > >> On Fri, Jun 2, 2017 at 4:02 PM Darrel Schneider <
> > dschneider@pivotal.io>
> > > >> wrote:
> > > >>
> > > >> Geode partitioned regions usually partition the data based on the
> > key's
> > > >>> hashcode.
> > > >>> You can do your own partitioning by implementing the
> > PartitionResolver
> > > >>> interface and configuring it on the partitioned region.
> > > >>>
> > > >>> In some use cases needing to deploy your class that implements
> > > >>> PartitionResolver can be problematic so we would like to find a way
> > to
> > > >>> offer partitioning based on a portion of the key (instead of the
> > > default
> > > >>> which uses the entire key) that does not require you to implement
> > your
> > > >>> own
> > > >>> PartitionResolver and does not require you to deploy your own code
> to
> > > do
> > > >>> the custom partitioning.
> > > >>>
> > > >>> Another group of users that do not want to implement
> > PartitionResolver
> > > >>> are
> > > >>> non-java clients. So the solution is required to be usable by
> > non-java
> > > >>> geode clients without needing to reimplement the client to support
> a
> > > new
> > > >>> feature.
> > > >>>
> > > >>> Another constraint on the solution is for it to be both easy to use
> > and
> > > >>> easy to implement.
> > > >>>
> > > >>> The proposed solution is to provide a class named:
> > > >>>      org.apache.geode.cache.StringPrefixPartitionResolver
> > > >>> This class will implement PartitionResolver and have a default
> > > >>> constructor.
> > > >>> To use it you need to configure a partitioned region's
> > > PartitionResolver
> > > >>> using the already existing mechanism for this (api, gfsh, or xml).
> > > >>> The StringPrefixPartitionResolver will require all keys on its
> region
> > > to
> > > >>> be
> > > >>> of type String.
> > > >>> It also requires that the string key contains at least one ':'
> > > character.
> > > >>> The substring of the key that precedes the first ':' is the prefix
> > that
> > > >>> will be returned from "getRoutingObject".
> > > >>>
> > > >>> An example of doing this in gfsh is:
> > > >>>      create region --name=r1 --type=PARTITION
> > > >>> --partition-resolver=org.apache.geode.cache.StringPrefixPart
> > > >>> itionResolver
> > > >>>
> > > >>> Note that attempting to use a key that is not a String or does not
> > > >>> contain
> > > >>> a ':' will throw an exception. This is to help developers realize
> > they
> > > >>> made
> > > >>> a mistake.
> > > >>>
> > > >>> Note that the delimiter is always a ':'. It would be easy to made
> the
> > > >>> delimiter configurable when using apis or xml but currently gfsh
> does
> > > not
> > > >>> provide a way to pass parameters to the --partition-resolver create
> > > >>> region
> > > >>> option.
> > > >>>
> > > >>> The only public api change this proposal makes is the new
> > > >>> StringPrefixPartitionResolver class.
> > > >>>
> > > >>>
> > > >
> > >
> >
>

--001a1140bf2c95144305513c5845--