geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Stolz <mst...@pivotal.io>
Subject Re: [jira] [Updated] (GEODE-3005) A developer can create a Region with Partition by Prefix using Java API
Date Fri, 02 Jun 2017 17:54:36 GMT
+1 to the RoutableKey. Does need to be supported on any non-java clients
too though.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: +1-631-835-4771

On Thu, Jun 1, 2017 at 8:31 PM, Darrel Schneider <dschneider@pivotal.io>
wrote:

> +1 to the RoutableKey idea.
> It also gets rid of any need for a delimiter since the two args passed when
> constructing this key define which part is for resolving routing and which
> part is the key.
>
> As far as memory overhead goes we could do some optimization of how this
> key is stored by not exposing the actual class we use to implement
> RoutableKey but instead RoutableKey would just be an interface and we would
> provide a factory method on Cache for creating a new key.
>
>
> On Thu, Jun 1, 2017 at 4:59 PM, Fred Krone <fkrone@pivotal.io> wrote:
>
> > Going this route we could do something like: RoutableKey<RO,K> implements
> > PartitionResolver and pass it in as a key (this was batted around in the
> > feature narrative).
> >
> > Here RO is the routing object, K is the original key.
> >
> > Region.put already looks for key to implement PartitonResolver and if it
> > does will use the routing object.
> >
> > This way we don't need any API changes, you don't even need to set this
> > PartitionResolver on gfsh create region and if we do this with generics
> you
> > can make whatever compound key you like.
> >
> > Basically we're just doing the implementation and making it generic.  One
> > caveat is that this neglects .Net clients and will use more memory
> > overhead.
> >
> >
> >
> >
> >
> > On Thu, Jun 1, 2017 at 2:20 PM, Michael Stolz <mstolz@pivotal.io> wrote:
> >
> > > I think that implementing a pre-built PartitionResolver that is bundled
> > > with Geode would enable users to leverage that PartitionResolver
> without
> > > writing any code just by mentioning its class name during region
> > creation.
> > >
> > > That's a huge win. Co-location without writing code. Period. Yay!
> > >
> > > Adding the ability to specify the delimiter is a nice add-on to that
> and
> > > could probably be done by parameterizing the PartitionResolver much
> like
> > > the way Listeners can be parameterized but its not there in the tooling
> > at
> > > this moment.
> > >
> > > What's bothering me is the fact that we already have a way to specify a
> > > PartitionResolver generically built into all our tooling, and now we
> are
> > > adding another way to specify this particular PartitionResolver that
> > could
> > > cause confusion with the current generic way, and could even run into
> > > conflicts that we need no handle specially.
> > >
> > > If we have already taken care of ensuring that the class is on the
> > > classpath, it is probably not really much easier to say
> > > partition-by-prefix=true
> > > than it is to say
> > > partition-resolver=org.apache.geode.somepackage.some.class
> > > and if we don't add "partition-by-prefix" into the API and tooling then
> > > there's no temptation to ever say both.
> > >
> > >
> > >
> > > --
> > > Mike Stolz
> > > Principal Engineer, GemFire Product Manager
> > > Mobile: +1-631-835-4771
> > >
> > > On Thu, Jun 1, 2017 at 4:54 PM, Darrel Schneider <
> dschneider@pivotal.io>
> > > wrote:
> > >
> > > > We already have a general purpose mechanism that, if you are willing
> to
> > > > write java code, handles all the permutations you mention. That
> > mechanism
> > > > is the PartitionResolver interface.
> > > >
> > > > This new feature is intended to give users a way to control how their
> > > keys
> > > > are partitioned without writing any java code. I think by
> "unnecessary
> > > API"
> > > > you mean "unnecessary feature". But our current features do not give
> > you
> > > a
> > > > way to control partitioning without implementing a PartitionResolver.
> > > This
> > > > new feature allows you to just configure your region (using gfsh,
> xml,
> > or
> > > > apis) and then format your string keys to control the partitioning.
> No
> > > need
> > > > to implement a PartitionResolver. I think that is the requirement for
> > > this
> > > > feature. Are you arguing that this requirement does not have enough
> > value
> > > > for our customers to justify adding product feature for it? Or do you
> > > think
> > > > this requirement should be met with a different solution?
> > > >
> > > > On Thu, Jun 1, 2017 at 11:59 AM, Michael Stolz <mstolz@pivotal.io>
> > > wrote:
> > > >
> > > > > I really think that we're going down the wrong road with this
> > > hard-coded
> > > > > partition-by-prefix thing.
> > > > >
> > > > > I expect that we will have several different flavors of pre-built
> > > > compound
> > > > > key classes.
> > > > > We will probably have one that takes two Integers, quite possibly
> one
> > > > with
> > > > > an Integer and a String.
> > > > > There could be an untold number of permutations all of which are
> > > handled
> > > > by
> > > > > the current way of specifying the classname.
> > > > > It also eliminates the possibility of having both
> "partitionByPrefix"
> > > > and a
> > > > > "PartitionResolver".
> > > > >
> > > > > We are introducing an unnecessary API which is introducing YET
> > ANOTHER
> > > > WAY
> > > > > of doing something and causing more confusion.
> > > > >
> > > > > As we are encountering things like both "partitionByPrefix" and a
> > > > > "PartitionResolver" and how to deal with it I am becoming more and
> > more
> > > > > convinced that this is a big mistake.
> > > > >
> > > > >
> > > > > --
> > > > > Mike Stolz
> > > > > Principal Engineer, GemFire Product Manager
> > > > > Mobile: +1-631-835-4771
> > > > >
> > > > > On Wed, May 31, 2017 at 8:03 PM, Fred Krone (JIRA) <
> jira@apache.org>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > >      [ https://issues.apache.org/jira/browse/GEODE-3005?page=
> > > > > > com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
> > > > > >
> > > > > > Fred Krone updated GEODE-3005:
> > > > > > ------------------------------
> > > > > >     Description:
> > > > > > A user should be able to set partition by prefix programmatically
> > > when
> > > > > > creating a partitioned region.
> > > > > >
> > > > > > This can only be done when creating a Region type Partition
> > > > > >
> > > > > > Implement:
> > > > > > PartitionByKey PartitionResolver
> > > > > > String key parser
> > > > > > Getters and setters PartitionAttributesFactory
> > > > > >
> > > > > >
> > > > > > Done:
> > > > > > Code is reviewed and checked in
> > > > > > Tests are created and pass
> > > > > >
> > > > > > Acceptance:
> > > > > > Partitioned region can be created using new 'partition by prefix'
> > > > > > attributes (on/off, delimiter)
> > > > > > Only String keys work -- all other keys throw an error
> > > > > > A String key without a delimiter is valid -- but will not be
> > > colocated
> > > > > > Providing a key with the correct delimiter routes the entry
to
> the
> > > > > correct
> > > > > > node
> > > > > > If partitionByPrefix has been set AND a PartitionResolver is
set
> an
> > > > > > IllegalArgumentException
> > > > > > When partition-by-prefix is 'on' AND the user also sets a
> specific
> > > > > > partition resolver (an implemented class), partitioning should
> > > default
> > > > to
> > > > > > the implemented class.
> > > > > >
> > > > > >
> > > > > >
> > > > > >   was:
> > > > > > A user should be able to set partition by prefix programmatically
> > > when
> > > > > > creating a partitioned region.
> > > > > >
> > > > > > This can only be done when creating a Region type Partition
> > > > > >
> > > > > > Implement:
> > > > > > Cache.xml XSD adding new 'partition by prefix' attribute
> > > > > > Implement XML parser
> > > > > >
> > > > > > Done:
> > > > > > Code is reviewed and checked in
> > > > > > Tests are created and pass
> > > > > >
> > > > > > Acceptance:
> > > > > > A partitioned region can be created using with 'partition by
> > prefix'
> > > > via
> > > > > > cache.xml
> > > > > > Only String keys work -- all other keys throw an error
> > > > > > Providing a key with the correct delimiter routes the entry
to
> the
> > > > > correct
> > > > > > node
> > > > > > Providing a key with no delimiter throws an error
> > > > > > When partition-by-prefix is 'on' AND the user also sets a
> specific
> > > > > > partition resolver (an implemented class), partitioning should
> > > default
> > > > to
> > > > > > the implemented class.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > A developer can create a Region with Partition by Prefix
using
> > Java
> > > > API
> > > > > > > ------------------------------------------------------------
> > > > > -----------
> > > > > > >
> > > > > > >                 Key: GEODE-3005
> > > > > > >                 URL: https://issues.apache.org/
> > > > jira/browse/GEODE-3005
> > > > > > >             Project: Geode
> > > > > > >          Issue Type: Wish
> > > > > > >          Components: regions
> > > > > > >            Reporter: Fred Krone
> > > > > > >            Assignee: Fred Krone
> > > > > > >
> > > > > > > A user should be able to set partition by prefix
> programmatically
> > > > when
> > > > > > creating a partitioned region.
> > > > > > > This can only be done when creating a Region type Partition
> > > > > > > Implement:
> > > > > > > PartitionByKey PartitionResolver
> > > > > > > String key parser
> > > > > > > Getters and setters PartitionAttributesFactory
> > > > > > > Done:
> > > > > > > Code is reviewed and checked in
> > > > > > > Tests are created and pass
> > > > > > > Acceptance:
> > > > > > > Partitioned region can be created using new 'partition
by
> prefix'
> > > > > > attributes (on/off, delimiter)
> > > > > > > Only String keys work -- all other keys throw an error
> > > > > > > A String key without a delimiter is valid -- but will not
be
> > > > colocated
> > > > > > > Providing a key with the correct delimiter routes the entry
to
> > the
> > > > > > correct node
> > > > > > > If partitionByPrefix has been set AND a PartitionResolver
is
> set
> > an
> > > > > > IllegalArgumentException
> > > > > > > When partition-by-prefix is 'on' AND the user also sets
a
> > specific
> > > > > > partition resolver (an implemented class), partitioning should
> > > default
> > > > to
> > > > > > the implemented class.
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > This message was sent by Atlassian JIRA
> > > > > > (v6.3.15#6346)
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message