Mailing-List: contact dev-help@geode.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@geode.apache.org
MIME-Version: 1.0
References: <CACT=EE5T=PUq=sFQBc3wagwo06Cxq7xx5Fek09QRokDZRA-55Q@mail.gmail.com>
 <CAJrNg01X9spD=QDuoi_=TAJcNotiktKBTntHwO9WM8E6x=WY+A@mail.gmail.com>
 <BN6PR10MB1956667268DEE05C55A8ADD4DE1E0@BN6PR10MB1956.namprd10.prod.outlook.com>
 <CACT=EE6=1EKd+Z=5j5KPzkr5XJER9CPnOVuyxD1-A3fhEJBu5Q@mail.gmail.com>
 <CAFh+7k19z4s_J17Yc1Y=mxsWLnVpsLApSN+xg+WUB3JMCsLbyw@mail.gmail.com>
 <CANZq1gDUOEt--dVy5oE8kbsGcgfF=Kvqd2X0+vL4KYLC49X4zw@mail.gmail.com>
 <CACT=EE6JTNqXyXGkzoy9t-gq1=H0EeJFyyzp3VRK=ZQXYgnu8A@mail.gmail.com>
 <CAE8gj24TrYFBev5UryPtA4M8ytcHuvmTnrxKtQ6cFythA3P9DQ@mail.gmail.com>
 <CAA1wVxr4bYRFFyQ7kPBFcksAxevL_1rVRRZ4=qU8wH3B3GYpYA@mail.gmail.com> <CAE8gj24QgdV-x2Hee_uF4L=7jACC=-ENupto4hpMSDreu6S-1A@mail.gmail.com>
In-Reply-To: <CAE8gj24QgdV-x2Hee_uF4L=7jACC=-ENupto4hpMSDreu6S-1A@mail.gmail.com>
From: Jacob Barrett <jbarrett@pivotal.io>
Date: Wed, 26 Apr 2017 15:28:35 +0000
Message-ID: <CAA1wVxq8D-54YnxPFjm6s6xyGO-CjEQ9Cpm_Sf1z=OXH6w9J3g@mail.gmail.com>
Subject: Re: Simple Java Client
To: dev@geode.apache.org
Content-Type: multipart/alternative; boundary=001a113e79e810330d054e137f09
archived-at: Wed, 26 Apr 2017 15:28:57 -0000

--001a113e79e810330d054e137f09
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Wes,

Those are almost all administrative commands and have no place on the
client API. They belong on an administrative API or as I'm arguing a series
of MBeans/JMX as it is already an established standard.

-Jake

On Wed, Apr 26, 2017 at 8:09 AM Wes Williams <wwilliams@pivotal.io> wrote:

> Now we're getting some precision. Let's talk about the "raw" Geode
> proprietary bad ass API!  Would that "raw" Geode proprietary bad ass API
> "raw"
> Geode proprietary bad ass API that we're talking about be centered around
> the commands found here:
>
> https://github.com/apache/geode/tree/rel/v1.1.1/geode-core/src/main/java/=
org/apache/geode/management/internal/cli/commands
>
> Or somewhere else?
>
> *Wes Williams | Pivotal Advisory **Data Engineer*
> 781.606.0325
> http://pivotal.io/big-data/pivotal-gemfire
>
> On Tue, Apr 25, 2017 at 11:41 PM, Jacob Barrett <jbarrett@pivotal.io>
> wrote:
>
> > Java and its community have standards for all of these issue so why
> > re-invent the wheel. The market doesn't want proprietary anymore, they
> want
> > standards and mobility.
> >
> > Configuration of the server should happen through MBeans. You can wrap
> that
> > in gfsh for command line, REST for remote web based admin, use JConsole
> or
> > any other number of JMX based enterprise management tools. By using
> MBeans
> > the server can easily expose new discovered services without the need t=
o
> > code specific gfsh commands, REST interfaces or APIs. There is no reaso=
n
> my
> > SDG can't be retooled to "discover" the configuration from these MBeans
> as
> > well rather than having to be touched every time we add or change
> > something. There are tools and books already written that implementors
> can
> > consult on MBeans. There isn't anything out there on gfsh commands.
> >
> > If we want to play in the Java community, especially J2EE (the other 50=
%
> of
> > Java that isn't Spring), then we had better have a JSR-107 answer no
> matter
> > what the pain is to provide it. I can pull dozens of books of the shelf
> > that teach me how to effectively use a JCache, how many can I pull off
> the
> > shelf that teach me Geode's API? How many engineers can I get
> applications
> > form by saying "must have Geode API knowledge"? I can find people with
> > JCache knowledge though. So from in implementor's perspective having
> > standards is a must. Now I don't think the JSR-107 interface should be
> the
> > root interface but rather a facade on the "raw" Geode proprietary bad a=
ss
> > API. That API should be 100% asynchronous (reactive, SEDA, whatever the
> > current buzzword for asynchronous is today). Around that API we can
> provide
> > facades for JSR 107, ConcurrentMap (our current yet not so well behavin=
g
> > API), List, Queue, etc. Maybe even JPA, JCA, etc. The thought of puttin=
g
> > all those features into a single API makes my head exploded and they
> don't
> > need to be like they are today.
> >
> >
> >
> > On Tue, Apr 25, 2017 at 8:25 PM Wes Williams <wwilliams@pivotal.io>
> wrote:
> >
> > > A couple of points to interact with John's points.
> > >
> > > GFSH as API
> > >
> > > We agree that GFSH is a DSL, and a really good and useful one. We agr=
ee
> > > that we don't want our API tied to GFSH syntax. I view the valuable
> part
> > of
> > > GemFire admin as the Java code underneath GFSH, or the "Commands."
> > >
> > > For example, to create a JAVA API to "Create Region",  why not create=
 a
> > > Java API around CreateAlterDestroyRegionCommands? In this way, GFSH a=
nd
> > the
> > > JAVA API share the same code. It seems too obvious yet I don't see it
> > being
> > > recommended.  Why not?  (Note: the hard-coded formatting would need t=
o
> be
> > > removed).
> > >
> > > Once you have the Java/GFSH/REST API as common code, you then refacto=
r
> > it.
> > > What's the objection to this approach? Once you open Java API's to do
> > > everything that GFSH does, then you have now unshackled the power of
> the
> > > developer (the JAVA API) and the admin (GFSH API).
> > >
> > >
> > > REST API
> > >
> > > I've found that most don't want to use the Dev REST API because it's
> > > attached to a server rather than the cluster. HA?
> > >
> > >
> > > *Wes Williams | Pivotal Advisory **Data Engineer*
> > > 781.606.0325 <(781)%20606-0325>
> > > http://pivotal.io/big-data/pivotal-gemfire
> > >
> > > On Tue, Apr 25, 2017 at 7:01 PM, Fred Krone <fkrone@pivotal.io> wrote=
:
> > >
> > > > Good feedback.
> > > >
> > > > This would use the new protocol.  I should have mentioned that.
> > > >
> > > > The original driver for this was the Region API needs either an
> update
> > or
> > > > an alternative.  Updating has a few drawbacks: Region wasn't design=
ed
> > > with
> > > > open source in-mind and as Swap mentioned it is naturally tightly
> > > coupled.
> > > > Members of the community are already working to update Region but
> > > something
> > > > gets fixed and it breaks something for someone else.  I think it's
> much
> > > > better to provide a new interface that implements the first part of
> JSR
> > > 107
> > > > (javax.cache) and get the ball rolling for the community and,
> perhaps,
> > > over
> > > > time deprecate Region (although that's not a primary objective).
> > > >
> > > > A Java driver will probably get built regardless just to give the n=
ew
> > > > protocol some legs. That driver also needs a decent caching
> interface.
> > > JSR
> > > > 107 has already solved that part.  So let's get started on it.  If
> the
> > > > community wants to go the whole way and continue JSR 107
> implementation
> > > > after that that's awesome.  Functions can also be added, etc.
> > > >
> > > > I intentionally did not mention anything about PCF as this pertains
> to
> > > > Geode itself as an open source offering and developer experience.
> I'm
> > > > writing as a member of the community. Ie: I'm a developer who would
> > like
> > > to
> > > > add some caching to my application -- I can download either Geode o=
r
> > > > Hazelcast for free and right now it's a no brainer.  Not that we
> > wouldn't
> > > > keep PCF in-mind but it's out of scope for this thread.  I do belie=
ve
> > > > getting started on a Java driver for the protocol and a standardize=
d
> > > > caching API are easily leveraged wins across the board.
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Apr 25, 2017 at 3:20 PM, Swapnil Bawaskar <
> > sbawaskar@pivotal.io>
> > > > wrote:
> > > >
> > > > > I had looked at the JCache in the past and here are some of the
> > things
> > > I
> > > > > noted:
> > > > >
> > > > > Naming convention: Geode's region is a Cache in JSR-107, and
> Geode's
> > > > Cache
> > > > > is a CacheManager. I think it would be too confusing to change th=
e
> > > > meaning
> > > > > of cache. Also, how do you document this given that Cache means
> > > different
> > > > > things if you are talking JCache vs Geode.
> > > > >
> > > > > The way to create a Cache in JSR-107 is:
> > > > > Cache<Integer, Date> cache =3D manager.createCache(cacheName,
> > > Configuration
> > > > > c);
> > > > > Where it is upto the implementation to extend Configuration. Give=
n
> > > this,
> > > > > users will not be able to switch from an existing implementation =
to
> > > ours;
> > > > > will have to write new code specially Configuration, making
> callbacks
> > > > > serializable etc.
> > > > >
> > > > > JSR-107 will not be limited to the client. Server side callbacks
> like
> > > > > CacheLoader, CacheListener etc. will need handle on jsr-107
> =E2=80=9Ccache=E2=80=9D.
> > > > >
> > > > > JSR-107 supports features like an EntryProcessor, which is a
> function
> > > > > invoked atomically on an entry operation. We will have to make
> > invasive
> > > > > changes to Geode to support this.
> > > > >
> > > > > Given these, I don't think supporting JSR-107 is trivial.
> > > > >
> > > > > On Tue, Apr 25, 2017 at 2:55 PM Dan Smith <dsmith@pivotal.io>
> wrote:
> > > > >
> > > > > > What transport are you planning on using? REST, or the current
> > binary
> > > > > > protocol? Or is this just a wrapper around the existing java
> client
> > > > APIs?
> > > > > >
> > > > > > If this about creating a new API, I agree with what John is
> saying
> > > that
> > > > > we
> > > > > > need reduce the number of APIs we have to do the same thing in
> > java.
> > > It
> > > > > > seems especially confusing if we end up with different APIs tha=
t
> > > > support
> > > > > > distinct features - like being able to create a region on the
> > server
> > > > with
> > > > > > this new API but only being able to invoke a function with the
> old
> > > API.
> > > > > >
> > > > > > The other thing I think we need to avoid is being able to do
> things
> > > > from
> > > > > > the client (or gfsh, or xml, ...) that don't have a java API on
> the
> > > > > server.
> > > > > > I'm assuming your getOrCreateRegion region is supposed behave
> like
> > > the
> > > > > gfsh
> > > > > > command and update the cluster configuration? +++1 to Wes's
> > > suggestion
> > > > > have
> > > > > > a public API for executing these gfsh operations.
> > > > > >
> > > > > > I really think we need to work on having *one* authoritative
> public
> > > API
> > > > > > that contains everything that we support on the server. The
> > protocols
> > > > we
> > > > > > support (REST, binary, ...) and the client drivers that use tho=
se
> > > > > protocols
> > > > > > should just be ways of accessing that API. The Java API on the
> > server
> > > > > right
> > > > > > now the closest thing we have to this, but there are things you
> can
> > > do
> > > > > with
> > > > > > gfsh and things you can do with the current client that have no
> > java
> > > > API
> > > > > > equivalent. I think we really need to fix that!
> > > > > >
> > > > > > Also, preferably we won't have to hand code a bunch of stuff in
> > every
> > > > > > client driver and protocol whenever we add a new feature. For
> > example
> > > > if
> > > > > > were to add a geode function to invoke each new API we add, tha=
t
> > new
> > > > API
> > > > > > would be accessible from REST, gfsh, the java client etc. Later
> we
> > > > could
> > > > > > add syntatic sugar to make the new API prettier, but if we had =
a
> > low
> > > > > level
> > > > > > API that automatically exposed all new features that would make
> > > adding
> > > > > new
> > > > > > features much less painful.
> > > > > >
> > > > > > I do like the idea of adding a reactive API.
> > > > > >
> > > > > >  Supporting JSR-107 might be nice - but that could probably be
> > > written
> > > > > just
> > > > > > as a wrapper around the current API without too much work. I
> don't
> > > > think
> > > > > we
> > > > > > should do anything for JSR-107 other than provide a JSR-107
> > > compatible
> > > > > > wrapper - if someone wants additional geode specific features
> they
> > > > should
> > > > > > drop into the existing API.
> > > > > >
> > > > > > I also like the idea of a smaller client jar and dependencies,
> but
> > I
> > > > > think
> > > > > > there is a huge amount of room for improvement in our existing
> > client
> > > > > just
> > > > > > by refactoring the code a bit more and shinking geode-core into=
 a
> > > bare
> > > > > > minimum.
> > > > > >
> > > > > > -Dan
> > > > > >
> > > > > > On Mon, Apr 24, 2017 at 8:20 PM, Fred Krone <fkrone@pivotal.io>
> > > wrote:
> > > > > >
> > > > > > > That's great, Wes.  I will take a look.  Thanks.
> > > > > > >
> > > > > > > John -- good feedback to consider and I was hoping this would
> > come
> > > > up.
> > > > > > >
> > > > > > > "In my mind, there really are only 2 approaches... *Spring* a=
nd
> > > > > > > non-*Spring*,
> > > > > > > or rather PCF and non-PCF, and since PCF is primarily based o=
n
> > Boot
> > > > > > (given
> > > > > > > Microservices/applications are the new concurrency), then I a=
m
> > > really
> > > > > > > saying the same thing."
> > > > > > >
> > > > > > > This would be cover non spring and attempt to give the
> community
> > > > > > something
> > > > > > > that had the same standardized experience as JSR 107 --
> starting
> > > with
> > > > > the
> > > > > > > Cache interface itself.  Although we don't necessarily have t=
o
> > > > > implement
> > > > > > > Cache, it's method signatures are essentially a (non spring)
> > > caching
> > > > > > > standard for Java developers at this point.   We considered
> full
> > > > blown
> > > > > > JSR
> > > > > > > 107 implementation but thought it was too robust for the need=
s
> > > > > mentioned
> > > > > > > (that's not to say we couldn't get there incrementally).  I
> think
> > > > those
> > > > > > > needs still exist open-source outside of PCF and don't
> > cannibalize.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Apr 24, 2017 at 7:44 PM, Real Wes <
> > TheRealWes@outlook.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Here is a simple Java client _for enterprise use_ that I
> > > developed
> > > > > for
> > > > > > > > Geode and distributed to several enterprises. It has
> > similarities
> > > > and
> > > > > > > > differences for your goal. This project creates both server
> and
> > > > > client
> > > > > > > > regions dynamically.  It lists regions, alters regions=E2=
=80=A6
> really
> > > > > anything
> > > > > > > > that GFSH can do. It differs in that it calls GFSH to do it=
s
> > work
> > > > > > rather
> > > > > > > > than a Java interface. You can think of this as a Java API =
on
> > top
> > > > of
> > > > > > > GFSH.
> > > > > > > >
> > > > > > > > You can use this model and keep the similarities while
> > adjusting
> > > > for
> > > > > > the
> > > > > > > > Java interface.
> > > > > > > >
> > > > > > > > Similarities
> > > > > > > > - Client uses SDG and complies with JSR-107
> > > > > > > >      [the Cache Manager is here: https://github.com/Pivotal=
-
> > > > > > > > Data-Engineering/ClusterManagement/tree/master/
> > > > > > > >
> cluster-management-client/src/main/java/com/geode/cache/manager
> > ]
> > > > > > > > - After the server creates its region, client creates its
> > region
> > > > > > > >      [ see createRegions at: https://github.com/Pivotal-
> > > > > > > Data-Engineering/
> > > > > > > > ClusterManagement/blob/master/cluster-management-
> > > > > > > >
> > > > > > client/src/main/java/com/geode/cache/manager/
> > > > RegionCreator.java<https://
> > > > > > > > github.com/Pivotal-Data-Engineering/ClusterManagement/
> > > > > > > blob/master/cluster-
> > > > > > > > management-client/src/main/java/com/geode/cache/manager/
> > > > > > > RegionCreator.java>
> > > > > > > > ]
> > > > > > > >
> > > > > > > > Differences
> > > > > > > > Server dynamically calls GFSH to execute commands
> > > > > > > >
> > > > > > > > How it works
> > > > > > > > - Whenever the client calls a region that does not exist, t=
he
> > > > client
> > > > > > > looks
> > > > > > > > for a <region name>.properties file. If the properties file
> > > exists,
> > > > > the
> > > > > > > > region is created with the dynamic properties.
> > > > > > > > - If <region name>.properties does not exist, default regio=
n
> > > > > properties
> > > > > > > > are loaded and used to create the region.
> > > > > > > > - properties are formed into a GFSH string and passed to a
> > server
> > > > > > > function
> > > > > > > > - The server function calls GFSH using internal calls. It
> calls
> > > the
> > > > > > > > GfshParser, executes the command, and then returns the pars=
ed
> > > > > results.
> > > > > > > > [see https://github.com/Pivotal-Data-Engineering/
> > > > > > > > ClusterManagement/blob/master/cluster-management-server/src=
/
> > > > > > > > main/java/com/geode/gfsh/GfshCommand.java]
> > > > > > > >
> > > > > > > > Limitations
> > > > > > > > It uses internal calls to call GFSH. I have made requests t=
o
> > make
> > > > > this
> > > > > > a
> > > > > > > > stable public interface. Andrew Baker had a good idea to
> return
> > > > gfsh
> > > > > > > > results in JSON format with a new =E2=80=94results=3Djson p=
roperty to
> > pass
> > > to
> > > > > > GFSH.
> > > > > > > >
> > > > > > > > Strengths
> > > > > > > > That is uses GFSH can be viewed as a strength to keep a
> > > consistent
> > > > > API,
> > > > > > > > whether Java or GFSH
> > > > > > > >
> > > > > > > > TESTS
> > > > > > > > You can start by running server tests at:
> > > > > https://github.com/Pivotal-
> > > > > > > > Data-Engineering/ClusterManagement/tree/master/
> > > > > > > > cluster-management-server/src/test/java/com/geode/gfsh
> > > > > > > >
> > > > > > > > Client tests are at:
> > > https://github.com/Pivotal-Data-Engineering/
> > > > > > > > ClusterManagement/tree/master/cluster-management-client/src=
/
> > > > > > > > test/java/com/geode/management/client
> > > > > > > >
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Wes Williams
> > > > > > > >
> > > > > > > > On Apr 24, 2017, at 6:51 PM, Kirk Lund <klund@apache.org
> > <mailto:
> > > > > klund
> > > > > > > > @apache.org>> wrote:
> > > > > > > >
> > > > > > > > A couple things I'd like to see:
> > > > > > > >
> > > > > > > > 1) completely new API that doesn't reuse the old API classe=
s
> > (or
> > > at
> > > > > > least
> > > > > > > > not the giant classes such as Cache and Region interfaces)
> > > > > > > > 2) separation of API and Impl so that users can compile the=
ir
> > > code
> > > > > > > against
> > > > > > > > a dedicated client API jar
> > > > > > > >
> > > > > > > > On Mon, Apr 24, 2017 at 3:03 PM, Fred Krone <
> fkrone@pivotal.io
> > > > > <mailto:
> > > > > > > fkro
> > > > > > > > ne@pivotal.io>> wrote:
> > > > > > > >
> > > > > > > > In an effort to improve Java developer ease of use for
> caching
> > > with
> > > > > > > Geode I
> > > > > > > > am looking for feedback on going forward with creating a Ja=
va
> > > > client.
> > > > > > > This
> > > > > > > > client will allow for server-side region creation and
> > distributed
> > > > > data
> > > > > > > > caching.  This would allow for a thin client that fits with
> > > > > > microservice
> > > > > > > > caching patterns and also abstracts a cleaner client-server
> > > > > experience
> > > > > > > > driven interface.
> > > > > > > >
> > > > > > > > Initially we were going to update the Region interface but
> were
> > > > > > concerned
> > > > > > > > with breaking existing applications.  We also would like to
> > > provide
> > > > > > > Region
> > > > > > > > creation to a client application and so propose here solvin=
g
> > both
> > > > of
> > > > > > > these
> > > > > > > > areas with a Java client.
> > > > > > > >
> > > > > > > > It would have new project repo for the client.
> > > > > > > >
> > > > > > > > It would provide new Region interface for clients.  The
> > specifics
> > > > of
> > > > > > the
> > > > > > > > API design are too lengthy for this conversation but
> > > implementation
> > > > > > will
> > > > > > > > resemble JSR 107 Cache interface.
> > > > > > > >
> > > > > > > > It would use the new security framework.
> > > > > > > >
> > > > > > > >
> > > > > > > > *An example*,
> > > > > > > >
> > > > > > > > The client application simply creates an instance of client
> and
> > > > > points
> > > > > > it
> > > > > > > > to the locator:
> > > > > > > >
> > > > > > > >
> > > > > > > > org.apache.geode.client.Client client =3D
> > > Client.create(locatorHost,
> > > > > > > > locatorPort);
> > > > > > > >
> > > > > > > >
> > > > > > > > Client has the following methods:
> > > > > > > >
> > > > > > > > package org.apache.geode.client;
> > > > > > > >
> > > > > > > >
> > > > > > > > public interface GeodeClient {
> > > > > > > >
> > > > > > > >  /**
> > > > > > > >
> > > > > > > >   * creates the region on the servers, or gets the region i=
f
> it
> > > > > exits,
> > > > > > > > returns a PROXY region
> > > > > > > >
> > > > > > > >   */
> > > > > > > >
> > > > > > > >  public Region getOrCreateRegion(RegionAttributes attribute=
s,
> > > > String
> > > > > > > > name);
> > > > > > > >
> > > > > > > >
> > > > > > > >  /**
> > > > > > > >
> > > > > > > >   * Returns a PROXY region if the region exists on the serv=
er
> > > > > > > >
> > > > > > > >   */
> > > > > > > >
> > > > > > > >  public Region getRegion(String name);
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > MVP
> > > > > > > >
> > > > > > > > The smallest set of features to test this idea, learn and
> > > iterate,
> > > > > and
> > > > > > > get
> > > > > > > > the client into the communities hands for future iterations
> is:
> > > > > > > >
> > > > > > > >
> > > > > > > > Create a server side Region from a client
> > > > > > > >
> > > > > > > > Region interface has CRUD on par with javax.cache.Cache (fr=
om
> > JSR
> > > > > 107)
> > > > > > > >
> > > > > > > > Calls are asynchronous -- futures
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Also would like feedback on which future functionality woul=
d
> be
> > > > most
> > > > > > > useful
> > > > > > > > from a thin client:
> > > > > > > >
> > > > > > > > Function execution
> > > > > > > >
> > > > > > > > Durable clients
> > > > > > > >
> > > > > > > > User defined serialization
> > > > > > > >
> > > > > > > > Register interest
> > > > > > > >
> > > > > > > > Queries
> > > > > > > >
> > > > > > > > CQ
> > > > > > > >
> > > > > > > > Near side caching
> > > > > > > >
> > > > > > > > Create disk stores from client
> > > > > > > >
> > > > > > > > Region group membership
> > > > > > > >
> > > > > > > > Client subscription load balancing
> > > > > > > > Transactions
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > -Fred
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

--001a113e79e810330d054e137f09--