geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Barrett <jbarr...@pivotal.io>
Subject Re: Simple Java Client
Date Wed, 26 Apr 2017 03:41:32 GMT
Java and its community have standards for all of these issue so why
re-invent the wheel. The market doesn't want proprietary anymore, they want
standards and mobility.

Configuration of the server should happen through MBeans. You can wrap that
in gfsh for command line, REST for remote web based admin, use JConsole or
any other number of JMX based enterprise management tools. By using MBeans
the server can easily expose new discovered services without the need to
code specific gfsh commands, REST interfaces or APIs. There is no reason my
SDG can't be retooled to "discover" the configuration from these MBeans as
well rather than having to be touched every time we add or change
something. There are tools and books already written that implementors can
consult on MBeans. There isn't anything out there on gfsh commands.

If we want to play in the Java community, especially J2EE (the other 50% of
Java that isn't Spring), then we had better have a JSR-107 answer no matter
what the pain is to provide it. I can pull dozens of books of the shelf
that teach me how to effectively use a JCache, how many can I pull off the
shelf that teach me Geode's API? How many engineers can I get applications
form by saying "must have Geode API knowledge"? I can find people with
JCache knowledge though. So from in implementor's perspective having
standards is a must. Now I don't think the JSR-107 interface should be the
root interface but rather a facade on the "raw" Geode proprietary bad ass
API. That API should be 100% asynchronous (reactive, SEDA, whatever the
current buzzword for asynchronous is today). Around that API we can provide
facades for JSR 107, ConcurrentMap (our current yet not so well behaving
API), List, Queue, etc. Maybe even JPA, JCA, etc. The thought of putting
all those features into a single API makes my head exploded and they don't
need to be like they are today.



On Tue, Apr 25, 2017 at 8:25 PM Wes Williams <wwilliams@pivotal.io> wrote:

> A couple of points to interact with John's points.
>
> GFSH as API
>
> We agree that GFSH is a DSL, and a really good and useful one. We agree
> that we don't want our API tied to GFSH syntax. I view the valuable part of
> GemFire admin as the Java code underneath GFSH, or the "Commands."
>
> For example, to create a JAVA API to "Create Region",  why not create a
> Java API around CreateAlterDestroyRegionCommands? In this way, GFSH and the
> JAVA API share the same code. It seems too obvious yet I don't see it being
> recommended.  Why not?  (Note: the hard-coded formatting would need to be
> removed).
>
> Once you have the Java/GFSH/REST API as common code, you then refactor it.
> What's the objection to this approach? Once you open Java API's to do
> everything that GFSH does, then you have now unshackled the power of the
> developer (the JAVA API) and the admin (GFSH API).
>
>
> REST API
>
> I've found that most don't want to use the Dev REST API because it's
> attached to a server rather than the cluster. HA?
>
>
> *Wes Williams | Pivotal Advisory **Data Engineer*
> 781.606.0325 <(781)%20606-0325>
> http://pivotal.io/big-data/pivotal-gemfire
>
> On Tue, Apr 25, 2017 at 7:01 PM, Fred Krone <fkrone@pivotal.io> wrote:
>
> > Good feedback.
> >
> > This would use the new protocol.  I should have mentioned that.
> >
> > The original driver for this was the Region API needs either an update or
> > an alternative.  Updating has a few drawbacks: Region wasn't designed
> with
> > open source in-mind and as Swap mentioned it is naturally tightly
> coupled.
> > Members of the community are already working to update Region but
> something
> > gets fixed and it breaks something for someone else.  I think it's much
> > better to provide a new interface that implements the first part of JSR
> 107
> > (javax.cache) and get the ball rolling for the community and, perhaps,
> over
> > time deprecate Region (although that's not a primary objective).
> >
> > A Java driver will probably get built regardless just to give the new
> > protocol some legs. That driver also needs a decent caching interface.
> JSR
> > 107 has already solved that part.  So let's get started on it.  If the
> > community wants to go the whole way and continue JSR 107 implementation
> > after that that's awesome.  Functions can also be added, etc.
> >
> > I intentionally did not mention anything about PCF as this pertains to
> > Geode itself as an open source offering and developer experience.  I'm
> > writing as a member of the community. Ie: I'm a developer who would like
> to
> > add some caching to my application -- I can download either Geode or
> > Hazelcast for free and right now it's a no brainer.  Not that we wouldn't
> > keep PCF in-mind but it's out of scope for this thread.  I do believe
> > getting started on a Java driver for the protocol and a standardized
> > caching API are easily leveraged wins across the board.
> >
> >
> >
> >
> > On Tue, Apr 25, 2017 at 3:20 PM, Swapnil Bawaskar <sbawaskar@pivotal.io>
> > wrote:
> >
> > > I had looked at the JCache in the past and here are some of the things
> I
> > > noted:
> > >
> > > Naming convention: Geode's region is a Cache in JSR-107, and Geode's
> > Cache
> > > is a CacheManager. I think it would be too confusing to change the
> > meaning
> > > of cache. Also, how do you document this given that Cache means
> different
> > > things if you are talking JCache vs Geode.
> > >
> > > The way to create a Cache in JSR-107 is:
> > > Cache<Integer, Date> cache = manager.createCache(cacheName,
> Configuration
> > > c);
> > > Where it is upto the implementation to extend Configuration. Given
> this,
> > > users will not be able to switch from an existing implementation to
> ours;
> > > will have to write new code specially Configuration, making callbacks
> > > serializable etc.
> > >
> > > JSR-107 will not be limited to the client. Server side callbacks like
> > > CacheLoader, CacheListener etc. will need handle on jsr-107 “cache”.
> > >
> > > JSR-107 supports features like an EntryProcessor, which is a function
> > > invoked atomically on an entry operation. We will have to make invasive
> > > changes to Geode to support this.
> > >
> > > Given these, I don't think supporting JSR-107 is trivial.
> > >
> > > On Tue, Apr 25, 2017 at 2:55 PM Dan Smith <dsmith@pivotal.io> wrote:
> > >
> > > > What transport are you planning on using? REST, or the current binary
> > > > protocol? Or is this just a wrapper around the existing java client
> > APIs?
> > > >
> > > > If this about creating a new API, I agree with what John is saying
> that
> > > we
> > > > need reduce the number of APIs we have to do the same thing in java.
> It
> > > > seems especially confusing if we end up with different APIs that
> > support
> > > > distinct features - like being able to create a region on the server
> > with
> > > > this new API but only being able to invoke a function with the old
> API.
> > > >
> > > > The other thing I think we need to avoid is being able to do things
> > from
> > > > the client (or gfsh, or xml, ...) that don't have a java API on the
> > > server.
> > > > I'm assuming your getOrCreateRegion region is supposed behave like
> the
> > > gfsh
> > > > command and update the cluster configuration? +++1 to Wes's
> suggestion
> > > have
> > > > a public API for executing these gfsh operations.
> > > >
> > > > I really think we need to work on having *one* authoritative public
> API
> > > > that contains everything that we support on the server. The protocols
> > we
> > > > support (REST, binary, ...) and the client drivers that use those
> > > protocols
> > > > should just be ways of accessing that API. The Java API on the server
> > > right
> > > > now the closest thing we have to this, but there are things you can
> do
> > > with
> > > > gfsh and things you can do with the current client that have no java
> > API
> > > > equivalent. I think we really need to fix that!
> > > >
> > > > Also, preferably we won't have to hand code a bunch of stuff in every
> > > > client driver and protocol whenever we add a new feature. For example
> > if
> > > > were to add a geode function to invoke each new API we add, that new
> > API
> > > > would be accessible from REST, gfsh, the java client etc. Later we
> > could
> > > > add syntatic sugar to make the new API prettier, but if we had a low
> > > level
> > > > API that automatically exposed all new features that would make
> adding
> > > new
> > > > features much less painful.
> > > >
> > > > I do like the idea of adding a reactive API.
> > > >
> > > >  Supporting JSR-107 might be nice - but that could probably be
> written
> > > just
> > > > as a wrapper around the current API without too much work. I don't
> > think
> > > we
> > > > should do anything for JSR-107 other than provide a JSR-107
> compatible
> > > > wrapper - if someone wants additional geode specific features they
> > should
> > > > drop into the existing API.
> > > >
> > > > I also like the idea of a smaller client jar and dependencies, but I
> > > think
> > > > there is a huge amount of room for improvement in our existing client
> > > just
> > > > by refactoring the code a bit more and shinking geode-core into a
> bare
> > > > minimum.
> > > >
> > > > -Dan
> > > >
> > > > On Mon, Apr 24, 2017 at 8:20 PM, Fred Krone <fkrone@pivotal.io>
> wrote:
> > > >
> > > > > That's great, Wes.  I will take a look.  Thanks.
> > > > >
> > > > > John -- good feedback to consider and I was hoping this would come
> > up.
> > > > >
> > > > > "In my mind, there really are only 2 approaches... *Spring* and
> > > > > non-*Spring*,
> > > > > or rather PCF and non-PCF, and since PCF is primarily based on Boot
> > > > (given
> > > > > Microservices/applications are the new concurrency), then I am
> really
> > > > > saying the same thing."
> > > > >
> > > > > This would be cover non spring and attempt to give the community
> > > > something
> > > > > that had the same standardized experience as JSR 107 -- starting
> with
> > > the
> > > > > Cache interface itself.  Although we don't necessarily have to
> > > implement
> > > > > Cache, it's method signatures are essentially a (non spring)
> caching
> > > > > standard for Java developers at this point.   We considered full
> > blown
> > > > JSR
> > > > > 107 implementation but thought it was too robust for the needs
> > > mentioned
> > > > > (that's not to say we couldn't get there incrementally).  I think
> > those
> > > > > needs still exist open-source outside of PCF and don't cannibalize.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Apr 24, 2017 at 7:44 PM, Real Wes <TheRealWes@outlook.com>
> > > > wrote:
> > > > >
> > > > > >
> > > > > > Here is a simple Java client _for enterprise use_ that I
> developed
> > > for
> > > > > > Geode and distributed to several enterprises. It has similarities
> > and
> > > > > > differences for your goal. This project creates both server
and
> > > client
> > > > > > regions dynamically.  It lists regions, alters regions… really
> > > anything
> > > > > > that GFSH can do. It differs in that it calls GFSH to do its
work
> > > > rather
> > > > > > than a Java interface. You can think of this as a Java API on
top
> > of
> > > > > GFSH.
> > > > > >
> > > > > > You can use this model and keep the similarities while adjusting
> > for
> > > > the
> > > > > > Java interface.
> > > > > >
> > > > > > Similarities
> > > > > > - Client uses SDG and complies with JSR-107
> > > > > >      [the Cache Manager is here: https://github.com/Pivotal-
> > > > > > Data-Engineering/ClusterManagement/tree/master/
> > > > > > cluster-management-client/src/main/java/com/geode/cache/manager
]
> > > > > > - After the server creates its region, client creates its region
> > > > > >      [ see createRegions at: https://github.com/Pivotal-
> > > > > Data-Engineering/
> > > > > > ClusterManagement/blob/master/cluster-management-
> > > > > >
> > > > client/src/main/java/com/geode/cache/manager/
> > RegionCreator.java<https://
> > > > > > github.com/Pivotal-Data-Engineering/ClusterManagement/
> > > > > blob/master/cluster-
> > > > > > management-client/src/main/java/com/geode/cache/manager/
> > > > > RegionCreator.java>
> > > > > > ]
> > > > > >
> > > > > > Differences
> > > > > > Server dynamically calls GFSH to execute commands
> > > > > >
> > > > > > How it works
> > > > > > - Whenever the client calls a region that does not exist, the
> > client
> > > > > looks
> > > > > > for a <region name>.properties file. If the properties
file
> exists,
> > > the
> > > > > > region is created with the dynamic properties.
> > > > > > - If <region name>.properties does not exist, default
region
> > > properties
> > > > > > are loaded and used to create the region.
> > > > > > - properties are formed into a GFSH string and passed to a server
> > > > > function
> > > > > > - The server function calls GFSH using internal calls. It calls
> the
> > > > > > GfshParser, executes the command, and then returns the parsed
> > > results.
> > > > > > [see https://github.com/Pivotal-Data-Engineering/
> > > > > > ClusterManagement/blob/master/cluster-management-server/src/
> > > > > > main/java/com/geode/gfsh/GfshCommand.java]
> > > > > >
> > > > > > Limitations
> > > > > > It uses internal calls to call GFSH. I have made requests to
make
> > > this
> > > > a
> > > > > > stable public interface. Andrew Baker had a good idea to return
> > gfsh
> > > > > > results in JSON format with a new —results=json property to
pass
> to
> > > > GFSH.
> > > > > >
> > > > > > Strengths
> > > > > > That is uses GFSH can be viewed as a strength to keep a
> consistent
> > > API,
> > > > > > whether Java or GFSH
> > > > > >
> > > > > > TESTS
> > > > > > You can start by running server tests at:
> > > https://github.com/Pivotal-
> > > > > > Data-Engineering/ClusterManagement/tree/master/
> > > > > > cluster-management-server/src/test/java/com/geode/gfsh
> > > > > >
> > > > > > Client tests are at:
> https://github.com/Pivotal-Data-Engineering/
> > > > > > ClusterManagement/tree/master/cluster-management-client/src/
> > > > > > test/java/com/geode/management/client
> > > > > >
> > > > > >
> > > > > > Regards,
> > > > > > Wes Williams
> > > > > >
> > > > > > On Apr 24, 2017, at 6:51 PM, Kirk Lund <klund@apache.org<mailto:
> > > klund
> > > > > > @apache.org>> wrote:
> > > > > >
> > > > > > A couple things I'd like to see:
> > > > > >
> > > > > > 1) completely new API that doesn't reuse the old API classes
(or
> at
> > > > least
> > > > > > not the giant classes such as Cache and Region interfaces)
> > > > > > 2) separation of API and Impl so that users can compile their
> code
> > > > > against
> > > > > > a dedicated client API jar
> > > > > >
> > > > > > On Mon, Apr 24, 2017 at 3:03 PM, Fred Krone <fkrone@pivotal.io
> > > <mailto:
> > > > > fkro
> > > > > > ne@pivotal.io>> wrote:
> > > > > >
> > > > > > In an effort to improve Java developer ease of use for caching
> with
> > > > > Geode I
> > > > > > am looking for feedback on going forward with creating a Java
> > client.
> > > > > This
> > > > > > client will allow for server-side region creation and distributed
> > > data
> > > > > > caching.  This would allow for a thin client that fits with
> > > > microservice
> > > > > > caching patterns and also abstracts a cleaner client-server
> > > experience
> > > > > > driven interface.
> > > > > >
> > > > > > Initially we were going to update the Region interface but were
> > > > concerned
> > > > > > with breaking existing applications.  We also would like to
> provide
> > > > > Region
> > > > > > creation to a client application and so propose here solving
both
> > of
> > > > > these
> > > > > > areas with a Java client.
> > > > > >
> > > > > > It would have new project repo for the client.
> > > > > >
> > > > > > It would provide new Region interface for clients.  The specifics
> > of
> > > > the
> > > > > > API design are too lengthy for this conversation but
> implementation
> > > > will
> > > > > > resemble JSR 107 Cache interface.
> > > > > >
> > > > > > It would use the new security framework.
> > > > > >
> > > > > >
> > > > > > *An example*,
> > > > > >
> > > > > > The client application simply creates an instance of client
and
> > > points
> > > > it
> > > > > > to the locator:
> > > > > >
> > > > > >
> > > > > > org.apache.geode.client.Client client =
> Client.create(locatorHost,
> > > > > > locatorPort);
> > > > > >
> > > > > >
> > > > > > Client has the following methods:
> > > > > >
> > > > > > package org.apache.geode.client;
> > > > > >
> > > > > >
> > > > > > public interface GeodeClient {
> > > > > >
> > > > > >  /**
> > > > > >
> > > > > >   * creates the region on the servers, or gets the region if
it
> > > exits,
> > > > > > returns a PROXY region
> > > > > >
> > > > > >   */
> > > > > >
> > > > > >  public Region getOrCreateRegion(RegionAttributes attributes,
> > String
> > > > > > name);
> > > > > >
> > > > > >
> > > > > >  /**
> > > > > >
> > > > > >   * Returns a PROXY region if the region exists on the server
> > > > > >
> > > > > >   */
> > > > > >
> > > > > >  public Region getRegion(String name);
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > MVP
> > > > > >
> > > > > > The smallest set of features to test this idea, learn and
> iterate,
> > > and
> > > > > get
> > > > > > the client into the communities hands for future iterations
is:
> > > > > >
> > > > > >
> > > > > > Create a server side Region from a client
> > > > > >
> > > > > > Region interface has CRUD on par with javax.cache.Cache (from
JSR
> > > 107)
> > > > > >
> > > > > > Calls are asynchronous -- futures
> > > > > >
> > > > > >
> > > > > >
> > > > > > Also would like feedback on which future functionality would
be
> > most
> > > > > useful
> > > > > > from a thin client:
> > > > > >
> > > > > > Function execution
> > > > > >
> > > > > > Durable clients
> > > > > >
> > > > > > User defined serialization
> > > > > >
> > > > > > Register interest
> > > > > >
> > > > > > Queries
> > > > > >
> > > > > > CQ
> > > > > >
> > > > > > Near side caching
> > > > > >
> > > > > > Create disk stores from client
> > > > > >
> > > > > > Region group membership
> > > > > >
> > > > > > Client subscription load balancing
> > > > > > Transactions
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > -Fred
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message