geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Blum <jb...@pivotal.io>
Subject Re: [DISCUSS] New annotation to identify Functions whose class hierarchy spans multiple jar files
Date Sat, 12 Aug 2017 18:20:22 GMT
*@Jared*- actually, I should mention that *Spring* too provides other
infrastructure support to inspect byte code for certain Annotation
meta-data without needed to load classes.  Although, *Spring* uses ASM [1]
given FCS did not exist at that time.  *Spring* definitely needed this from
2.5 and later when the *Spring Framework* was rebased on Java 5 and
Annotation support was added.


Now, onto another related note, since we are on the topic of using Java
Annotations to identify Geode *Functions*, which is something that *Spring
Data Geode* has already and similarly solved, many moons ago.

The M&M team may recall this JIRA ticket...

https://issues.apache.org/jira/browse/GEODE-28

"*Gfsh's 'deploy' command needs enhancements to recognize Spring Data
GemFire Annotated (deployed) Functions.*"


Any chance you are willing to consider having *Gfsh's* `deploy` command
also recognize SD Geode's Function Annotations (effectively, just
@GemfireFunction [2]) in addition to Geode's @RegisterableFunction during
the scan?


See the following link for further details on SDG's Function annotation
support [3], along with the difference between Implementation vs. Execution
[4], on implementing a Function [5], and finally, Annotations for
Implementation [6].

1 significant difference between SDG's Function support and Geode's is that
SDG allows the use of POJO "methods" to be used as Geode *Functions*.

I'd be happy to discuss with you offline how we could support this using a
variation (streamlined implementation) of SDG's Function annotation
infrastructure support without unduly introducing a dependency on SDG
(again, just) to support the SDG @GemfireFunction annotation.

In a nutshell, my thinking/idea would be to "duplicate" the functionality
in the PojoFunctionWrapper SDG class [7] (source here [8]) in Geode.

There are a bunch of other things the SDG PojoFunctionWrapper supports like
method argument injection, but some of that could be stripped out knowing
that a Geode Server will not necessarily provide a *Spring* context where
things can be "injected".  The result would be that the method signature
would be more strict (such as strictly following the
o.a.g.c.execute.Function.execute(..) method signature [9]), but that for
the first time, a user would be allowed to provide POJO methods for
*Function* execution!  A huge plus IMO.  And, it would give *Spring* users
another option when using SDG and situations where the developer has less
control over the servers and cluster and "what" gets "deployed".

Since you are "scanning" anyway, I suspect recognizing SDG's Function
annotations (i.e. @GemfireFunction) is well, nearly as trivial as
recognizing Geode's @RegisterableFunction.  There will be some additional
logic needed to "adapt" the POJO method into an actual Geode Function, as
SDG's o.s.d.g.function.PojoFunctionWrapper suggests, but that is not
difficult.

Thoughts?

Regards,
John


[1] http://asm.ow2.org/
[2]
http://docs.spring.io/spring-data-gemfire/docs/current/api/org/springframework/data/gemfire/function/annotation/GemfireFunction.html
[3]
http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#function-annotations
[4]
http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#_implementation_vs_execution
[5]
http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#_implementing_a_function
[6]
http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#_annotations_for_function_implementation
[7]
http://docs.spring.io/spring-data-gemfire/docs/current/api/org/springframework/data/gemfire/function/PojoFunctionWrapper.html
[8]
http://docs.spring.io/spring-data-gemfire/docs/current/api/org/springframework/data/gemfire/function/PojoFunctionWrapper.html
[9]
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/Function.html#execute-org.apache.geode.cache.execute.FunctionContext-

On Sat, Aug 12, 2017 at 10:38 AM, John Blum <jblum@pivotal.io> wrote:

> *@Jared* - yes, makes sense.  Thanks for the reference to FCS.  I am
> familiar with it and I am also aware that Geode is starting to use this in
> other capacities.
>
> *Regarding name...*
>
> The one thing I dislike about the name "RegisterFunction" is it implies
> some action will be performed by Geode (in general) just by the addition of
> the annotation.  However, this is simply not true.
>
> AFAIK (or have heard so far) this annotation only works when "deploying"
> Functions via *Gfsh* in order to limit which classes loaded by the scan,
> which won't happen in any other context.  As such, unfortunately (IMO) it
> is very context driven and dependent.
>
> Consider this...
>
> If a developer creates an "embedded" Geode "peer cache" application
> connected to the cluster, bundles it up in a JAR file that contains "
> RegisterFunction" annotated *Function* classes, and starts it up (e.g. `java
> -jar peerCacheApplication.jar`) Geode is not going to recognize the
> presence of this annotation on classes in the JAR.
>
> The developer cannot deploy the entire JAR (i.e. `deploy jar`) and expect
> the application to be run (e.g. *Initializer* style [1]) and so is likely
> to separate out (perhaps in another JAR) the *Functions* and deploy them
> independently, breaking encapsulation (microservices-style).
>
> Of course, *Gfsh* and the *Manager* do not require that *Functions* be
> explicitly broken out, but imagine deploying a "FAT" JAR (possibly
> containing all the *libs* the application requires at runtime).  That
> most certainly will be much more expensive in terms of network bandwidth,
> particularly where *Cluster Config* is concerned.
>
> I only mention this since some frameworks/containers do support
> Annotations that "do affect the configuration" just by the mere presence of
> the annotation, which usually, but not always, is accompanied by some
> additional configuration that the framework/container will apply, every
> single time, not just based on tools.
>
> By way of example, using SDG, have a look at the @EnableSecurity [2]
> annotation and then look at @EnablePdx [3].  @EnableSecurity provides
> additional configuration for the container to apply where as @EnablePdx
> triggers "other" configuration supplied by SDG to the container.  In both
> cases, the mere presence of the annotation(s) makes those things happen.
>
> Therefore, I concluded that @RegisterableFunction is more intuitive since
> it implies the *Function* "can be" registered, but at least to me, it
> also implies that the *Function* is not necessarily being registered.
>
> Regards,
> John
>
>
> [1] http://geode.apache.org/docs/guide/12/basic_config/
> the_cache/setting_cache_initializer.html
> [2] https://github.com/spring-projects/spring-data-geode/
> blob/2.0.0.RC2/src/main/java/org/springframework/data/
> gemfire/config/annotation/EnableSecurity.java#L50-L51
> [3] https://github.com/spring-projects/spring-data-geode/
> blob/2.0.0.RC2/src/main/java/org/springframework/data/
> gemfire/config/annotation/EnablePdx.java#L40-L45
>
>
> On Fri, Aug 11, 2017 at 4:16 PM, Kirk Lund <klund@apache.org> wrote:
>
>> I can't even say "RegisterableFunction" flibble flobble ;)
>>
>> On Fri, Aug 11, 2017 at 4:09 PM, Jared Stewart <jstewart@pivotal.io>
>> wrote:
>>
>> > Hi John,
>> >
>> > Thanks for the suggestions. I like “@RegisterableFunction” better than
>> > “@RegisterFunction".  I think that we don’t want @RegisterableFunction
>> to
>> > be @Inherited, in order to avoid creating a new variant of the problem
>> we
>> > are trying to fix.  As you suggest, we should be mindful to document the
>> > behavior clearly.
>> >
>> > In case you’re interested, here’s <https://github.com/lukehutch/
>> > fast-classpath-scanner/wiki/3.4.-Finding-classes-with-
>> > specific-annotations-or-meta-annotations> some documentation of the API
>> > for the library we’re using for the annotation scanning.  It scans the
>> > binary byte code files directly, rather than using a Class reference
>> like
>> > Spring utilities you pointed out (which AFAIK requires having that class
>> > loaded in the JVM).
>> >
>> > Thanks,
>> > Jared
>> >
>> > > On Aug 11, 2017, at 11:00 AM, John Blum <jblum@pivotal.io> wrote:
>> > >
>> > > Hi Jared-
>> > >
>> > > In general, I like this idea since Annotations are a great form of
>> > > meta-data and essentially meaningless outside of the intended context
>> and
>> > > therefore do not impose any adverse effects on any existing behavior.
>> > >
>> > > However, 2 things... 1 suggestion and 1 caution...
>> > >
>> > >
>> > > 1. Perhaps "RegisterableFunction" as opposed to "RegisterFunction".
>> > >
>> > >
>> > > 2. Annotations can be "inherited" in the same way that your
>> > ConcreteFunction
>> > > extends (inherits from) AbstractFunction, so too can a more concrete
>> > > Function possibly inherit from a less-concrete-but-not-abstract
>> Function.
>> > > A developer might expect that they don't have to re-annotated his/her
>> > > function.
>> > >
>> > > For example...
>> > >
>> > > @RegisterableFunction
>> > > class ProcessOrder extends FunctionAdapter { ... }
>> > >
>> > > class CreateAccountAtPointOfSale extends ProcessOrder { ... }
>> > >
>> > > If these are in separate JAR files, depending on the "application", I
>> may
>> > > only want to "register" the CreateAccountAtPointOfSale Function and
>> not
>> > the
>> > > ProcessOrder Function explicitly.  Please forgive my example here
>> since
>> > it
>> > > seems like a bad example given it feels like 2 separate actions but is
>> > > often part of the same workflow and so really depends on how
>> application
>> > > developers think about and organize their logic, which usually leads
>> to
>> > > "unexpected" things you never anticipated when introducing something
>> like
>> > > this.
>> > >
>> > > SIDE NOTE: Of course, from a Java SE standpoint, the
>> "RegisteredFunction"
>> > > Annotation could be (but does not have to be) meta-annotated with
>> > @Inherited.,
>> > > such as...
>> > >
>> > > @Target(ElementType.TYPE)
>> > > @Retention(RetentionPolicy.RUNTIME)
>> > > @Inherited
>> > > ...
>> > > @interface RegisteredFunction { .. }
>> > >
>> > >
>> > > *Spring* takes all of these things into account when it processes
>> > > annotations of this nature (e.g. @Enable...).  Especially have a look
>> at
>> > > o.s.core.annotation.AnnotatedElementUtils [1] and even
>> > > o.s.core.annotation.AnnotationUtils [2].  These are very robust and
>> very
>> > > powerful classes underpinning much of *Spring's* Annotation
>> configuration
>> > > and processing.  *Boot* also extends this functionality and *Spring*
>> > > Annotation config in very specific/custom ways.
>> > >
>> > > I think it is reasonable to set limitations to keep the initial scope
>> > > small, but be sure those are well documented since users will be
>> coming
>> > > from many different frameworks having many different expectations.
>> > >
>> > > Food for thought/hope this helps.
>> > >
>> > > Regards,
>> > > -John
>> > >
>> > >
>> > > [1]
>> > > http://docs.spring.io/spring/docs/current/javadoc-api/org/
>> > springframework/core/annotation/AnnotatedElementUtils.html
>> > > [2]
>> > > http://docs.spring.io/spring/docs/current/javadoc-api/org/
>> > springframework/core/annotation/AnnotationUtils.html
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Fri, Aug 11, 2017 at 10:22 AM, Jared Stewart <jstewart@pivotal.io>
>> > wrote:
>> > >
>> > >> Recent changes introduced to avoid the eager loading of classes have
>> > lead
>> > >> to Functions not getting registered correctly if the class hierarchy
>> > >> leading up to Function (or FunctionAdapter) is split up across
>> multiple
>> > jar
>> > >> files.  We propose to introduce a new annotation to identify
>> functions
>> > in
>> > >> such a case.
>> > >> Consider the following scenario:
>> > >>> Abstract.jar - public abstract class AbstractFunction implements
>> > >> Function {...}
>> > >>> Concrete.jar - public class ConcreteFunction extends
>> AbstractFunction
>> > >> {...}
>> > >> When Concrete.jar is deployed, we only scan the classes inside
>> > >> Concrete.jar.  This means that we have no way of knowing that
>> > >> AbstractFunction eventually leads up to Function.  (We could load
>> > >> ConcreteFunction to see if it implements Function via reflection or
>> > >> Function.class.isAssignableFrom(), but then we would be back to
>> eagerly
>> > >> loading all of the classes to see whether or not they implement
>> > Function.)
>> > >> We propose a new annotation (perhaps @RegisterFunction, suggestions
>> are
>> > >> welcome) to designate a class as a Function in such a case.  Since
we
>> > are
>> > >> able to scan classes for annotations without loading them, this will
>> > allow
>> > >> us to identify ConcreteFunction as a Function without eagerly loading
>> > all
>> > >> of the classes in Concrete.jar.
>> > >> I should emphasize that ConcreteFunction already is registered as
>> > expected
>> > >> if it resides in the same jar file as AbstractFunction.  This
>> annotation
>> > >> would only be relevant when the class hierarchy leading up to
>> Function
>> > is
>> > >> spread across multiple jar files.
>> > >> - Jared
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > -John
>> > > john.blum10101 (skype)
>> >
>> >
>>
>
>
>
> --
> -John
> john.blum10101 (skype)
>



-- 
-John
john.blum10101 (skype)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message