streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sblackmon <sblack...@apache.org>
Subject Using Streams Twitter Pojos in other apache projects
Date Sun, 26 Mar 2017 17:45:02 GMT
The approach we take with beans is to define json/xml schema in src/main/resources, and then
generate pojos into target/generated-sources, which are then packaged up in our module jars.

Our goal is to publish a comprehensive set of schemas and beans describing popular Rest APIs,
which hopefully will be useful to many other open-source projects.

For beans in the twitter module or other providers, any project can get them by depending
on the appropriate streams module.

However, that brings along the entire set of classes in the module and its transitive dependencies,
which in this case might wind up circular.

To any other project that wasn't about to become a dependency of streams, I'd suggest just
depend on streams-provider-twitter and call it a day.

It’s still possible though to get access to these beans without a direct dependency - you
can rely on the same resources and plugins that streams does to generate your own copy of
the beans as part of the build.

In streams-provider-twitter/pom.xml that looks like:
<plugin>
    <groupId>org.apache.streams.plugins</groupId>
    <artifactId>streams-plugin-pojo</artifactId>
    <version>0.5-incubating</version>
    <configuration>
        <sourcePaths>
            <sourcePath>${project.basedir}/src/main/jsonschema</sourcePath>
        </sourcePaths>
        <targetDirectory>${project.basedir}/target/generated-sources/pojo</targetDirectory>
        <targetPackage>org.apache.streams.twitter.pojo</targetPackage>
    </configuration>
</plugin>
To get a copy of the twitter beans in a juneau module, you could:
* create a src/main/jsonschema directory
* create a placeholder jsonschema for each object you want: tweet.json for example, pointing
to the reference jsonschema on the streams website.
The placeholder schemas would each look like:
{
    "type": "object",
    "$schema": "http://json-schema.org/draft-03/schema",
    "$license": [
        "http://www.apache.org/licenses/LICENSE-2.0"
    ],
    "id": "#",
    "javaInterfaces": [
        "java.io.Serializable"
    ],
    "extends": {
        "$ref": "http://streams.incubator.apache.org/site/latest/streams-project/streams-contrib/streams-provider-twitter/com/twitter/tweet.json"
    }
}
mvn generate-sources will then create all of the pojos that are needed for each of the schemas
in your directory.  In this case, you’d get Tweet, User, and many other supporting objects
as well, so you’ll have a full set of twitter beans that are kept up to date by the streams
project, but which ‘live’ in your project simultaneously.  
Steve
On March 25, 2017 at 12:51:23 PM, James Bognar (james.bognar@salesforce.com) wrote:
@Steve,

Are those general-purpose Twitter beans?  If so, they might be a good fit in the Juneau library. 
We have general-purpose beans for Atom, Cognos, HTML5, JSON-Schema, and Swagger so that you
can serialize those to any of the supported languages.  Twitter may be a good addition.  

At some point, Juneau needs a long-term strategy for where to keep these DTOs.  Maybe a separate
Apache project for all the various DTO libraries.


As far as OAuth goes, I've attached a predecessor class that the RestClient class is based
on that used to support OIDC authentication (and FORM/BASIC as well) for Rational Jazz applications. 
I'm not sure if it would help, but it may be a starting point.

On Sat, Mar 25, 2017 at 9:49 AM, sblackmon <sblackmon@apache.org> wrote:
Hello,

I’ve just pushed again to https://github.com/steveblackmon/incubator-streams/tree/STREAMS-496
As you can see, I've removed twitter4j entirely and replaced it with new code that uses juneau
to achieve the same outcome.

The primary client class is:
https://github.com/steveblackmon/incubator-streams/blob/STREAMS-496/streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Twitter.java

It’s currently > 300 lines - I expect it will shrink to < 100 once support for GET
requests are added to RemoteableProxy in juneau 6.2.0.
 
Next step is to get the unit and integration tests passing.  

Key to that are:
 - Performing oauth setup when creating the RestClient
 - Embedding error handling into the RestClient using juneau libraries
 - Embedding retry management into the RestClient using juneau libraries

More to come, but we’re on track to wrap this up in a release before April 30.

Steve
On March 19, 2017 at 5:02:51 PM, sblackmon (sblackmon@apache.org) wrote:

I created https://cwiki.apache.org/confluence/display/STREAMS/Twitter+Provider+Refactor with
a more concrete proposal for how to do this.  I welcome feedback or any volunteers who want
to help.

Steve
On March 6, 2017 at 10:59:44 AM, sblackmon (sblackmon@apache.org) wrote:

 
So this is still something we need to take care of, and the clock is ticking down to April
30 2017. I opened STREAMS-496 this morning.

I think it will be straight-forward to remove the twitter4j client from the providers, at
least from a request / response perspective, using our twitter provider pojos.

Where I have zero useful expertise is replicating the process of turning an instance of org.apache.streams.twitter.TwitterOAuthConfiguration
into the headers that actually get sent with each request. 

Does anyone have enough background in how oauth works (on twitter specifically, but also more
generally) to help out with this?

Steve

On November 28, 2016 at 5:03:23 AM, Stian Soiland-Reyes (stain@apache.org(mailto:stain@apache.org))
wrote:

>
>
> On 2016-11-14 17:04 (-0000), Joey Frazee wrote:
> > The ASF recently reclassified the JSON license for org.json as category-x because
of its "shall be used for Good, not Evil" clause [1].
> >
> > There does not appear to be any direct usage of it in Streams but there are a number
of dependencies in Streams that do depend on org.json, most notably Twitter4j, and it does
appear in the poms.
>
> There are also repackaging like org/apache/geronimo/bundles/json/20090211_1/json-20090211_1.jar
> to be aware of, which is used by jackson-datatype-json.org, which at least for Streams
0.4 was referenced here:
>
> /org/apache/streams/streams-core/0.4-incubating/streams-core-0.4-incubating.pom: jackson-datatype-json-org
> /org/apache/streams/streams-provider-moreover/0.4-incubating/streams-provider-moreover-0.4-incubating.pom:
jackson-datatype-json-org
> /org/apache/streams/streams-master/0.4-incubating/streams-master-0.4-incubating.pom:
jackson-datatype-json-org
>
>




--
James Bognar
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message