Mailing-List: contact dev-help@gearpump.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@gearpump.incubator.apache.org
Date: Sat, 14 May 2016 09:13:12 +0000 (UTC)
From: "Manu Zhang (JIRA)" <jira@apache.org>
To: dev@gearpump.incubator.apache.org
Message-ID: <JIRA.12960831.1461258485000.190347.1463217192867@Atlassian.JIRA>
In-Reply-To: <JIRA.12960831.1461258485000@Atlassian.JIRA>
References: <JIRA.12960831.1461258485000@Atlassian.JIRA> <JIRA.12960831.1461258485034@arcas>
Subject: [jira] [Updated] (GEARPUMP-85) Shared Data Types
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Sat, 14 May 2016 09:13:18 -0000


     [ https://issues.apache.org/jira/browse/GEARPUMP-85?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:all-tabpanel ]

Manu Zhang updated GEARPUMP-85:
-------------------------------
    Fix Version/s:     (was: 0.8.1)

> Shared Data Types
> -----------------
>
>                 Key: GEARPUMP-85
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-85
>             Project: Apache Gearpump
>          Issue Type: Bug
>          Components: restapi
>    Affects Versions: 0.8.0
>            Reporter: Kam Kasravi
>
> h1. GearPump Shared Data Types
> h2. Definition
> {quote}
> A shared data type is a type that crosses the REST layer either from the =
client (eg POST) or to the client (eg GET). =20
> {quote}
> h2. Requirements
> # Shared data types should be easily demarked. We should have a way of ea=
sily identifying these data types which is automated. That is - a programma=
tic way of finding these data types reliably that requires no separate list=
 or manual maintenance which could become outdated.
> # The REST API should have documentation auto-generated. This includes th=
e endpoints and the shared data types. It should be part of the build proce=
ss.
> # All shared data types should be JSON friendly. That is no data type sho=
uld require special serde (serialization/deserialization) methods on client=
 or server to handle unserializable members.
> # There should be compilation checks that prevent type related errors. Th=
at is changing a shared data type on either the client or server will resul=
t in a compilation error on the flip side.
> # Shared data types should not need to be repackaged. Current types shoul=
d be able to remain as is, new shared data types should be able to be defin=
ed where appropriate.
> # Annotations or other constructs should be minimal. Given the constraint=
 that shared data types can be defined anywhere, there needs to be a way to=
 identify them. Naming conventions are not a possibility due to no naming c=
onvention has been followed to date. If annotations are used they should be=
 minimal.
> # Shared data types should provide a [json schema (v4)](http://json-schem=
a.org/documentation.html) or equivalent that allows tooling to transpile th=
ese shared data types to potentially other languages. 3rd party clients may=
 require a schema to develop any reasonable client-side functionality withi=
n reasonable time frames.
> h2. Use Cases
> GearPump provides a REST API that is used to expose a wide variety of fun=
ctionality both for the user of GearPump and the administrator. The REST AP=
I leverages JSON in many HTTP GET, POST, PUT, DELETE operations. These JSON=
 structures are used in javascript as javascript objects, within the REST s=
erver and within the master, worker and executor actors. They permeate all =
environments. Given their scope and prevalence we should have a set of guid=
elines on how these data types should be created, changed and transported. =
This will minimize a variety of problems.=20
> h2. Current usage=20
> Currently shared data types as defined above are found across a large num=
ber of packages on the server and inline with other classes, objects and ot=
her scala definitions. There is no devoted package or equivalent construct =
to isolate these data types. They are also not annotated or otherwise ident=
ified within the code base. That is - there is no way to reliably gather th=
ese types other than a tedious review of the services classes and their rou=
te paths. Additionally, within the client layer there is no typing of these=
 shared data types within javascript. For GET operations these data types a=
re reified into untyped javascript object literals, where field naming is p=
reserved but type naming is not. In other words a scala data type of `case =
class MasterData(masterDescription: MasterSummary)` is not identified by th=
at name in the client code. There is also no tooling in place to auto-gener=
ate documentation for these shared data types or the REST API in general. A=
ny 3rd party wishing to build their own client would need to scour the code=
 and attempt to cobble together the REST endpoints and their related data. =
This would be untenable for most 3rd parties and represents a significant o=
bstacle in promoting GearPump's many other strengths in a very competitive =
landscape.
> h3. Positives
> There are some things that are done on the server with these shared data =
types that are positive.=20
> # upickle is used to read and write all shared data types on the server. =
(Unfortunately upickle is not used on the client).
> # There is minimal specialized type conversions. Upickle provides a way t=
o take a type that cannot normally be serialized and allows one to provide =
a customized JSON mapping. For example Actor is not a serializable data typ=
e, if one were to serialize an Actor using upickle they would need to provi=
de a customized serializer/deserializer for it. There are few places (if an=
y) where we do this.
> # All data types are case classes.
> h3. Suggested Design approaches
> Design solutions should attempt to leverage best-in-class tooling - for e=
xample for REST related documentation. Any design solution should not tax t=
he current work flow with a heavy weight process related to development on =
either the client or server side.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)