apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhupesh Chawda <bhup...@datatorrent.com>
Subject Re: Ability to generate POJO class on the fly
Date Fri, 22 Jan 2016 13:34:52 GMT
Yes. So in the JDBC output case, a json object could be readily constructed
by the operator to get a POJO class rather than playing around with JSON
syntax.

On Fri, Jan 22, 2016 at 6:54 PM, Chinmay Kolhatkar <chinmay@datatorrent.com>
wrote:

> @bhupesh, would it help if JSONObject is taken as input to this utility
> rather than json string?
>
> Thanks,
> Chinmay
> On 22 Jan 2016 18:45, "Chinmay Kolhatkar" <chinmay@datatorrent.com> wrote:
>
> > Got it. I think approach 2 is still possible with json based schema.
> > So basically, the operator can create json schema as mention in first
> mail
> > based on column name and type. Then pass that json schema to this utility
> > to get class to be used directly.
> >
> > Thanks,
> > Chinmay.
> > On 22 Jan 2016 18:38, "Priyanka Gugale" <priyanka@datatorrent.com>
> wrote:
> >
> >> Say I need to emit pojo from JDBCInputOperator. I will use on fly POJO
> >> generator for this purpose instead of providing predefined class.
> >> Now to generate POJO on fly there are two options:
> >> 1. Get json as config from user as you specified.
> >> 2. Based on column name and type generate pojo fields and hence pojo
> class
> >> without much of user configuration. (I am only considering one level
> here,
> >> not the nested pojo)
> >>
> >> Option 1 is must have to provide user a way to control how we generate
> >> pojo. But option 2 is useful when user simply want to put column values
> in
> >> some object.
> >>
> >> -Priyanka
> >>
> >> On Fri, Jan 22, 2016 at 6:30 PM, Chinmay Kolhatkar <
> >> chinmay@datatorrent.com>
> >> wrote:
> >>
> >> > Hi Priyanka,
> >> >
> >> > Can you give an example?
> >> >
> >> > Thanks,
> >> > Chinmay.
> >> >
> >> > ~ Chinmay
> >> > On 22 Jan 2016 18:14, "Priyanka Gugale" <priyanka@datatorrent.com>
> >> wrote:
> >> >
> >> > > We do need configuration in some form like json, but I was thinking
> we
> >> > can
> >> > > have default wherever possible. This is an option and not
> replacement
> >> to
> >> > > json config.
> >> > >
> >> > > -Priyanka
> >> > >
> >> > > On Fri, Jan 22, 2016 at 4:34 PM, Chinmay Kolhatkar <
> >> > > chinmay@datatorrent.com>
> >> > > wrote:
> >> > >
> >> > > > @priyanka
> >> > > >
> >> > > > Advantage with JSON schema is that one can represent nested POJO
> >> > > definition
> >> > > > in that. If you're referring to FieldInfo, I'm not sure if that
is
> >> > > possible
> >> > > > there.
> >> > > >
> >> > > > Even if we chose not to have nested POJO now, I think JSON schema
> >> input
> >> > > to
> >> > > > this utility gives more chance to expand easily later.
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Fri, Jan 22, 2016 at 3:30 PM, Chinmay Kolhatkar <
> >> > > > chinmay@datatorrent.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Sure. We can make this utility to provide byte[] for compiled
> >> class.
> >> > > > > Though the utility will give a OutputStream object.. One
can get
> >> > byte[]
> >> > > > > out of that.
> >> > > > >
> >> > > > > On Fri, Jan 22, 2016 at 2:56 PM, Timothy Farkas <
> >> tim@datatorrent.com
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > >> Suggestion: +1 for automatic generation of schema. The
user
> >> should
> >> > be
> >> > > > able
> >> > > > >> to override the a manually specified schema though.
> >> > > > >>
> >> > > > >> Question: If the mechanism used to generate the class
can
> >> produce a
> >> > > > byte[]
> >> > > > >> array, then you can send the byte[] array for the class
to
> >> > downstream
> >> > > > >> operators and load the class from the byte[] array in
each
> >> operator.
> >> > > An
> >> > > > >> example of how to do this is here:
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> http://stackoverflow.com/questions/1781091/java-how-to-load-class-stored-as-byte-into-the-jvm
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >> Tim
> >> > > > >>
> >> > > > >> On Fri, Jan 22, 2016 at 12:59 AM, Priyanka Gugale <
> >> > > > >> priyanka@datatorrent.com>
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >> > Hi,
> >> > > > >> >
> >> > > > >> > Suggestion:
> >> > > > >> > This came up with one of the discussion with Pramod,
will it
> >> be a
> >> > > good
> >> > > > >> > idea, for database input operators to generate
pojo based on
> >> the
> >> > > > >> selected
> >> > > > >> > column field names and types? No need to accept
json input
> from
> >> > > user.
> >> > > > >> >
> >> > > > >> > Question:
> >> > > > >> > How can we share this POJO among multiple operators
when
> >> > application
> >> > > > is
> >> > > > >> > already launched and class is generated on the
fly?
> >> > > > >> >
> >> > > > >> > -Priyanka
> >> > > > >> >
> >> > > > >> > On Thu, Jan 21, 2016 at 11:16 PM, Chinmay Kolhatkar
<
> >> > > > >> > chinmay@datatorrent.com
> >> > > > >> > > wrote:
> >> > > > >> >
> >> > > > >> > > Hi All,
> >> > > > >> > >
> >> > > > >> > > We're planning to add a utility in malhar-library
for
> >> > generating a
> >> > > > >> POJO
> >> > > > >> > > class on the fly from given JSON schema.
> >> > > > >> > >
> >> > > > >> > > Use case is where the application is provided
with schema
> and
> >> > that
> >> > > > >> needs
> >> > > > >> > to
> >> > > > >> > > be used in one or more operators either as
a tuple over the
> >> > stream
> >> > > > OR
> >> > > > >> for
> >> > > > >> > > processing.
> >> > > > >> > >
> >> > > > >> > > General Design:
> >> > > > >> > > 1. Utility will be provided with fqcn of the
class and
> schema
> >> > > > >> definition
> >> > > > >> > > provided as json.
> >> > > > >> > > 2. The schema definition will look like following:
> >> > > > >> > >     {
> >> > > > >> > >        "fqcn":"<qualified class name>",
> >> > > > >> > >        "fields": [
> >> > > > >> > >          {
> >> > > > >> > >            "name":"field1",
> >> > > > >> > >            "type":"long"
> >> > > > >> > >          },
> >> > > > >> > >          {
> >> > > > >> > >            "name":"field2",
> >> > > > >> > >            "type": "string"
> >> > > > >> > >          }
> >> > > > >> > >        ]
> >> > > > >> > >     }
> >> > > > >> > > 3. Supported types identified in "type" JSON
field are:
> >> > > > >> > >       boolean, char, byte, short, int, float,
long, double
> >> > > > >> > > 4. The output of this utility will be a generated
.class
> >> file in
> >> > > the
> >> > > > >> form
> >> > > > >> > > of FSDataOutputStream.
> >> > > > >> > > 5. Xbean asm5 library will be used for this.
> >> > > > >> > > 6. Following methods will be added to the
generated class:
> >> > > > >> > >     a. Getter/Setter methods for given fields.
> >> > > > >> > >     b. simple toString - Generate string equivalent
for all
> >> the
> >> > > > fields
> >> > > > >> > >     c. hashCode method - calculate the overall
hashCode
> using
> >> > > > >> individual
> >> > > > >> > > field hashcodes, similar to how String.hashCode
generates
> >> > hashCode
> >> > > > of
> >> > > > >> > > string using chars in it.
> >> > > > >> > >     d. equals method - Similar to how String.equals
method
> >> has
> >> > > done
> >> > > > it
> >> > > > >> > > using individual char.
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > > Questions I have about the functionality:
> >> > > > >> > > 1. Should the utility also support nested
pojo definition
> via
> >> > > schema
> >> > > > >> > json?
> >> > > > >> > >     In such case the field definition can
look like
> >> following:
> >> > > > >> > >     { "name":"nestedField", "type":"<fqcn
of nested
> class>",
> >> > > > >> > > "fields":[....] }
> >> > > > >> > >     The nested class will be provided
> >> > > > >> > >
> >> > > > >> > > 2. Should the utility also support giving
out the
> >> > JarOutputStream
> >> > > > >> which
> >> > > > >> > > will have a generated jar?
> >> > > > >> > >
> >> > > > >> > > Please let me know your thoughts.
> >> > > > >> > >
> >> > > > >> > > Thanks,
> >> > > > >> > > Chinmay.
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message