apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chinmay Kolhatkar <chin...@datatorrent.com>
Subject Re: Ability to generate POJO class on the fly
Date Fri, 22 Jan 2016 13:24:49 GMT
@bhupesh, would it help if JSONObject is taken as input to this utility
rather than json string?

Thanks,
Chinmay
On 22 Jan 2016 18:45, "Chinmay Kolhatkar" <chinmay@datatorrent.com> wrote:

> Got it. I think approach 2 is still possible with json based schema.
> So basically, the operator can create json schema as mention in first mail
> based on column name and type. Then pass that json schema to this utility
> to get class to be used directly.
>
> Thanks,
> Chinmay.
> On 22 Jan 2016 18:38, "Priyanka Gugale" <priyanka@datatorrent.com> wrote:
>
>> Say I need to emit pojo from JDBCInputOperator. I will use on fly POJO
>> generator for this purpose instead of providing predefined class.
>> Now to generate POJO on fly there are two options:
>> 1. Get json as config from user as you specified.
>> 2. Based on column name and type generate pojo fields and hence pojo class
>> without much of user configuration. (I am only considering one level here,
>> not the nested pojo)
>>
>> Option 1 is must have to provide user a way to control how we generate
>> pojo. But option 2 is useful when user simply want to put column values in
>> some object.
>>
>> -Priyanka
>>
>> On Fri, Jan 22, 2016 at 6:30 PM, Chinmay Kolhatkar <
>> chinmay@datatorrent.com>
>> wrote:
>>
>> > Hi Priyanka,
>> >
>> > Can you give an example?
>> >
>> > Thanks,
>> > Chinmay.
>> >
>> > ~ Chinmay
>> > On 22 Jan 2016 18:14, "Priyanka Gugale" <priyanka@datatorrent.com>
>> wrote:
>> >
>> > > We do need configuration in some form like json, but I was thinking we
>> > can
>> > > have default wherever possible. This is an option and not replacement
>> to
>> > > json config.
>> > >
>> > > -Priyanka
>> > >
>> > > On Fri, Jan 22, 2016 at 4:34 PM, Chinmay Kolhatkar <
>> > > chinmay@datatorrent.com>
>> > > wrote:
>> > >
>> > > > @priyanka
>> > > >
>> > > > Advantage with JSON schema is that one can represent nested POJO
>> > > definition
>> > > > in that. If you're referring to FieldInfo, I'm not sure if that is
>> > > possible
>> > > > there.
>> > > >
>> > > > Even if we chose not to have nested POJO now, I think JSON schema
>> input
>> > > to
>> > > > this utility gives more chance to expand easily later.
>> > > >
>> > > >
>> > > >
>> > > > On Fri, Jan 22, 2016 at 3:30 PM, Chinmay Kolhatkar <
>> > > > chinmay@datatorrent.com>
>> > > > wrote:
>> > > >
>> > > > > Sure. We can make this utility to provide byte[] for compiled
>> class.
>> > > > > Though the utility will give a OutputStream object.. One can
get
>> > byte[]
>> > > > > out of that.
>> > > > >
>> > > > > On Fri, Jan 22, 2016 at 2:56 PM, Timothy Farkas <
>> tim@datatorrent.com
>> > >
>> > > > > wrote:
>> > > > >
>> > > > >> Suggestion: +1 for automatic generation of schema. The user
>> should
>> > be
>> > > > able
>> > > > >> to override the a manually specified schema though.
>> > > > >>
>> > > > >> Question: If the mechanism used to generate the class can
>> produce a
>> > > > byte[]
>> > > > >> array, then you can send the byte[] array for the class to
>> > downstream
>> > > > >> operators and load the class from the byte[] array in each
>> operator.
>> > > An
>> > > > >> example of how to do this is here:
>> > > > >>
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>> http://stackoverflow.com/questions/1781091/java-how-to-load-class-stored-as-byte-into-the-jvm
>> > > > >>
>> > > > >> Thanks,
>> > > > >> Tim
>> > > > >>
>> > > > >> On Fri, Jan 22, 2016 at 12:59 AM, Priyanka Gugale <
>> > > > >> priyanka@datatorrent.com>
>> > > > >> wrote:
>> > > > >>
>> > > > >> > Hi,
>> > > > >> >
>> > > > >> > Suggestion:
>> > > > >> > This came up with one of the discussion with Pramod,
will it
>> be a
>> > > good
>> > > > >> > idea, for database input operators to generate pojo
based on
>> the
>> > > > >> selected
>> > > > >> > column field names and types? No need to accept json
input from
>> > > user.
>> > > > >> >
>> > > > >> > Question:
>> > > > >> > How can we share this POJO among multiple operators
when
>> > application
>> > > > is
>> > > > >> > already launched and class is generated on the fly?
>> > > > >> >
>> > > > >> > -Priyanka
>> > > > >> >
>> > > > >> > On Thu, Jan 21, 2016 at 11:16 PM, Chinmay Kolhatkar
<
>> > > > >> > chinmay@datatorrent.com
>> > > > >> > > wrote:
>> > > > >> >
>> > > > >> > > Hi All,
>> > > > >> > >
>> > > > >> > > We're planning to add a utility in malhar-library
for
>> > generating a
>> > > > >> POJO
>> > > > >> > > class on the fly from given JSON schema.
>> > > > >> > >
>> > > > >> > > Use case is where the application is provided with
schema and
>> > that
>> > > > >> needs
>> > > > >> > to
>> > > > >> > > be used in one or more operators either as a tuple
over the
>> > stream
>> > > > OR
>> > > > >> for
>> > > > >> > > processing.
>> > > > >> > >
>> > > > >> > > General Design:
>> > > > >> > > 1. Utility will be provided with fqcn of the class
and schema
>> > > > >> definition
>> > > > >> > > provided as json.
>> > > > >> > > 2. The schema definition will look like following:
>> > > > >> > >     {
>> > > > >> > >        "fqcn":"<qualified class name>",
>> > > > >> > >        "fields": [
>> > > > >> > >          {
>> > > > >> > >            "name":"field1",
>> > > > >> > >            "type":"long"
>> > > > >> > >          },
>> > > > >> > >          {
>> > > > >> > >            "name":"field2",
>> > > > >> > >            "type": "string"
>> > > > >> > >          }
>> > > > >> > >        ]
>> > > > >> > >     }
>> > > > >> > > 3. Supported types identified in "type" JSON field
are:
>> > > > >> > >       boolean, char, byte, short, int, float, long,
double
>> > > > >> > > 4. The output of this utility will be a generated
.class
>> file in
>> > > the
>> > > > >> form
>> > > > >> > > of FSDataOutputStream.
>> > > > >> > > 5. Xbean asm5 library will be used for this.
>> > > > >> > > 6. Following methods will be added to the generated
class:
>> > > > >> > >     a. Getter/Setter methods for given fields.
>> > > > >> > >     b. simple toString - Generate string equivalent
for all
>> the
>> > > > fields
>> > > > >> > >     c. hashCode method - calculate the overall
hashCode using
>> > > > >> individual
>> > > > >> > > field hashcodes, similar to how String.hashCode
generates
>> > hashCode
>> > > > of
>> > > > >> > > string using chars in it.
>> > > > >> > >     d. equals method - Similar to how String.equals
method
>> has
>> > > done
>> > > > it
>> > > > >> > > using individual char.
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > Questions I have about the functionality:
>> > > > >> > > 1. Should the utility also support nested pojo
definition via
>> > > schema
>> > > > >> > json?
>> > > > >> > >     In such case the field definition can look
like
>> following:
>> > > > >> > >     { "name":"nestedField", "type":"<fqcn of
nested class>",
>> > > > >> > > "fields":[....] }
>> > > > >> > >     The nested class will be provided
>> > > > >> > >
>> > > > >> > > 2. Should the utility also support giving out the
>> > JarOutputStream
>> > > > >> which
>> > > > >> > > will have a generated jar?
>> > > > >> > >
>> > > > >> > > Please let me know your thoughts.
>> > > > >> > >
>> > > > >> > > Thanks,
>> > > > >> > > Chinmay.
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message