impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Behm <alex.b...@cloudera.com>
Subject Re: impala support json format table
Date Wed, 10 May 2017 14:00:51 GMT
Great!

On Tue, May 9, 2017 at 11:40 PM, yu feng <olaptestyu@gmail.com> wrote:

> Thanks for reminding me, I very glad to do some contribution for impala, I
> will try to solve IMPALA-5016
> <https://issues.apache.org/jira/browse/IMPALA-5016> once I have time, and
> keep communication with community.
>
> 2017-05-10 12:59 GMT+08:00 Alexander Behm <alex.behm@cloudera.com>:
>
> > Hi Yu,
> >
> > glad to hear that you are considering contributing to Impala! As others
> > have mentioned before, I'd strongly recommend starting with a smaller
> task
> > to get accustomed our development workflows.
> >
> > I'm happy to help with finding a suitable task. For example, you might
> find
> > this JIRA interesting:
> > https://issues.apache.org/jira/browse/IMPALA-5016
> >
> > Please do reach out if I can help.
> >
> > Alex
> >
> > On Tue, Apr 18, 2017 at 10:55 AM, Alexander Behm <alex.behm@cloudera.com
> >
> > wrote:
> >
> > > The existing attempt used the Rapidjson library to do the parsing.
> > > Unfortunately, the Rapidjson API is not very convenient for Impala
> > because
> > > it returns typed data, i.e., it internally converts to
> > > float/double/int/whatever which is problematic for decimal (among
> > others).
> > > Ideally, we would use the same Impala code to convert data types from
> > > strings.
> > >
> > > On Tue, Apr 18, 2017 at 9:27 AM, Tim Armstrong <
> tarmstrong@cloudera.com>
> > > wrote:
> > >
> > >> Seems like useful functionality that would be great to have in Impala.
> > >> There was an earlier attempt to do this that didn't make it in - I'm
> not
> > >> sure that the approach was quite right:
> > >> https://gerrit.cloudera.org/#/c/1201/1 . I'm not sure what the exact
> > >> problems were but I remember we didn't think it was quite the right
> > >> approach.
> > >>
> > >> I think we'd need to talk through a design first because there are a
> lot
> > >> of
> > >> considerations and we want to make sure to get it right. I had some
> > >> initial
> > >> questions that I'd want to think through before adding a JSON scanner.
> > >>
> > >>    - What JSON does it accept?
> > >>    - How do we declare a table schema and map it to the JSON
> > >>    - How does it handle missing or extra fields - does it just return
> > null
> > >>    or drop the fields? What if the field type is wrong?
> > >>    - How do the numeric types work? JSON only supports floating point,
> > but
> > >>    I think many people would like to store higher-precision decimal or
> > >> 64-bit
> > >>    integer types (which is technically outside of the JSON standard).
> > >>    - Will it support codegen? If not, is it written in a way that
> allows
> > >> it
> > >>    in future?
> > >>
> > >> Cheers,
> > >> Tim
> > >>
> > >> - Tim
> > >>
> > >> On Tue, Apr 18, 2017 at 8:52 AM, Jim Apple <jbapple@cloudera.com>
> > wrote:
> > >>
> > >> > On Mon, Apr 17, 2017 at 8:02 PM, yu feng <olaptestyu@gmail.com>
> > wrote:
> > >> >
> > >> > > Hi impala community:
> > >> > >   I am Newly join to Impala,
> > >> >
> > >> >
> > >> > Welcome!
> > >> >
> > >> > I want to know what is the attitude of impala
> > >> > > community for supporting json format.
> > >> >
> > >> >
> > >> > I am in favor of it. I am only one person, though - anybody else
> > object
> > >> to
> > >> > JSON support?
> > >> >
> > >> > If this match the roadmap, maybe I
> > >> > > can make some contribution.
> > >> > >
> > >> >
> > >> > I do not recall much talk about Apache Impala's roadmap since we
> > joined
> > >> the
> > >> > ASF. Perhaps I missed a thread about it?
> > >> >
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message