drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Validate your JSON files
Date Tue, 25 Nov 2014 11:27:51 GMT
Chris,

Your tool could be updated to use Jackson and would then have the exact
same semantics as Drill.

It is still great as it is.... just could be slightly greater.


On Mon, Nov 24, 2014 at 11:09 PM, Steven Phillips <sphillips@maprtech.com>
wrote:

> No, Drill uses jackson to parse the json as a stream. It's fine if the json
> record has newline characters.
>
> Your validation tool is still useful, in the case where each json record is
> contained in a single line, which is common. Just be aware that it won't
> work in all cases.
>
> On Mon, Nov 24, 2014 at 3:04 PM, Christopher Matta <cmatta@mapr.com>
> wrote:
>
> > Steven,
> > Yes it does, doesn't Drill  also require that the entire JSON record be
> on
> > a single line?
> >
> > I wrote this for situations when the data set is too large to paste into
> a
> > web-based validator.
> >
> > Chris Matta
> > cmatta@mapr.com
> > 215-701-3146
> >
> > On Mon, Nov 24, 2014 at 6:01 PM, Steven Phillips <sphillips@maprtech.com
> >
> > wrote:
> >
> > > Christopher,
> > >
> > > Does your validator require that the entire json record be on a single
> > > line?
> > >
> > > On Mon, Nov 24, 2014 at 2:57 PM, Aman Sinha <asinha@maprtech.com>
> wrote:
> > >
> > > > BTW, there's a web based validator called jsonlint.com whose source
> is
> > > > available at :  https://github.com/arc90/jsonlintdotcom
> > > >
> > > > On Mon, Nov 24, 2014 at 2:07 PM, Christopher Matta <cmatta@mapr.com>
> > > > wrote:
> > > >
> > > > > I’ve been running across errors in Drill when a JSON record is
> > invalid.
> > > > To
> > > > > reduce the number of these errors, I wrote this small, simple
> > > application
> > > > > that will open a specified file, check if each line is a valid JSON
> > > > record,
> > > > > and error if it’s not:
> > > > >
> > > > > https://github.com/cjmatta/jsonr
> > > > >
> > > > > Usage:
> > > > >
> > > > > [cmatta@ip-172-16-1-173 jsonar]$ ./jsonar -f
> > > > > ../tweets/2014/11/24/21/tweets.json
> > > > > Checking for valid JSON in ../tweets/2014/11/24/21/tweets.json
> > > > > CWARNING:root:JSON load error on line 16640 of
> > > > > ../tweets/2014/11/24/21/tweets.json
> > > > > WARNING:root:JSON load error on line 16641 of
> > > > > ../tweets/2014/11/24/21/tweets.json
> > > > > WARNING:root:JSON load error on line 16642 of
> > > > > ../tweets/2014/11/24/21/tweets.json
> > > > > Checking line 17000
> > > > > Done.
> > > > >
> > > > > Please check it out, use it, contribute back if there’s something
> > > broken
> > > > or
> > > > > missing.
> > > > >
> > > > > Chris Matta
> > > > > cmatta@mapr.com
> > > > > 215-701-3146
> > > > > ​
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > >  Steven Phillips
> > >  Software Engineer
> > >
> > >  mapr.com
> > >
> >
>
>
>
> --
>  Steven Phillips
>  Software Engineer
>
>  mapr.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message