Mailing-List: contact dev-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Received-SPF: pass (nike.apache.org: domain of cwestin@maprtech.com designates
 209.85.215.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAKa9qDnDMHZdXHHN9nT-=xRNS58E-5jWouOJsOJSdaJJi7wx9Q@mail.gmail.com>
References: 
 <CAMpYv7B79LaKtkA5ie1z9Ot77vj16=0sig7=TTM=ruft-5Mhbw@mail.gmail.com>
	<CAKa9qDnDMHZdXHHN9nT-=xRNS58E-5jWouOJsOJSdaJJi7wx9Q@mail.gmail.com>
Date: Mon, 26 Jan 2015 15:11:56 -0800
Message-ID: 
 <CANxDS=+7g_NeA1JQU2CYOWUGSc9xyjkh1Gequs49BrGmH_=_5g@mail.gmail.com>
Subject: Re: [DISCUSS] Change default json read behavior for numbers
From: Chris Westin <cwestin@maprtech.com>
To: dev@drill.apache.org
Content-Type: multipart/alternative; boundary=047d7b3441c6bea573050d964325

--047d7b3441c6bea573050d964325
Content-Type: text/plain; charset=UTF-8

JavaScript (and therefore JSON) defines all numbers to be 64 bit floating
point, even if they're written without decimals. So, if someone is writing
JSON, this would be their expectation. I would read them all as doubles.

=> http://www.w3schools.com/js/js_numbers.asp

On Mon, Jan 26, 2015 at 2:17 PM, Jacques Nadeau <jacques@apache.org> wrote:

> Writing zero int to a float column should be allowed.  Basically, if we
> found a float previously and then we run across a zero, that should be
> accepted.  This doesn't fix the situation where the first value was zero
> but definitely fixes many situations.  I'm up for a second option to treat
> all numbers as doubles but I'm not in support of it for the default as once
> we finish embedded types, this would be our desired behavior.
>
> On Mon, Jan 26, 2015 at 1:36 PM, Jason Altekruse <altekrusejason@gmail.com
> >
> wrote:
>
> > Hello Drillers,
> >
> > I am currently working on improving the error reporting in the JSON
> reader
> > to help users with files that Drill cannot read using the default
> > configuration today.
> >
> > As a part of this change I think it may be useful to change the default
> > behavior for reading numbers in JSON documents. Currently we fail on a
> > simple case with reading numbers with decimal points and then hit a value
> > of 0 (or any number without a decimal point) in a later record. The
> reason
> > for the current behavior is to allow better precision in the case of
> files
> > with only integers. The issue however is that we currently fail on the
> > basic case with a mix of intergers and decimal numbers. See [1] for more
> > discussion on this.
> >
> > I propose that we switch the JSON reader to read all numbers as doubles
> by
> > default. The reader already contains a workaround that allows lossless
> > casting to integers and decimal types with some extra computational
> > overhead using all_text_mode, see more info below. [2]
> >
> > Please share your thoughts on this change.
> >
> > [1] https://issues.apache.org/jira/browse/DRILL-1460
> > [2] https://issues.apache.org/jira/browse/DRILL-2071
> >
> > -Jason
> >
>

--047d7b3441c6bea573050d964325--