drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@apache.org>
Subject RE: Enable caching in Drill
Date Thu, 30 Oct 2014 05:28:52 GMT
Line breaks are allowed in the JSON standard even if your file doesn't have
them.
On Oct 29, 2014 10:07 PM, "Tridib Samanta" <tridib.samanta@live.com> wrote:

> Hmm...
> Each line of my file has one json document. The file is in HDFS. Though I
> am not familiar with Drill's code, but looks like it reads each line as
> separate record and process them. Not sure why it can't split the files
> with 1 million json records. Anyway it treats each line as independent
> record.
>
> > Date: Wed, 29 Oct 2014 21:54:24 -0700
> > Subject: RE: Enable caching in Drill
> > From: jacques@apache.org
> > To: drill-user@incubator.apache.org
> >
> > Drill doesn't currently cache data and relies on the underlying file
> system
> > cache.
> >
> > Also,  json is not splittable so adding nodes with a single json file
> will
> > generally have little impact.
> > On Oct 29, 2014 9:48 PM, "Tridib Samanta" tridib.samanta@live.com wrote:
> >
> > > select count(*) from myhdfs.json.`x00.json`;
> > >
> > > Surprising thing is, I get same performance when I use 1 drillbit
> compare
> > > to 4 drillbits.
> > >
> > > > Date: Thu, 30 Oct 2014 10:08:04 +0530
> > > > Subject: Re: Enable caching in Drill
> > > > From: mufeed.usman@gmail.com
> > > > To: drill-user@incubator.apache.org
> > > >
> > > > The query didn't get through :-).
> > > >
> > > >
> > > > ---
> > > > Mufeed Usman
> > > > My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400>
|
> My
> > > > Social Cause <http://www.vision2016.org.in/> | My Blogs :
> LiveJournal
> > > > <http://mufeed.livejournal.com>
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Oct 30, 2014 at 2:54 AM, Tridib Samanta <
> tridib.samanta@live.com
> > > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > > I am doing a count query like bellow. I understand that it will
> take
> > > long
> > > > > time at first attempt. But not sure why it takes same time in
> > > subsequent
> > > > > execution. Will I have to enable caching or something like that?
> > > > >
> > > > > Thanks
> > > > > Tridib
> > > > >
> > >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message