drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tridib Samanta <tridib.sama...@live.com>
Subject RE: Enable caching in Drill
Date Thu, 30 Oct 2014 05:06:05 GMT
Hmm... 
Each line of my file has one json document. The file is in HDFS. Though I am not familiar
with Drill's code, but looks like it reads each line as separate record and process them.
Not sure why it can't split the files with 1 million json records. Anyway it treats each line
as independent record.
 
> Date: Wed, 29 Oct 2014 21:54:24 -0700
> Subject: RE: Enable caching in Drill
> From: jacques@apache.org
> To: drill-user@incubator.apache.org
> 
> Drill doesn't currently cache data and relies on the underlying file system
> cache.
> 
> Also,  json is not splittable so adding nodes with a single json file will
> generally have little impact.
> On Oct 29, 2014 9:48 PM, "Tridib Samanta" tridib.samanta@live.com wrote:
> 
> > select count(*) from myhdfs.json.`x00.json`;
> >
> > Surprising thing is, I get same performance when I use 1 drillbit compare
> > to 4 drillbits.
> >
> > > Date: Thu, 30 Oct 2014 10:08:04 +0530
> > > Subject: Re: Enable caching in Drill
> > > From: mufeed.usman@gmail.com
> > > To: drill-user@incubator.apache.org
> > >
> > > The query didn't get through :-).
> > >
> > >
> > > ---
> > > Mufeed Usman
> > > My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | My
> > > Social Cause <http://www.vision2016.org.in/> | My Blogs : LiveJournal
> > > <http://mufeed.livejournal.com>
> > >
> > >
> > >
> > >
> > > On Thu, Oct 30, 2014 at 2:54 AM, Tridib Samanta <tridib.samanta@live.com
> > >
> > > wrote:
> > >
> > > > Hello,
> > > > I am doing a count query like bellow. I understand that it will take
> > long
> > > > time at first attempt. But not sure why it takes same time in
> > subsequent
> > > > execution. Will I have to enable caching or something like that?
> > > >
> > > > Thanks
> > > > Tridib
> > > >
> >
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message