incubator-drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Parth Chandra <pchan...@maprtech.com>
Subject Re: Query on CSV data not returning to the sqlline prompt
Date Mon, 13 Oct 2014 17:50:12 GMT
There is a mechanism in place where the fragments running on different
drillbits are informed that they can stop producing data. The same
mechanism is used to cancel queries (for example when you Ctrl-C from
sqlline), as well as when the limit operator is used.
Initial testing shows that this is working fine, but there is some
condition where it is not working correctly.
I'll be working on trying to reproducing this in a debug environment.

On Sat, Oct 11, 2014 at 1:36 PM, Ted Dunning <ted.dunning@gmail.com> wrote:

> The question implied by what Chris asks is whether there is a mechanism
> whereby a consumer can notify a producer that no more data is required?
>
> In the case of this query, once Chris has his 10 rows, shouldn't the
> consumer signal (transitively) all upstream parts of the query that they
> can go home now?
>
>
> On Sat, Oct 11, 2014 at 1:24 PM, Christopher Matta <cmatta@mapr.com>
> wrote:
>
> > My data is in a directory on a 4 node MapR cluster, the csv files are
> > partitioned into 4 dirs, each with 100 csv files with 10 million records,
> > totaling 4 billion records.
> >
> > When I do a SELECT * FROM FACT LIMIT 10; the data returns, however the
> > sqlline prompt never comes back and the profile says the query is still
> > running. I can ctrl+c the query which returns the10 rows selected
> (360.949
> > seconds) report and the sqlline prompt, but the profiles page shows the
> > query as still running.
> >
> > The drillbit.log doesn’t show any errors during this.
> >
> > Seems like a bug, but I would like to gather more evidence before
> > submitting it, any ideas?
> >
> > Chris Matta
> > cmatta@mapr.com
> > 215-701-3146
> > ​
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message