drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aman Sinha <amansi...@apache.org>
Subject Re: Drill over Hive
Date Wed, 13 Dec 2017 17:46:20 GMT
I don't think we test Drill in this 3-tier configuration .. i.e
Drill-->Hive-->Hbase.  It is mainly Drill-->Hive or Drill-->Hbase.
We would need to investigate why the filter pushdown did not get completely
propagated down to Hbase.    Can you pls file a JIRA ?

-Aman

On Wed, Dec 13, 2017 at 4:00 AM, Dor Ben Dov <dor.ben-dov@amdocs.com> wrote:

> Kunal
> Correct this is the exact issue, it looks like drill is trying to push
> while hive doesn't pass it.
> When you work without hive, it works fine.
> +
> As result - you get very bad performance.
>
> Regards,
> Dor
>
> -----Original Message-----
> From: Kunal Khatua [mailto:kkhatua@mapr.com]
> Sent: יום ד 13 דצמבר 2017 09:43
> To: dev@drill.apache.org
> Subject: RE: Drill over Hive
>
> It appears you are trying to use Drill to query an Hbase via Hive. It is
> possible that Drill is not able to push down the filters to Hive, though
> I'm not sure why.
>
> Aman/Chunhui
> Any idea why is Drill triggering a full table scan?
>
> From: Dor Ben Dov [mailto:dor.ben-dov@amdocs.com]
> Sent: Tuesday, December 12, 2017 11:39 PM
> To: dev@drill.apache.org
> Subject: Drill over Hive
>
> Hi
>
> This is our query
> select * from hive.mytable where key >= '0001:10:2017:0410:0000000000003157781'
>  and key < '0001:10:2017:0410:0000000000003157782';
>
> Same query in Drill over hbase takes less than a sec, in hive it takes few
> seconds and with drill over hive it quits with a timeout.
> When you look in the sql profile of drill (using the web), you see a table
> full scan for millions of records, while actually it was supposed to return
> 9 rows.
>
> Does Drill on top of hive is using the key to access only the relevant
> region of the table?
>
> Regards,
> Dor Ben Dov
> Research Manager, Digital - BSS - Research
> +972-9-7764043   (office)
> +972-52-3573492 (mobile)
> [cid:image001.png@01D28AC9.B559B360]
>
> Follow us on Facebook<https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__www.facebook.com_amdocs_&d=DwMFAg&c=cskdkSMqhcnjZxdQVpwTXg&r=-
> cT6otg6lpT_XkmYy7yg3A&m=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn
> 6q32M&s=bS0vtglEQKbSVkT6_3T5N1ckKa-48J2rFdz0A-GkFxg&e=>, Twitter<
> https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_amdocs&d=
> DwMFAg&c=cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=DAE0fF7_
> P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M&s=HwJx9gvHHvHV9u0nW_
> 0dLjEPhSGcfsWLH-iTUFNV6mc&e=>, LinkedIn<https://urldefense.
> proofpoint.com/v2/url?u=http-3A__www.linkedin.com_company_
> amdocs&d=DwMFAg&c=cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=
> DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M&s=
> cjcmCjbbkgov6zg8GJnuJOOAm6G9HTtY4ppZ170IDb4&e=>, YouTube<
> https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__www.youtube.com_amdocsinc&d=DwMFAg&c=cskdkSMqhcnjZxdQVpwTXg&r=-
> cT6otg6lpT_XkmYy7yg3A&m=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M&s=
> fp9qmbPyYRdiNYDwUDH44HUALaqsbKvfWBv_Dcy7UAo&e=>, Google+<
> https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_
> 105657940751678445194&d=DwMFAg&c=cskdkSMqhcnjZxdQVpwTXg&r=-
> cT6otg6lpT_XkmYy7yg3A&m=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn
> 6q32M&s=n0coUEV3Sf7O_MbrqI7cAr5f6d7OG_vL1jK2n3z-AvM&e=> and the Amdocs
> blog network<https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__blogs.amdocs.com_&d=DwMFAg&c=cskdkSMqhcnjZxdQVpwTXg&r=-
> cT6otg6lpT_XkmYy7yg3A&m=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn
> 6q32M&s=RxKGDS-92_svSBQlnTdpn8NSr3-1A9Mcp7UbmZsv1bE&e=>| Digital BSS
> Research@Yammer<https://urldefense.proofpoint.com/v2/
> url?u=https-3A__www.yammer.com_amdocs.com_-23_threads_
> inGroup-3Ftype-3Din-5Fgroup-26feedId-3D7043838&d=DwMFAg&c=
> cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=DAE0fF7_
> P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M&s=A20YnBgQgpBLzoXN-
> rOK49KCStW9haqs7atynBoQz-s&e=>
>
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement, you may review at
> https://www.amdocs.com/about/email-disclaimer<https://
> urldefense.proofpoint.com/v2/url?u=https-3A__www.amdocs.
> com_about_email-2Ddisclaimer&d=DwMFAg&c=cskdkSMqhcnjZxdQVpwTXg&r=-
> cT6otg6lpT_XkmYy7yg3A&m=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M&s=
> xqkKqSeAK9HGozYmTy7lO0eKTkY4CG5npyWKUM9XeAY&e=>
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message