hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nishant Bangarwa (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20279) HiveContextAwareRecordReader slows down Druid Scan queries.
Date Tue, 31 Jul 2018 12:13:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563552#comment-16563552
] 

Nishant Bangarwa commented on HIVE-20279:
-----------------------------------------

+cc [~ashutoshc] Can you point to more details about the header/footer functionality. 
I don't think we need to check any of the checks for header/footer buffer for Druid - 
https://github.com/nishantmonu51/hive/blob/ba0217ff17501fb849d8999e808d37579db7b4f1/ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java#L317
Can you please confirm ? 

> HiveContextAwareRecordReader slows down Druid Scan queries. 
> ------------------------------------------------------------
>
>                 Key: HIVE-20279
>                 URL: https://issues.apache.org/jira/browse/HIVE-20279
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Nishant Bangarwa
>            Assignee: Nishant Bangarwa
>            Priority: Major
>         Attachments: scan2.svg
>
>
> HiveContextAwareRecordReader add lots of overhead for Druid Scan Queries. 
> See attached flame graph. 
> Looks like the operations for checking for existence of footer/header buffer takes most
of time For druid and other storage handlers that do not have footer buffer we should skip
the logic for checking the existence for storage handlers atleast. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message