hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <>
Subject [jira] [Commented] (HIVE-21305) LLAP: Option to skip cache for ETL queries
Date Tue, 26 Mar 2019 21:24:00 GMT


Prasanth Jayachandran commented on HIVE-21305:

[~gopalv] LLAP sets ROWS_EMITTED as VRB batch size, whereas tez counts VRB as 1 record, hence
the difference. 

[~rajesh.balamohan] This disables read through cache behavior for ETL queries reading from
text table and writing elsewhere (which could be temp table as well). 

> LLAP: Option to skip cache for ETL queries
> ------------------------------------------
>                 Key: HIVE-21305
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: llap
>    Affects Versions: 4.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Major
>         Attachments: HIVE-21305.1.patch, HIVE-21305.2.patch, HIVE-21305.3.patch
> To avoid ETL queries from polluting the cache, would be good to detect such queries at
compile time and optional skip llap io for such queries. 
> org.apache.hadoop.hive.ql.parse.QBParseInfo.hasInsertTables() is the simplest way  to
catch ETL queries.

This message was sent by Atlassian JIRA

View raw message