hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Vary (JIRA)" <>
Subject [jira] [Commented] (HIVE-21305) LLAP: Option to skip cache for ETL queries
Date Fri, 22 Feb 2019 08:00:00 GMT


Peter Vary commented on HIVE-21305:

[~prasanth_j]: We decide if the query inserts into a table then we do not add entries to the
cache, but we still use the existing cache elements?

What do you think about using the row number statistics for the input tables? ETL queries
still might use/and reuse some smaller tables in joins. We might be better off caching the
small tables but skipping the big ones.




> LLAP: Option to skip cache for ETL queries
> ------------------------------------------
>                 Key: HIVE-21305
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: llap
>    Affects Versions: 4.0.0
>            Reporter: Prasanth Jayachandran
>            Priority: Major
> To avoid ETL queries from polluting the cache, would be good to detect such queries at
compile time and optional skip llap io for such queries. 
> org.apache.hadoop.hive.ql.parse.QBParseInfo.hasInsertTables() is the simplest way  to
catch ETL queries.

This message was sent by Atlassian JIRA

View raw message