impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcel Kornacker (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (IMPALA-2400) Unpredictable locality behavior for reading Parquet files
Date Wed, 15 Mar 2017 15:41:41 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Marcel Kornacker reassigned IMPALA-2400:
----------------------------------------

    Assignee: Joe McDonnell

> Unpredictable locality behavior for reading Parquet files
> ---------------------------------------------------------
>
>                 Key: IMPALA-2400
>                 URL: https://issues.apache.org/jira/browse/IMPALA-2400
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Perf Investigation
>    Affects Versions: Impala 2.3.0
>            Reporter: Mostafa Mokhtar
>            Assignee: Joe McDonnell
>            Priority: Minor
>              Labels: ramp-up
>         Attachments: LocalRead.txt, RemoteRead.txt
>
>
> When running the query below I noticed exceptionally high variance even after running
"invalidate metadata". 
> select * from tpch_bin_flat_parquet_30.lineitem limit 10;
> * Fetched 10 row(s) in 1.08s
> WARNINGS: Read 139.48 MB of data across network that was expected to be local. Block
locality metadata for table 'tpch_bin_flat_parquet_30.lineitem' may be stale. Consider running
"INVALIDATE METADATA `tpch_bin_flat_parquet_30`.`lineitem`".
> * Fetched 10 row(s) in 1.32s
> * Fetched 10 row(s) in 0.09s
> * Fetched 10 row(s) in 1.08s
> * "invalidate metadata"
> * Fetched 10 row(s) in 0.89s
> * Fetched 10 row(s) in 0.07s
> WARNINGS: Read 76.15 MB of data across network that was expected to be local. Block locality
metadata for table 'tpch_bin_flat_parquet_30.lineitem' may be stale. Consider running "INVALIDATE
METADATA `tpch_bin_flat_parquet_30`.`lineitem`".
> * Fetched 10 row(s) in 1.11s
> * Fetched 10 row(s) in 0.73s
> * Fetched 10 row(s) in 0.09s
> The behavior above is tied to Parquet tables and doesn't repro against text data.
> Profile files attached.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message