impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sailesh Mukil (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-5378) Disk IO manager needs to understand ADLS
Date Fri, 02 Jun 2017 19:55:04 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-5378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sailesh Mukil resolved IMPALA-5378.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.9.0

Commit in:
https://github.com/apache/incubator-impala/commit/9c4f75a6a877f2b5e56d24f1e73fcf72d74f9129

> Disk IO manager needs to understand ADLS
> ----------------------------------------
>
>                 Key: IMPALA-5378
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5378
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 2.9.0
>            Reporter: Joe McDonnell
>            Assignee: Sailesh Mukil
>            Priority: Critical
>             Fix For: Impala 2.9.0
>
>
> The Disk IO Manager has customized support for S3 and remote HDFS that allows for these
to use a separate queue and have a customized number of IO threads. Currently, ADLS does not
have this support.
> Based on the code in DiskIoMgr::Init and DiskIoMgr::AssignQueue, IOs for ADLS would currently
be put in local disk queues. Since local disks are considered rotational unless we can confirm
otherwise by looking at the /sys filesystem, this means that THREADS_PER_ROTATIONAL_DISK=1
will apply as the thread count.
> This reduced thread count may hurt ADLS performance.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message