impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IMPALA-5169) Parallelise read I/O of BufferPool::Pin()
Date Wed, 05 Apr 2017 18:33:41 GMT
Tim Armstrong created IMPALA-5169:
-------------------------------------

             Summary: Parallelise read I/O of BufferPool::Pin()
                 Key: IMPALA-5169
                 URL: https://issues.apache.org/jira/browse/IMPALA-5169
             Project: IMPALA
          Issue Type: Improvement
          Components: Backend
    Affects Versions: Impala 2.9.0
            Reporter: Tim Armstrong
            Assignee: Tim Armstrong


Currently read I/O in BufferPool is synchronous. In some cases this can lead to poor resource
utilisation and I/O throughput, because:
* We don't dispatch parallel reads to multiple scratch disks or high-throughput SSDs
* Issuing reads of contiguous scratch ranges at the same time improves the odds that the second
read can be served without a disk seek or by the disks internal cache.

* Expose a batched Pin() interface that can pin multiple buffers at the same time
* Expose an asynchronous Pin() interface that can start the read, and allow the client to
wait for it.

The first alternative is probably simplest.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message