impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <>
Subject [jira] [Created] (IMPALA-5169) Parallelise read I/O of BufferPool::Pin()
Date Wed, 05 Apr 2017 18:33:41 GMT
Tim Armstrong created IMPALA-5169:

             Summary: Parallelise read I/O of BufferPool::Pin()
                 Key: IMPALA-5169
             Project: IMPALA
          Issue Type: Improvement
          Components: Backend
    Affects Versions: Impala 2.9.0
            Reporter: Tim Armstrong
            Assignee: Tim Armstrong

Currently read I/O in BufferPool is synchronous. In some cases this can lead to poor resource
utilisation and I/O throughput, because:
* We don't dispatch parallel reads to multiple scratch disks or high-throughput SSDs
* Issuing reads of contiguous scratch ranges at the same time improves the odds that the second
read can be served without a disk seek or by the disks internal cache.

* Expose a batched Pin() interface that can pin multiple buffers at the same time
* Expose an asynchronous Pin() interface that can start the read, and allow the client to
wait for it.

The first alternative is probably simplest.

This message was sent by Atlassian JIRA

View raw message