tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keuntae Park (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TAJO-223) Maximize disk read bandwidth utilization of StorageManagerV2 by moving Tuple creation role to next()
Date Wed, 02 Oct 2013 06:53:23 GMT
Keuntae Park created TAJO-223:
---------------------------------

             Summary: Maximize disk read bandwidth utilization of StorageManagerV2 by moving
Tuple creation role to next()
                 Key: TAJO-223
                 URL: https://issues.apache.org/jira/browse/TAJO-223
             Project: Tajo
          Issue Type: Improvement
            Reporter: Keuntae Park


Currently, Tuple creation mechanism of StorageManagerV2 is as follows:
1) At file scan, scheduled scanner reads data from disk, makes a Tuple, and insert it to the
Tuple pool
2) next() of the scanner just pulls an already created Tuple from the Tuple pool asynchronously

Because of Tuple creation time, scanner cannot fully use its time to read disk, which results
in less disk read bandwidth utilization

So, if Tuple creation role is moved to next() and scanners spend their whole time to read
file at file scan,
we can fully utilize disk read bandwidth 



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message