tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "hyoungjunkim (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (TAJO-178) Implements StorageManager for Vectorized Engine
Date Wed, 11 Sep 2013 12:02:53 GMT

     [ https://issues.apache.org/jira/browse/TAJO-178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

hyoungjunkim updated TAJO-178:
------------------------------

    Attachment: TAJO-178.path

In the current implementation origin StorageManager and new StorageManager are optionally
available. Setting property name is "tajo.storage.manager.v2" and default value is false.
Performance compared to the origin StorageManager is similar, but is expected to continue
to improve performance.
                
> Implements StorageManager for Vectorized Engine
> -----------------------------------------------
>
>                 Key: TAJO-178
>                 URL: https://issues.apache.org/jira/browse/TAJO-178
>             Project: Tajo
>          Issue Type: Improvement
>          Components: storage
>    Affects Versions: 0.2-incubating
>            Reporter: hyoungjunkim
>            Assignee: hyoungjunkim
>         Attachments: TAJO-178.path, tajo_storage_manager.png
>
>
> The current StorageManager does not provide scan scheduling function. All scan operations
run concurrently. This is the cause of random disk access and disk read performance is not
good.
> The proposed StorageManager is based on double buffering. Each disk has a scheduler to
schedule by order of scanned adjust. Each Scanner has a InputStream and a Tuple pool. The
next() operation of ScanNode is blocked until Tuple pool is filled. Assigned Scanner by the
scheduler read data(xMB) and fills Tuple Pool and notifies to next() operation. After scanning
Scanner re-enter DiskScanQueue.
> In this way Scanner can pass column vector to Vectorized Query Engine.
> See the attached file.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message