tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henry Saputra (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TAJO-5) Cache mechanism to keep instances of opened BSTIndexs in PullServerAuxService
Date Tue, 24 Sep 2013 22:22:03 GMT

    [ https://issues.apache.org/jira/browse/TAJO-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776838#comment-13776838
] 

Henry Saputra commented on TAJO-5:
----------------------------------

If no one object I can take a look at this one
                
> Cache mechanism to keep instances of opened BSTIndexs in PullServerAuxService
> -----------------------------------------------------------------------------
>
>                 Key: TAJO-5
>                 URL: https://issues.apache.org/jira/browse/TAJO-5
>             Project: Tajo
>          Issue Type: Improvement
>          Components: repartitioning
>            Reporter: Hyunsik Choi
>              Labels: newbie
>
> PullServerAuxService is an auxiliary service of Yarn to repartition intermediate data.
It is similar to ShuffleHandler of MRv2. PullServerAuxService supports hash repartition as
well as range repartition. It works through netty-based HTTP web server.
> For retrieval of range partition data, PullServerAuxService uses a binary search tree
(BSTIndex.java). For each request of range partitioned data, it opens BSTIndex every time.
It may cause overheads. See messageReceived in PullServer and getFileChunks in PullServerAuxService.
> If PullServerAuxService uses some cache mechanism that keeps instances of opened BSTIndex
and data files, it could get rid of this overhead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message