tajo-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TAJO-717) Improve file splitting for large number of splits
Date Thu, 10 Apr 2014 06:28:17 GMT

    [ https://issues.apache.org/jira/browse/TAJO-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13965057#comment-13965057
] 

Hudson commented on TAJO-717:
-----------------------------

SUCCESS: Integrated in Tajo-master-build #164 (See [https://builds.apache.org/job/Tajo-master-build/164/])
TAJO-717: Improve file splitting for large number of splits. (jinho) (jinossy: rev d99bd085e4f02bd11a9133c6bef8942b9dc27723)
* tajo-storage/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java
* tajo-storage/src/main/java/org/apache/tajo/storage/AbstractStorageManager.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java
* CHANGES.txt
* tajo-storage/pom.xml
* tajo-storage/src/test/java/org/apache/tajo/storage/TestStorageManager.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/DefaultTaskScheduler.java
* tajo-storage/src/main/java/org/apache/tajo/storage/fragment/FileFragment.java


> Improve file splitting for large number of splits
> -------------------------------------------------
>
>                 Key: TAJO-717
>                 URL: https://issues.apache.org/jira/browse/TAJO-717
>             Project: Tajo
>          Issue Type: Improvement
>          Components: storage
>            Reporter: Jinho Kim
>            Assignee: Jinho Kim
>             Fix For: 0.8-incubating, 1.0-incubating
>
>         Attachments: TAJO-717.patch, TAJO-717_20140404_14:07:24.patch, TAJO-717_20140407_17:28:49.patch
>
>
> In currently, The storageManager invoke the getFileBlockStorageLocations() per input
path, it occurred too many rpc to the associated datanodes
> * reducing remote call to datanode



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message