tajo-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hyunsik Choi" <hyun...@apache.org>
Subject Re: Review Request 19821: TAJO-717: Improve file splitting for large number of splits
Date Thu, 10 Apr 2014 03:23:53 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19821/#review39978
-----------------------------------------------------------

Ship it!


+1

This patch includes nice code optimization for splitting and enough unit tests. The patch
looks good to me.

- Hyunsik Choi


On April 7, 2014, 5:30 p.m., Jinho Kim wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19821/
> -----------------------------------------------------------
> 
> (Updated April 7, 2014, 5:30 p.m.)
> 
> 
> Review request for Tajo.
> 
> 
> Bugs: TAJO-717
>     https://issues.apache.org/jira/browse/TAJO-717
> 
> 
> Repository: tajo
> 
> 
> Description
> -------
> 
> In currently, The storageManager invoke the getFileBlockStorageLocations() per input
path, it occurred too many rpc to the associated datanodes
> 
> 
> Diffs
> -----
> 
>   tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/DefaultTaskScheduler.java
61fa84e 
>   tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java
df8b31b 
>   tajo-storage/pom.xml b9a162a 
>   tajo-storage/src/main/java/org/apache/tajo/storage/AbstractStorageManager.java a7ed981

>   tajo-storage/src/main/java/org/apache/tajo/storage/fragment/FileFragment.java ea8bf9f

>   tajo-storage/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java bbb9df1 
>   tajo-storage/src/test/java/org/apache/tajo/storage/TestStorageManager.java 083670a

> 
> Diff: https://reviews.apache.org/r/19821/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jinho Kim
> 
>


Mime
View raw message