hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <>
Subject [jira] [Commented] (HIVE-4936) PTest increase performance by parallelizing rsync
Date Thu, 25 Jul 2013 20:47:50 GMT


Brock Noland commented on HIVE-4936:

I implemented a fix for this by rsyncing master \-> slave (thread 0) once and then executing
rsync on the slave (thread 0) \-> slave (thread 1 and 2) and achieved significant speedup.
I will post a patch soon.
> PTest increase performance by parallelizing rsync
> -------------------------------------------------
>                 Key: HIVE-4936
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Brock Noland
>            Assignee: Brock Noland
> Currently the bottom neck in PTest2 when adding hosts is the rsync to the slaves. Today
if we have 3 "threads" which will execute on a slave we rsync the source from the master to
the slave 3 times. This causes the process to be CPU bound on the master.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message