hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eremikhin Alexey <a.eremi...@corp.badoo.com>
Subject Re: WebHDFS listing takes 0.7ms per item
Date Fri, 29 Jan 2016 15:30:03 GMT
OK. I've found what was the issue.
It comes from https://issues.apache.org/jira/browse/HDFS-6565
And slows down HDFS listing in 37 times.

I've created a reverse patch to 
https://issues.apache.org/jira/browse/HDFS-6565 .
How can I raise that question for mainline developers?

*2.7.1 without patch
*alexxz@hadoop1.mlan:~> time curl 
'http://hadoop1:50070/webhdfs/v1/tmp/test?op=LISTSTATUS' | wc
   % Total    % Received % Xferd  Average Speed   Time Time     Time  
Current
                                  Dload  Upload   Total Spent    Left  Speed
100 2469k    0 2469k    0     0   335k      0 --:--:-- 0:00:07 --:--:--  
338k
   10002   10002 2528929

real    0m7.397s
user    0m0.084s
sys    0m0.028s


*2.7.1 with patch
*alexxz@hadoop1.mlan:~> time curl 
'http://hadoop1:50070/webhdfs/v1/tmp/test?op=LISTSTATUS' | wc
   % Total    % Received % Xferd  Average Speed   Time Time     Time  
Current
                                  Dload  Upload   Total Spent    Left  Speed
100 2469k    0 2469k    0     0  14.5M      0 --:--:-- --:--:-- --:--:-- 
15.0M
   10002   10002 2528929

real    0m0.180s
user    0m0.100s
sys    0m0.012s
*
*
On 27/01/16 15:24, Eremikhin Alexey wrote:
> Hi,
> I've met a performance issue after migration from Hadoop 1.2.1 to 2.7.1
> Listing of directories in WebHDFS started taking significantly more 
> time than it took before.
> I've performed some experiments and concluded that it takes 0.7 per 
> item in directory.
> That means listing of directory with 10000 objects takes 7 seconds.
> But that was checked on Hadoop 1.2.1 it takes about 700ms (on the same 
> sever).
> Java version does not affect test results (7/8)
>
> Does anyone know about such a behaviour?
>
> There are details on tests and configuration.
> *
> alexxz@hadoop1.mlan:~> hadoop version*
> Hadoop 2.7.1
> Subversion Unknown -r Unknown
> Compiled by root on 2015-09-11T13:58Z
> Compiled with protoc 2.5.0
> From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
> This command was run using 
> /local/hadoop/share/hadoop/common/hadoop-common-2.7.1.jar
>
> *alexxz@hadoop1.mlan:~> java -version*
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>
> *Experiments*
> alexxz@hadoop1.mlan:~> hadoop fs -rm -r /tmp/test
> 16/01/27 11:43:02 INFO fs.TrashPolicyDefault: Namenode trash 
> configuration: Deletion interval = 0 minutes, Emptier interval = 0 
> minutes.
> Deleted /tmp/test
> alexxz@hadoop1.mlan:~> hadoop fs -mkdir /tmp/test
> alexxz@hadoop1.mlan:~> hadoop fs -touchz /tmp/test/{1..10}
> alexxz@hadoop1.mlan:~> /usr/sbin/ab2 -n 40 -c 1 
> 'http://hadoop1:50070/webhdfs/v1/tmp/test?op=LISTSTATUS' | grep 'Time 
> per request:' | head -1
> Time per request:       16.132 [ms] (mean)
> alexxz@hadoop1.mlan:~> hadoop fs -touchz /tmp/test/{1..100}
> alexxz@hadoop1.mlan:~> /usr/sbin/ab2 -n 40 -c 1 
> 'http://hadoop1:50070/webhdfs/v1/tmp/test?op=LISTSTATUS' | grep 'Time 
> per request:' | head -1
> Time per request:       88.119 [ms] (mean)
> alexxz@hadoop1.mlan:~> hadoop fs -touchz /tmp/test/{1..400}
> alexxz@hadoop1.mlan:~> /usr/sbin/ab2 -n 40 -c 1 
> 'http://hadoop1:50070/webhdfs/v1/tmp/test?op=LISTSTATUS' | grep 'Time 
> per request:' | head -1
> Time per request:       302.568 [ms] (mean)
> alexxz@hadoop1.mlan:~> hadoop fs -touchz /tmp/test/{1..1000}
> alexxz@hadoop1.mlan:~> /usr/sbin/ab2 -n 40 -c 1 
> 'http://hadoop1:50070/webhdfs/v1/tmp/test?op=LISTSTATUS' | grep 'Time 
> per request:' | head -1
> Time per request:       746.773 [ms] (mean)
> alexxz@hadoop1.mlan:~> hadoop fs -touchz /tmp/test/{1..2000}
> alexxz@hadoop1.mlan:~> /usr/sbin/ab2 -n 40 -c 1 
> 'http://hadoop1:50070/webhdfs/v1/tmp/test?op=LISTSTATUS' | grep 'Time 
> per request:' | head -1
> Time per request:       1530.869 [ms] (mean)
> alexxz@hadoop1.mlan:~> hadoop fs -touchz /tmp/test/{1..4000}
> alexxz@hadoop1.mlan:~> /usr/sbin/ab2 -n 40 -c 1 
> 'http://hadoop1:50070/webhdfs/v1/tmp/test?op=LISTSTATUS' | grep 'Time 
> per request:' | head -1
> Time per request:       2891.558 [ms] (mean)
> alexxz@hadoop1.mlan:~> hadoop fs -touchz /tmp/test/{1..10000}
> alexxz@hadoop1.mlan:~> /usr/sbin/ab2 -n 40 -c 1 
> 'http://hadoop1:50070/webhdfs/v1/tmp/test?op=LISTSTATUS' | grep 'Time 
> per request:' | head -1
> Time per request:       7625.777 [ms] (mean)
>


Mime
View raw message