hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vishwakarma, Chhaya" <Chhaya.Vishwaka...@Thinkbiganalytics.com>
Subject Hdfs put VS webhdfs
Date Mon, 27 Jul 2015 12:15:40 GMT
I'm loading 28 GB file in hadoop hdfs using webhdfs and it takes ~25 mins to load. I tried
loading same file using hdfs put and It took ~6 mins. Why there is so much difference in performance?
What is recommended to use? Can somebody explain or direct me to some good link it will be
really helpful.
Below us the command I'm using
curl -i --negotiate -u: -X PUT "http://$hostname:$port/webhdfs/v1/$destination_file_location/$source_filename.temp?op=CREATE&overwrite=true"

this will redirect to a datanode address which I use in next step to write the data


View raw message