hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Performance impact of curl command on linux server
Date Sat, 05 Sep 2015 08:47:10 GMT
That depends - what resource did it consume, concerning your admins? CPU?

Were you uploading using the chunked technique?

On Fri, Sep 4, 2015 at 9:06 PM Shashi Vishwakarma <shashi.vish123@gmail.com>

> Hi
> I have been using curl command for ingesting data into HDFS using WebHDFS.
> Admins are saying that resource utilization for curl is very high which is
> hampering server performance.
> I would like to know if curl can use so much of resource which could lead
> to performance issue.
> Just to give more idea , I have been ingesting around 25Gb of data into
> HDFS. Not to sure if curl can cause so much of issue with such a small data
> set.
> Any suggestion would be a great help.
> Thanks,
> Shashi

View raw message