hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arpit Gupta <ar...@hortonworks.com>
Subject Re: WEBHDFS API GETCONTENTSUMMARY issue
Date Fri, 11 Jan 2013 00:30:32 GMT
forgot to mention that the path you are using in the api will also count towards the directory
count.

--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/

On Jan 10, 2013, at 3:55 PM, Arpit Gupta <arpit@hortonworks.com> wrote:

> Rodrigo
> 
> GETCONTENTSUMMARY will return the summary of everything under the path you specified,
even the subdirectories. So i would suggest take a look in the directories and see what content
they have and then the numbers should add up.
> 
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
> 
> On Jan 10, 2013, at 6:59 AM, "Pastrana, Rodrigo (RIS-BCT)" <Rodrigo.Pastrana@lexisnexis.com>
wrote:
> 
>> I’m using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY
counts aren’t returning expected counts.
>>  
>> For example, when I query content summary for the directory /user/hadoop/tutorial,
webhdfs returns the following:
>>  
>> x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
>> {"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
>>  
>> But looking at the content of that dir through the web portal, I see 6 files and
3 subdirs:
>>  
>> accounts
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-12-12 10:59
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-07-31 16:48
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2-parts
>> dir
>> 2012-08-17 15:34
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-11 13:19
>> rw-r--r--
>> hadoop
>> supergroup
>> persons-parts
>> dir
>> 2012-12-12 10:35
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons2
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-20 13:53
>> rw-r--r--
>> hadoop
>> supergroup
>> persons3
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-12-12 10:53
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts
>> file
>> 59.88 MB
>> 1
>> 64 MB
>> 2012-12-06 12:26
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts-parts
>> dir
>> 2012-12-06 15:00
>> rwxr-xr-x
>> hadoop
>> supergroup
>>  
>> Can anybody help make sense of the summary numbers?
>>  
>>  
>> Thanks, Rodrigo.
>>  
>> 
>> 
>> The information contained in this e-mail message is intended only for the personal
and confidential use of the recipient(s) named above. This message may be an attorney-client
communication and/or work product and as such is privileged and confidential. If the reader
of this message is not the intended recipient or an agent responsible for delivering it to
the intended recipient, you are hereby notified that you have received this document in error
and that any review, dissemination, distribution, or copying of this message is strictly prohibited.
If you have received this communication in error, please notify us immediately by e-mail,
and delete the original message.
>> 
> 


Mime
View raw message