cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Burwell <jburw...@basho.com>
Subject Re: S3 API to get S3 bucket usage data?
Date Wed, 15 Jan 2014 18:58:59 GMT
Min,

Yes, this feels incredibly out of scope for CloudStack.  Given the number of buckets and objects
involved, getting this information realtime in a scalable (both UX and system) manner would
be very difficult.

Thanks,
-John
  
On Jan 15, 2014, at 1:54 PM, Min Chen <min.chen@citrix.com> wrote:

> Thanks a lot for your clarification, John. This issue was raised by some of CloudStack
S3 customers, and they are questioning why we didn't display the usage information for their
S3 secondary storage in CloudStack Dashboard UI, which includes both size of objects in the
bucket and physical space in the cluster a bucket occupies. I totally agree with you that
these will be vendor specific, and we should not bother to display them on CloudStack UI.
> 
> Thanks
> -min
> 
> From: John Burwell <jburwell@basho.com>
> Date: Wednesday, January 15, 2014 10:42 AM
> To: Min Chen <min.chen@citrix.com>
> Cc: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>, Thomas O'Dowd <tpodowd@cloudian.com>
> Subject: Re: S3 API to get S3 bucket usage data?
> 
> Min,
> 
> Such a notion is completely anachronistic to a client object store due to the abstraction
— it is solely an operational.  Hence, you won’t find space information available in object
store client APIs.
> 
> The first question is what space information are you seeking?  The logical size of the
objects in the bucket or how much physical space in the cluster a bucket occupies?  The first
is simply a sum of all object sizes in the bucket.  The second is very difficult to answer
through the client interface due to factors such as the intra-cluster replication factor for
the bucket, the use of mechanisms such as erasure encoding and compression, and the number
of data centers to which the object is replicated. For these reasons, space information is
typically only exposed via the operational monitoring interfaces, and would likely need to
be acquired through vendor specific drivers.
> 
> Thanks,
> -John 
> 
> On Jan 15, 2014, at 1:27 PM, Min Chen <min.chen@citrix.com> wrote:
> 
>> Hi there,
>> 
>> Does anybody out there know if AWS S3 provides programmatical way to get bucket-level
usage data? For example, used space or available space as we current show in CloudStack dashboard
for NFS secondary storage. I googled around the web, and didn't find any S3 api to do that.
Any comments will be appreciated.
>> 
>> Thanks
>> -min
> 


Mime
View raw message