incubator-cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sheng Liang <Sheng.Li...@citrix.com>
Subject RE: How to integrate Hadoop to CloudStack
Date Mon, 10 Sep 2012 04:41:07 GMT
I think HDFS could be an option for object storage. It certainly cannot be a requirement for
CloudStack. There are many great implementations of object storage already in the market.
An object storage API front-end on NAS is also a perfectly good option for small-scale deployments.

I hope to be able to make some progress in the thinking soon so we can post some more substantial
details.

Sheng

-----Original Message-----
From: Caleb Call [mailto:calebcall@me.com] 
Sent: Sunday, September 09, 2012 9:34 PM
To: cloudstack-users@incubator.apache.org
Cc: cloudstack-dev@incubator.apache.org
Subject: Re: How to integrate Hadoop to CloudStack

I sure hope HDFS is not required in the future as that's a complete waste when you're already
on redundant Enterprise level storage. 


On Sep 9, 2012, at 7:22 PM, Nguyen Anh Tu <ng.tuna@gmail.com> wrote:

> Hello Shanker,
> 
> I mean that with CS, I want to replace NFS to HDFS. You know NFS is 
> not a suitable solution for storage, because it has not fault-tolerant feature.
> So I want to use HDFS for Secondary Storage in CS. I see this 
> http://www.slideshare.net/kkitase/cloudstack-architecture-future. I 
> think in near future, Hadoop will be used as a storage solution in CS.
> 
> 2012/9/7 Shanker Balan <mail@shankerbalan.net>
> 
>> (Moving to cloudstack-users@ with Bcc to
>> cloudstack-dev@incubator.apache.org)
>> 
>> Hello Nguyen,
>> 
>> Nguyen Anh Tu wrote,
>>> Hi guy,
>>> 
>>> Anyone can help me to integrate Hadoop to CloudStack. I read the 
>>> article "CloudStack and Hadoop: a match made in the cloud" but can 
>>> not find a way to do this.
>> 
>> Could you explain a bit more on what you mean by "Integrating Hadoop 
>> To Cloudstack"? I am not using CS yet, but I have a bunch of use 
>> cases I have been thinking about lately.
>> 
>> You can use Cloudstack to provision Hadoop instances very easily.
>> Cloudstack's
>> bare metal provisioning capabilities allows you to build high 
>> performance clusters.
>> 
>> 
>> http://www.cloudstack.org/blog/63-cloudstack-the-best-kept-secret-in-
>> cloud-computing.html.html
>> 
>> Cloudstack also provides an S3 compatible interface over supported 
>> object stores like Swift and Caringo. So instead of using HDFS, you 
>> can choose to store your data on CS backed by object store+s3 bridge.
>> 
>> http://www.slideshare.net/sebastiengoasguen/cloudstack-s3
>> http://wiki.apache.org/hadoop/AmazonS3
>> 
>> On the other hand, if you are expecting a hosted Hadoop solution 
>> (like AWS EMR), I dont think that's quite ready yet (or if its even 
>> on the roadmap anytime soon).
>> 
>> --
>> http://shankerbalan.net/
>> 
>> PS: cloudstack-users@ might be a more appropriate list to discuss 
>> this further.
>> 
> 
> 
> 
> --
> 
> N.g.U.y.e.N.A.n.H.t.U


Mime
View raw message