hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shi Yu <sh...@uchicago.edu>
Subject Re: Hadoop on Ec2
Date Wed, 07 Sep 2011 16:40:45 GMT
Interested in this topic.  We have experienced plenty of difficulties 
running hadoop in Eucalyptus based virtual instance clusters. Typical 
issues like

java.net.SocketTimeoutException: 69000 millis timeout while waiting for 
channel to be ready for read. ch : java.nio.channels.SocketChannel

kill the whole job. The IO of HDFS based on network storage is very 
slow.  I am wondering whether Apache Whirr has made any significant 
improvement for hadoop implementation in virtual instances like Ec2.


On 9/7/2011 9:58 AM, John Conwell wrote:
> I second that.  Whirr is an invaluable resource for automagically spinning
> up resources on EC2
>
> On Wed, Sep 7, 2011 at 4:28 AM, Harsh J<harsh@cloudera.com>  wrote:
>
>> You are looking for the Apache Whirr project: http://whirr.apache.org/
>>
>> Here's a great article at Phil Whelan's site that covers getting HBase
>> up in a jiffy on ec2:
>> http://www.philwhln.com/run-the-latest-whirr-and-deploy-hbase-in-minutes
>>
>> On Wed, Sep 7, 2011 at 4:48 PM, Shahnawaz Saifi<shahsaifi@gmail.com>
>> wrote:
>>> Hi,
>>>
>>> I was trying to set-up hadoop/hbase cluster on ec2 which took me few
>> hours
>>> to set-up from scratch on bundled image from s3. I am curious to know,
>> what
>>> is the best way to setting hadoop/hbase cluster on amazon ec2? How do we
>> do
>>> it fast?
>>>
>>> Thanks in advance!
>>>
>>> regards,
>>> Shah
>>>
>>
>>
>> --
>> Harsh J
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message