hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Norbert Burger" <norbert.bur...@gmail.com>
Subject Re: Urgent
Date Thu, 17 Apr 2008 15:52:50 GMT
Yes -- you need to set your DynDNS record to point at the host (IP address)
printed after the text "Master is...".  That's the master node of your EC2
cluster.  The EC2UI Firefox plugin can be useful here to verify
(independently) that EC2 instances have started correctly.

Norbert

On Wed, Apr 16, 2008 at 5:02 PM, Prerna Manaktala <
prerna.manaktala@gmail.com> wrote:

> I ran bin/hadoop-ec2 run
> It gives:
> $ bin/hadoop-ec2 run
> Starting cluster with AMI ami-64f6130d
> Waiting for instance i-37ee2f5e to start
> ......Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException:
> 0
>        at
> com.amazon.aes.webservices.client.cmd.BaseCmd.resolvePath(BaseCmd.jav
> a:323)
>        at
> com.amazon.aes.webservices.client.cmd.BaseCmd.parseOpts(BaseCmd.java:
> 291)
>        at
> com.amazon.aes.webservices.client.cmd.DescribeInstances.<init>(Descri
> beInstances.java:11)
>        at
> com.amazon.aes.webservices.client.cmd.DescribeInstances.main(Describe
> Instances.java:46)
> .Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
>        at
> com.amazon.aes.webservices.client.cmd.BaseCmd.resolvePath(BaseCmd.jav
> a:323)
>        at
> com.amazon.aes.webservices.client.cmd.BaseCmd.parseOpts(BaseCmd.java:
> 291)
>        at
> com.amazon.aes.webservices.client.cmd.DescribeInstances.<init>(Descri
> beInstances.java:11)
>        at
> com.amazon.aes.webservices.client.cmd.DescribeInstances.main(Describe
> Instances.java:46)
> .started as ec2-75-101-218-42.compute-1.amazonaws.com
> Waiting for instance i-36ee2f5f to start
> .started as ec2-75-101-204-106.compute-1.amazonaws.com
> Appointing master
>
> /cygdrive/c/Users/prerna/Desktop/DS/hadoop-0.15.3/hadoop-0.15.3/src/contrib/ec2/
> bin/launch-hadoop-cluster: line 40: dig: command not found
> Master is ec2-75-101-217-228.compute-1.amazonaws.com
> ec2-75-101-218-42.compute-1.amazonaws.com. Please set up DNS so localhost
> points
>  to .
> Then it asks for en enter key.
> The java exception was not coming earlier.
> U mean I should set prerna.dyndns.org to 75.101.217.228?
>
> Thanks
> Prerna
>
> On Wed, Apr 16, 2008 at 8:27 AM, Norbert Burger
> <norbert.burger@gmail.com> wrote:
> > There is no need to maintain a "server" and "client" Cygwin session on a
> >  local machine.  In the typical Hadoop-on-EC2 setup, all of your nodes
> are
> >  EC2 hosts, spawned dynamically by the hadoop-ec2 script.  You only need
> a
> >  single Cygwin session to provide a Unix env for shell script execution.
> >
> >  Have you read the "Setting Up" section on
> >  http://wiki.apache.org/hadoop/AmazonEC2?  In particular:
> >  *
> >  ** Edit user-specific variables in
> ./src/contrib/ec2/bin/hadoop-ec2-env.sh
> >  * Start the cluster by running ./src/contrib/ec2/bin/hadoop-ec2 run
> >  * Provided the cluster nodes start properly, the run script will return
> back
> >  to you the public IP address of your master node.  At this point, you
> need
> >  to remap your DynDNS record (prerna.dyndns.org) to this IP address
> before
> >  continuing.
> >
> >  On Tue, Apr 15, 2008 at 7:33 PM, Prerna Manaktala <
> >
> >
> > prerna.manaktala@gmail.com> wrote:
> >
> >  > hey
> >  > I am working with the EC2 environment.
> >  > I registered and am being billed for EC2 and S3.
> >  > Right now I have two cygwin windows open.
> >  > 1 is as an administrator-server(on which sshd running) in which I
> have
> >  > a separate folder for hadoop files
> >  > and am able to do bin/hadoop
> >  > 1 as  a normal user-client.
> >  > In the client there is no hadoop folder and hence I cant run
> bin/hadoop.
> >  > From here I do not know how to proceed?
> >  > I basically want to implement
> >  >
> http://developer.amazonwebservices.com/connect/entry.jspa?externalID=873.
> >  > Hence I created a host using dyndns.
> >  > If you can help me,it will be great.
> >  >
> >  > On Tue, Apr 15, 2008 at 2:15 PM, Norbert Burger
> >  > <norbert.burger@gmail.com> wrote:
> >  > > Are you trying to run Hadoop on a local cluster, or in the EC2
> >  > environment?
> >  > >  If EC2, then your MASTER_HOST setting is wrong, becuase it points
> to a
> >  > >  residential ISP (*.rr.com).  It should instead point to your
> jobtracker
> >  > node
> >  > >  (the first node started by the EC2 scripts).
> >  > >
> >  > >  The startup scripts for a standard Hadoop cluster and an EC2-based
> >  > Hadoop
> >  > >  cluster are different.  If you're working with a local cluster,
> try
> >  > >  referencing Michael Noll's articles instead:
> >  > >
> >  > >  http://wiki.apache.org/hadoop/HadoopArticles
> >  > >
> >  > >
> >  > >
> >  > >  On Tue, Apr 15, 2008 at 2:00 PM, Prerna Manaktala <
> >  > >  prerna.manaktala@gmail.com> wrote:
> >  > >
> >  > >  > I tried to set up hadoop with cygwin according to the
> >  > >  >  paper:
> >  > >  >
> >  >
> http://developer.amazonwebservices.com/connect/entry.jspa?externalID=873
> >  > >  >  But I had problems working with dyndns.I created a new host
> >  > >  >  there:prerna.dyndns.org
> >  > >  >  and gave the ip address of it in hadoop-ec2-env.sh as a value
> of
> >  > >  > MASTER_HOST.
> >  > >  >  But when I do bin/hadoop-ec2 start-hadoop the error comes:
> >  > >  >  ssh:connect to host prerna.dyndns.org,port 22:connection
> refused
> >  > >  >  ssh failed for root@prerna.dyndns.org
> >  > >  >  Also a warning comes that:id_rsa_gsg-keypair not accesible:No
> such
> >  > >  >  file or directory though there is this file.
> >  > >  >
> >  > >  >  Thanks
> >  > >  >  Prerna
> >  > >  >
> >  > >
> >  >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message