Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6B794E654 for ; Fri, 4 Jan 2013 18:51:58 +0000 (UTC) Received: (qmail 91842 invoked by uid 500); 4 Jan 2013 18:51:53 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 91724 invoked by uid 500); 4 Jan 2013 18:51:53 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 91717 invoked by uid 99); 4 Jan 2013 18:51:53 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Jan 2013 18:51:53 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of use.hadoop@gmail.com designates 209.85.215.181 as permitted sender) Received: from [209.85.215.181] (HELO mail-ea0-f181.google.com) (209.85.215.181) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Jan 2013 18:51:45 +0000 Received: by mail-ea0-f181.google.com with SMTP id k14so7024090eaa.40 for ; Fri, 04 Jan 2013 10:51:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=kfRM39WkRxVixXQKjLkmlKbh2t7jP3qvQTaRMwe2HgY=; b=b2kADYXWCvWZiFMbxHbAwIbvabt91i6oU9+gOufHxSWkzDzil1JO8h0a/THG5bKlOi kl88wAbgg2ffzldWKePkUMZokcaapRtNbMeT7hS+FNdVnzNcLHmNgUN19ihBxHrVHCXp qB0ivv7v5FYwxLvZyOO5guZSful+q5Y0TSPRjX5k0DGqAe5FGjzoYUTbXa2I3+KKZCC+ x3ghwIR/xHIyHI39gyRRPL/gQb81sbxC10PZI24XuxWKu6ecqVDxNUNmsozHKhtiyCja ZSrdEPNIO5k98T7yCSDpIA7zOTXiFzHJx2wD4gW8V1ymputtk3brh/aN9ZB3Qm3Vsbk+ siKw== MIME-Version: 1.0 Received: by 10.14.215.6 with SMTP id d6mr144983719eep.40.1357325485635; Fri, 04 Jan 2013 10:51:25 -0800 (PST) Received: by 10.223.201.9 with HTTP; Fri, 4 Jan 2013 10:51:25 -0800 (PST) In-Reply-To: References: Date: Fri, 4 Jan 2013 10:51:25 -0800 Message-ID: Subject: Re: Hello and request some advice. From: Gangadhar Ramini To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7b5dbec26a78ff04d27af712 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b5dbec26a78ff04d27af712 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Following is the configuration i put in config. core-site.xml hadoop.tmp.dir /usr/local/hadoop/datastore/hadoop-${user.name} hdfs-site.xml dfs.name.dir C:/cygwin/dfs/logs dfs.data.dir C:/cygwin/dfs/data Thanks -Gangadhar On Fri, Jan 4, 2013 at 10:19 AM, Nitin Pawar wrote= : > what about other directories like hdfs directories and tmp directories ? > > > On Fri, Jan 4, 2013 at 11:34 PM, Gangadhar Ramini w= rote: > >> Yes user owns the directory and had right permissions, still i don't >> understand what could be the issue. >> >> >> ls -ltr ~/hadoop-1.0.4/logs/history >> total 0 >> drwxr-xr-x+ 1 garamini mkgroup 0 Jan 2 22:15 >> >> Thanks >> -Gangadhar >> >> >> >> >> >> >> >> >> On Fri, Jan 4, 2013 at 9:55 AM, Nitin Pawar wro= te: >> >>> Does your user have permissions to read/write on the dfs directories yo= u >>> made? >>> >>> try changing the directory ownerships to the user which is running >>> hadoop. >>> >>> >>> On Fri, Jan 4, 2013 at 11:20 PM, Gangadhar Ramini wrote: >>> >>>> Hi Nitin, >>>> >>>> I tried latest stable Hadoop version on windows with cygwin, I see >>>> following error in JobTracker logs. Do you have any advice? >>>> >>>> C:\cygwin\home\garamini\hadoop-1.0.4\logs\history to 0755^M >>>> at >>>> org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)^M >>>> at >>>> org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:670)^M >>>> at >>>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSyst= em.java:509)^M >>>> at >>>> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java= :344)^M >>>> at >>>> org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189= )^M >>>> at >>>> org.apache.hadoop.mapred.JobHistory.init(JobHistory.java:510)^M >>>> at >>>> org.apache.hadoop.mapred.JobTracker$2.run(JobTracker.java:2334)^M >>>> at >>>> org.apache.hadoop.mapred.JobTracker$2.run(JobTracker.java:2331)^M >>>> at java.security.AccessController.doPrivileged(Native Method)^= M >>>> at javax.security.auth.Subject.doAs(Subject.java:415)^M >>>> at >>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat= ion.java:1121)^M >>>> at >>>> org.apache.hadoop.mapred.JobTracker.(JobTracker.java:2331)^M >>>> at >>>> org.apache.hadoop.mapred.JobTracker.(JobTracker.java:2192)^M >>>> at >>>> org.apache.hadoop.mapred.JobTracker.(JobTracker.java:2186)^M >>>> at >>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:300)^= M >>>> at >>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:291)^= M >>>> at >>>> org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4978)^M >>>> Thanks >>>> -Gangadhar >>>> >>>> >>>> On Fri, Jan 4, 2013 at 9:45 AM, Nitin Pawar w= rote: >>>> >>>>> - Is Ubuntu a good O.S. for running Hadoop? I=92ve tried to learn in >>>>> the past using Red Hat & Infosphere Biginsights, but I need a free O.= S. >>>>> If you want a free O.S , ubuntu is good but if you are familiar with >>>>> RedHat then you may want to have look at Scientific Linux (Its free a= s well) >>>>> >>>>> - Is there a way to learn Hadoop without a static IP address? That wa= s >>>>> really a problem for me when dealing with Infosphere, since I travel >>>>> frequently. >>>>> If you got a laptop with you then thats all you need to learn hadoop. >>>>> Hadoop can be installed in a pseudo distributed mode on a single mach= ine >>>>> and you can use it with localhost all the time. >>>>> >>>>> >>>>> On Fri, Jan 4, 2013 at 10:00 PM, Cristian Carranza < >>>>> cristiancarranza_1@hotmail.com> wrote: >>>>> >>>>>> Hi all in this list! >>>>>> >>>>>> My name is Cristi=E1n Carranza, a statistician and quality consultan= t >>>>>> that for the second time, intends to learn Hadoop and Big Data relat= ed >>>>>> issues. >>>>>> I=92am requesting advice in order to plan my learning. >>>>>> I read the page =93 Products that include Apache Hadoop or derivativ= e >>>>>> works and Commercial Support=94 >>>>>> but after that, I am still with doubts. >>>>>> >>>>>> Please, as a newcomer, I want some advice about the following topics= : >>>>>> >>>>>> - I want to begin by learning how to find specified strings of text >>>>>> in a hiuge data set (csv, txt format, huge means 10 Gb. >>>>>> - Is Ubuntu a good O.S. for running Hadoop? I=92ve tried to learn i= n >>>>>> the past using Red Hat & Infosphere Biginsights, but I need a free O= .S. >>>>>> - Is there a way to learn Hadoop without a static IP address? That >>>>>> was really a problem for me when dealing with Infosphere, since I tr= avel >>>>>> frequently. >>>>>> >>>>>> That=92s all for now. >>>>>> Any help will be greatly appreciated. >>>>>> >>>>>> thanks! >>>>>> >>>>>> Cristi=E1n Carranza >>>>>> Brazil. >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Nitin Pawar >>>>> >>>> >>>> >>> >>> >>> -- >>> Nitin Pawar >>> >> >> > > > -- > Nitin Pawar > --047d7b5dbec26a78ff04d27af712 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Following is the configuration i put in config.
core-s= ite.xml

=A0 <property>
=A0 = =A0 <name>hadoop.tmp.dir</name>
=A0 =A0 <value>= /usr/local/hadoop/datastore/hadoop-${user.name= }</value>
=A0 </property>

hdfs-site.x= ml

=A0<property>
= =A0 =A0 <name>dfs.name.dir</name>
=A0 =A0 <value&g= t;C:/cygwin/dfs/logs</value>
=A0</property>
=A0<property>
=A0 =A0 &= lt;name>dfs.data.dir</name>
=A0 =A0 <value>C:/cygw= in/dfs/data</value>
=A0</property>

Thanks
-Gangadhar


On Fri, Jan 4, 2013 at 10:19 AM, Niti= n Pawar <nitinpawar432@gmail.com> wrote:
what about other directorie= s like hdfs directories and tmp directories ?=A0


On Fri, Jan 4, 20= 13 at 11:34 PM, Gangadhar Ramini <use.hadoop@gmail.com> w= rote:
Yes user owns the= directory and had right permissions, still i don't understand what cou= ld be the issue.


ls -ltr ~/hadoop-1.0.4/logs/history=
total 0
drwxr-xr-x+ 1 garamini mkgroup 0 Jan =A02 22:15

Thanks
=
-Gangadhar








On Fri, J= an 4, 2013 at 9:55 AM, Nitin Pawar <nitinpawar432@gmail.com><= /span> wrote:
Does your user have permissions to read/w= rite on the dfs directories you made?=A0

try changing the directory ownerships to the user which is r= unning hadoop.=A0


On Fri, Jan 4, 2013 at 11:20 PM, Gangadh= ar Ramini <use.hadoop@gmail.com> wrote:
Hi Nitin,

=A0 =A0I tried latest stable = Hadoop version on windows with cygwin, I see following error in JobTracker = logs. Do you have any advice?

=A0C:\cygwin\home\garamini\hadoop-1.0.4\logs\history to 0755^M
= =A0 =A0 =A0 =A0 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.= java:689)^M
=A0 =A0 =A0 =A0 at org.apache.hadoop.fs.FileUtil.setP= ermission(FileUtil.java:670)^M
=A0 =A0 =A0 =A0 at org.apache.hadoop.fs.RawLocalFileSystem.setPermissi= on(RawLocalFileSystem.java:509)^M
=A0 =A0 =A0 =A0 at org.apache.h= adoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)^M
=A0 =A0 =A0 =A0 at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFile= System.java:189)^M
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobHistory.init(JobHistory= .java:510)^M
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobTrack= er$2.run(JobTracker.java:2334)^M
=A0 =A0 =A0 =A0 at org.apache.ha= doop.mapred.JobTracker$2.run(JobTracker.java:2331)^M
=A0 =A0 =A0 =A0 at java.security.AccessController.doPrivileged(Native = Method)^M
=A0 =A0 =A0 =A0 at javax.security.auth.Subject.doAs(Sub= ject.java:415)^M
=A0 =A0 =A0 =A0 at org.apache.hadoop.security.Us= erGroupInformation.doAs(UserGroupInformation.java:1121)^M
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobTracker.<init>(Jo= bTracker.java:2331)^M
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred= .JobTracker.<init>(JobTracker.java:2192)^M
=A0 =A0 =A0 =A0 = at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2186)^M=
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobTracker.startTracker(Jo= bTracker.java:300)^M
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.= JobTracker.startTracker(JobTracker.java:291)^M
=A0 =A0 =A0 =A0 at= org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4978)^M
Thanks
-Gangadhar
=


On Fri, Jan 4, 2013 at 9:45 AM, Nitin Pawar <nitinp= awar432@gmail.com> wrote:
- Is Ubuntu a good O.S. for running Hadoop?=A0 I=92ve trie= d to learn in the past using Red Hat & Infosphere Biginsights, but I ne= ed a free O.S.
If you want a free O.S , ubuntu is good but if you are familia= r with RedHat then you may want to have look at Scientific Linux (Its free = as well)

- Is there a way to learn Hadoop without a s= tatic IP address? That was really a problem for me when dealing with Infosp= here, since I travel frequently.
If= you got a laptop with you then thats all you need to learn hadoop. Hadoop = can be installed in a pseudo distributed mode on a single machine and you c= an use it with localhost all the time.


On Fri, Jan 4, 2013 at 10:00 PM, Cristian Carranza <= ;cristi= ancarranza_1@hotmail.com> wrote:
Hi all in this list!
=A0
My name is Cristi=E1n Carranza, a statistician and qu= ality=20 consultant that for the second time, intends to learn Hadoop and Big Data= =20 related issues.
I=92am requesting advice in order to plan my learning= .=20
I read the page =93 Products that=20 include Apache Hadoop or derivative works and Commercial Support=94<= /div>
but after that, I am still with doubts.
=A0
Please, as a newcomer, I want some advice about the f= ollowing=20 topics:
=A0
- I want to begin by learning how to find specified s= trings of=20 text in a hiuge data set (csv, txt format, huge means 10 Gb.
- Is Ubuntu a good O.S. for running Hadoop?=A0 I=92ve= tried=20 to learn in the past using Red Hat & Infosphere Biginsights, but I need= a=20 free O.S.
- Is there a way to learn Hadoop without a static IP = address?=20 That was really a problem for me when dealing with Infosphere, since I trav= el=20 frequently.
=A0
That=92s all for now.
Any help will be greatly appreciated.
=A0
thanks!
=A0
Cristi=E1n Carranza
Brazil.
=A0



<= font color=3D"#888888">--
Nitin Pawar




<= /div>--
Nitin Pawar




--
Nitin Pawar

--047d7b5dbec26a78ff04d27af712--