hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: about replication
Date Wed, 21 Aug 2013 11:58:36 GMT
Your DN is still not running. Showing me the logs would be helpful.

Warm Regards,
Tariq
cloudfront.blogspot.com


On Wed, Aug 21, 2013 at 5:11 PM, Irfan Sayed <irfu.sayed@gmail.com> wrote:

> i followed the url and did the steps mention in that. i have deployed on
> the windows platform
>
> Now, i am able to browse url : http://localhost:50070 (name node )
> however, not able to browse url : http://localhost:50030
>
> please refer below
>
> [image: Inline image 1]
>
> i have modified all the config files as mentioned and formatted the hdfs
> file system as well
> please suggest
>
> regards
>
>
>
> On Tue, Aug 20, 2013 at 4:14 PM, Irfan Sayed <irfu.sayed@gmail.com> wrote:
>
>> thanks. i followed this url :
>> http://blog.sqltrainer.com/2012/01/installing-and-configuring-apache.html
>> let me follow the url which you gave for pseudo distributed setup and
>> then will switch to distributed mode
>>
>> regards
>> irfan
>>
>>
>>
>> On Tue, Aug 20, 2013 at 3:23 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>
>>> You are welcome. Which link have you followed for the configuration?Your
>>> *core-site.xml* is empty. Remove the property *fs.default.name *from *
>>> hdfs-site.xml* and add it to *core-site.xml*. Remove *mapred.job.tracker
>>> * as well. It is required in *mapred-site.xml*.
>>>
>>> I would suggest you to do a pseudo distributed setup first in order to
>>> get yourself familiar with the process and then proceed to the distributed
>>> mode. You can visit this link<http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html#.UhM8d2T0-4I>if
you need some help. Let me know if you face any issue.
>>>
>>> HTH
>>>
>>> Warm Regards,
>>> Tariq
>>> cloudfront.blogspot.com
>>>
>>>
>>> On Tue, Aug 20, 2013 at 2:56 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>
>>>> thanks tariq for response.
>>>> as discussed last time, i have sent you all the config files in my
>>>> setup .
>>>> can you please go through that ?
>>>>
>>>> please let me know
>>>>
>>>> regards
>>>> irfan
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Aug 20, 2013 at 1:22 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>>>
>>>>> I'm sorry for being unresponsive. Was out of touch for sometime
>>>>> because of ramzan and eid. Resuming work today.
>>>>>
>>>>> What's the current status?
>>>>>
>>>>> Warm Regards,
>>>>> Tariq
>>>>> cloudfront.blogspot.com
>>>>>
>>>>>
>>>>> On Mon, Aug 19, 2013 at 7:18 PM, manish dunani <manishd207@gmail.com>wrote:
>>>>>
>>>>>> First of all read the concepts ..I hope you will like it..
>>>>>>
>>>>>> https://www.frcrc.org/sites/default/files/HadoopTutorialPart1.pdf
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 19, 2013 at 9:45 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>>
>>>>>>> please suggest
>>>>>>>
>>>>>>> regards
>>>>>>> irfan
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Aug 13, 2013 at 12:56 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>>>
>>>>>>>> hey Tariq,
>>>>>>>> i am still stuck ..
>>>>>>>> can you please suggest
>>>>>>>>
>>>>>>>> regards
>>>>>>>> irfan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Aug 8, 2013 at 5:56 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>>>>
>>>>>>>>> please suggest
>>>>>>>>>
>>>>>>>>> regards
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Aug 7, 2013 at 9:49 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>>>>>
>>>>>>>>>> attachment got quarantined
>>>>>>>>>> resending in txt format. please rename it to conf.rar
>>>>>>>>>>
>>>>>>>>>> regards
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Aug 7, 2013 at 9:41 AM, Irfan Sayed <irfu.sayed@gmail.com
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> thanks.
>>>>>>>>>>>
>>>>>>>>>>> if i run the jps command on namenode :
>>>>>>>>>>>
>>>>>>>>>>> Administrator@DFS-DC /cygdrive/c/Java/jdk1.7.0_25/bin
>>>>>>>>>>> $ ./jps.exe
>>>>>>>>>>> 3164 NameNode
>>>>>>>>>>> 1892 Jps
>>>>>>>>>>>
>>>>>>>>>>> same command on datanode :
>>>>>>>>>>>
>>>>>>>>>>> Administrator@DFS-1 /cygdrive/c/Java/jdk1.7.0_25/bin
>>>>>>>>>>> $ ./jps.exe
>>>>>>>>>>> 3848 Jps
>>>>>>>>>>>
>>>>>>>>>>> jps does not list any process for datanode. however,
on web
>>>>>>>>>>> browser i can see one live data node
>>>>>>>>>>> please find the attached conf rar file of namenode
>>>>>>>>>>>
>>>>>>>>>>> regards
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Aug 7, 2013 at 1:52 AM, Mohammad Tariq
<
>>>>>>>>>>> dontariq@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> OK. we'll start fresh. Could you plz show
me your latest config
>>>>>>>>>>>> files?
>>>>>>>>>>>>
>>>>>>>>>>>> BTW, are your daemons running fine?Use JPS
to verify that.
>>>>>>>>>>>>
>>>>>>>>>>>> Warm Regards,
>>>>>>>>>>>> Tariq
>>>>>>>>>>>> cloudfront.blogspot.com
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Aug 6, 2013 at 10:59 PM, Irfan Sayed
<
>>>>>>>>>>>> irfu.sayed@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> i have created these dir "wksp_data"
and "wksp_name" on both
>>>>>>>>>>>>> datanode and namenode
>>>>>>>>>>>>> made the respective changes in "hdfs-site.xml"
file
>>>>>>>>>>>>> formatted the namenode
>>>>>>>>>>>>> started the dfs
>>>>>>>>>>>>>
>>>>>>>>>>>>> but still, not able to browse the file
system through web
>>>>>>>>>>>>> browser
>>>>>>>>>>>>> please refer below
>>>>>>>>>>>>>
>>>>>>>>>>>>> anything still missing ?
>>>>>>>>>>>>> please suggest
>>>>>>>>>>>>>
>>>>>>>>>>>>> [image: Inline image 1]
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Aug 6, 2013 at 10:35 PM, Irfan
Sayed <
>>>>>>>>>>>>> irfu.sayed@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> these dir needs to be created on
all datanodes and namenodes ?
>>>>>>>>>>>>>> further,  hdfs-site.xml needs to
be updated on both datanodes
>>>>>>>>>>>>>> and namenodes for these new dir?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Aug 6, 2013 at 5:30 PM, Mohammad
Tariq <
>>>>>>>>>>>>>> dontariq@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Create 2 directories manually
corresponding to the values of
>>>>>>>>>>>>>>> dfs.name.dir and dfs.data.dir
properties and change the permissions of
>>>>>>>>>>>>>>> these directories to 755. When
you start pushing data into your HDFS, data
>>>>>>>>>>>>>>> will start going inside the directory
specified by dfs.data.dir and the
>>>>>>>>>>>>>>> associated metadata will go inside
dfs.name.dir. Remember, you store data
>>>>>>>>>>>>>>> in HDFS, but it eventually gets
stored in your local/native FS. But you
>>>>>>>>>>>>>>> cannot see this data directly
on your local/native FS.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Warm Regards,
>>>>>>>>>>>>>>> Tariq
>>>>>>>>>>>>>>> cloudfront.blogspot.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Aug 6, 2013 at 5:26 PM,
Irfan Sayed <
>>>>>>>>>>>>>>> irfu.sayed@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thanks.
>>>>>>>>>>>>>>>> however, i need this to be
working on windows environment
>>>>>>>>>>>>>>>> as project requirement.
>>>>>>>>>>>>>>>> i will add/work on Linux
later
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> so, now , at this stage ,
c:\\wksp is the HDFS file system
>>>>>>>>>>>>>>>> OR do i need to create it
from command line ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> please suggest
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> regards,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Aug 6, 2013 at 5:19
PM, Mohammad Tariq <
>>>>>>>>>>>>>>>> dontariq@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello Irfan,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Sorry for being unresponsive.
Got stuck with some imp work.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> HDFS webUI doesn't provide
us the ability to create file
>>>>>>>>>>>>>>>>> or directory. You can
browse HDFS, view files, download files etc. But
>>>>>>>>>>>>>>>>> operation like create,
move, copy etc are not supported.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> These values look fine
to me.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> One suggestion though.
Try getting a Linux machine(if
>>>>>>>>>>>>>>>>> possible). Or at least
use a VM. I personally feel that using Hadoop on
>>>>>>>>>>>>>>>>> windows is always messy.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Warm Regards,
>>>>>>>>>>>>>>>>> Tariq
>>>>>>>>>>>>>>>>> cloudfront.blogspot.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Aug 6, 2013 at
5:09 PM, Irfan Sayed <
>>>>>>>>>>>>>>>>> irfu.sayed@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> thanks.
>>>>>>>>>>>>>>>>>> when i browse the
file system , i am getting following :
>>>>>>>>>>>>>>>>>> i haven't seen any
make directory option there
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> i need to create
it from command line ?
>>>>>>>>>>>>>>>>>> further, in the hdfs-site.xml
file , i have given
>>>>>>>>>>>>>>>>>> following entries.
are they correct ?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> <property>
>>>>>>>>>>>>>>>>>>   <name>dfs.data.dir</name>
>>>>>>>>>>>>>>>>>>   <value>c:\\wksp</value>
>>>>>>>>>>>>>>>>>>   </property>
>>>>>>>>>>>>>>>>>> <property>
>>>>>>>>>>>>>>>>>>   <name>dfs.name.dir</name>
>>>>>>>>>>>>>>>>>>   <value>c:\\wksp</value>
>>>>>>>>>>>>>>>>>>   </property>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> please suggest
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> [image: Inline image
1]
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Aug 6, 2013
at 12:40 PM, manish dunani <
>>>>>>>>>>>>>>>>>> manishd207@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *You are wrong
at this:*
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Administrator@DFS-DC/cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>>>>>>>>>>>>>>>> $ ./hadoop dfs
-copyFromLocal
>>>>>>>>>>>>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar
/wksp
>>>>>>>>>>>>>>>>>>> copyFromLocal:
File
>>>>>>>>>>>>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar
does not exist.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Administrator@DFS-DC/cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>>>>>>>>>>>>>>>> $ ./hadoop dfs
-copyFromLocal
>>>>>>>>>>>>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz
/wksp
>>>>>>>>>>>>>>>>>>> copyFromLocal:
File
>>>>>>>>>>>>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz
does not exist.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Because,You had
wrote both the paths local and You need
>>>>>>>>>>>>>>>>>>> not to copy hadoop
into hdfs...Hadoop is already working..
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Just check out
in browser by after starting ur single
>>>>>>>>>>>>>>>>>>> node cluster
:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> localhost:50070
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> then go for browse
the filesystem link in it..
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If there is no
directory then make directory there.
>>>>>>>>>>>>>>>>>>> That is your
hdfs directory.
>>>>>>>>>>>>>>>>>>> Then copy any
text file there(no need to copy hadoop
>>>>>>>>>>>>>>>>>>> there).beacause
u are going to do processing on that data in text
>>>>>>>>>>>>>>>>>>> file.That's why
hadoop is used for ,first u need to make it clear in ur
>>>>>>>>>>>>>>>>>>> mind.Then and
then u will do it...otherwise not possible..
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *Try this: *
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Administrator@DFS-DC/cygdrive/c/hadoop-1.1.2/hadoop-1.1.2
>>>>>>>>>>>>>>>>>>> $ .bin/hadoop
dfs -copyFromLocal
>>>>>>>>>>>>>>>>>>> /full/local/path/to/ur/file
/hdfs/directory/path
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Aug 6,
2013 at 11:49 AM, Irfan Sayed <
>>>>>>>>>>>>>>>>>>> irfu.sayed@gmail.com>
wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> thanks. yes
, i am newbie.
>>>>>>>>>>>>>>>>>>>> however,
i need windows setup.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> let me surely
refer the doc and link which u sent but i
>>>>>>>>>>>>>>>>>>>> need this
to be working ...
>>>>>>>>>>>>>>>>>>>> can you please
help
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> MANISH DUNANI
>>>>>>>>>>>>>>>>>>> -THANX
>>>>>>>>>>>>>>>>>>> +91 9426881954,+91
8460656443
>>>>>>>>>>>>>>>>>>> manishd207@gmail.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards
>>>>>>
>>>>>> *Manish Dunani*
>>>>>> *Contact No* : +91 9408329137
>>>>>> *skype id* : manish.dunani*
>>>>>> *
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message