hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Irfan Sayed <irfu.sa...@gmail.com>
Subject Re: about replication
Date Wed, 07 Aug 2013 04:11:55 GMT
thanks.

if i run the jps command on namenode :

Administrator@DFS-DC /cygdrive/c/Java/jdk1.7.0_25/bin
$ ./jps.exe
3164 NameNode
1892 Jps

same command on datanode :

Administrator@DFS-1 /cygdrive/c/Java/jdk1.7.0_25/bin
$ ./jps.exe
3848 Jps

jps does not list any process for datanode. however, on web browser i can
see one live data node
please find the attached conf rar file of namenode

regards



On Wed, Aug 7, 2013 at 1:52 AM, Mohammad Tariq <dontariq@gmail.com> wrote:

> OK. we'll start fresh. Could you plz show me your latest config files?
>
> BTW, are your daemons running fine?Use JPS to verify that.
>
> Warm Regards,
> Tariq
> cloudfront.blogspot.com
>
>
> On Tue, Aug 6, 2013 at 10:59 PM, Irfan Sayed <irfu.sayed@gmail.com> wrote:
>
>> i have created these dir "wksp_data" and "wksp_name" on both datanode and
>> namenode
>> made the respective changes in "hdfs-site.xml" file
>> formatted the namenode
>> started the dfs
>>
>> but still, not able to browse the file system through web browser
>> please refer below
>>
>> anything still missing ?
>> please suggest
>>
>> [image: Inline image 1]
>>
>>
>> On Tue, Aug 6, 2013 at 10:35 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>
>>> these dir needs to be created on all datanodes and namenodes ?
>>> further,  hdfs-site.xml needs to be updated on both datanodes and
>>> namenodes for these new dir?
>>>
>>> regards
>>>
>>>
>>>
>>> On Tue, Aug 6, 2013 at 5:30 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>>
>>>> Create 2 directories manually corresponding to the values of
>>>> dfs.name.dir and dfs.data.dir properties and change the permissions of
>>>> these directories to 755. When you start pushing data into your HDFS, data
>>>> will start going inside the directory specified by dfs.data.dir and the
>>>> associated metadata will go inside dfs.name.dir. Remember, you store data
>>>> in HDFS, but it eventually gets stored in your local/native FS. But you
>>>> cannot see this data directly on your local/native FS.
>>>>
>>>> Warm Regards,
>>>> Tariq
>>>> cloudfront.blogspot.com
>>>>
>>>>
>>>> On Tue, Aug 6, 2013 at 5:26 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>
>>>>> thanks.
>>>>> however, i need this to be working on windows environment as project
>>>>> requirement.
>>>>> i will add/work on Linux later
>>>>>
>>>>> so, now , at this stage , c:\\wksp is the HDFS file system OR do i
>>>>> need to create it from command line ?
>>>>>
>>>>> please suggest
>>>>>
>>>>> regards,
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Aug 6, 2013 at 5:19 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>>>>
>>>>>> Hello Irfan,
>>>>>>
>>>>>> Sorry for being unresponsive. Got stuck with some imp work.
>>>>>>
>>>>>> HDFS webUI doesn't provide us the ability to create file or
>>>>>> directory. You can browse HDFS, view files, download files etc. But
>>>>>> operation like create, move, copy etc are not supported.
>>>>>>
>>>>>> These values look fine to me.
>>>>>>
>>>>>> One suggestion though. Try getting a Linux machine(if possible).
Or
>>>>>> at least use a VM. I personally feel that using Hadoop on windows
is always
>>>>>> messy.
>>>>>>
>>>>>> Warm Regards,
>>>>>> Tariq
>>>>>> cloudfront.blogspot.com
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 5:09 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>>
>>>>>>> thanks.
>>>>>>> when i browse the file system , i am getting following :
>>>>>>> i haven't seen any make directory option there
>>>>>>>
>>>>>>> i need to create it from command line ?
>>>>>>> further, in the hdfs-site.xml file , i have given following entries.
>>>>>>> are they correct ?
>>>>>>>
>>>>>>> <property>
>>>>>>>   <name>dfs.data.dir</name>
>>>>>>>   <value>c:\\wksp</value>
>>>>>>>   </property>
>>>>>>> <property>
>>>>>>>   <name>dfs.name.dir</name>
>>>>>>>   <value>c:\\wksp</value>
>>>>>>>   </property>
>>>>>>>
>>>>>>> please suggest
>>>>>>>
>>>>>>>
>>>>>>> [image: Inline image 1]
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Aug 6, 2013 at 12:40 PM, manish dunani <manishd207@gmail.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> *You are wrong at this:*
>>>>>>>>
>>>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>>>>> $ ./hadoop dfs -copyFromLocal
>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar
/wksp
>>>>>>>> copyFromLocal: File
>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar
does not exist.
>>>>>>>>
>>>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>>>>> $ ./hadoop dfs -copyFromLocal
>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz
/wksp
>>>>>>>> copyFromLocal: File
>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz
does not exist.
>>>>>>>>
>>>>>>>> Because,You had wrote both the paths local and You need not
to copy
>>>>>>>> hadoop into hdfs...Hadoop is already working..
>>>>>>>>
>>>>>>>> Just check out in browser by after starting ur single node
cluster :
>>>>>>>>
>>>>>>>> localhost:50070
>>>>>>>>
>>>>>>>> then go for browse the filesystem link in it..
>>>>>>>>
>>>>>>>> If there is no directory then make directory there.
>>>>>>>> That is your hdfs directory.
>>>>>>>> Then copy any text file there(no need to copy hadoop
>>>>>>>> there).beacause u are going to do processing on that data
in text
>>>>>>>> file.That's why hadoop is used for ,first u need to make
it clear in ur
>>>>>>>> mind.Then and then u will do it...otherwise not possible..
>>>>>>>>
>>>>>>>> *Try this: *
>>>>>>>>
>>>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2
>>>>>>>> $ .bin/hadoop dfs -copyFromLocal /full/local/path/to/ur/file
>>>>>>>> /hdfs/directory/path
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Aug 6, 2013 at 11:49 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>>>>
>>>>>>>>> thanks. yes , i am newbie.
>>>>>>>>> however, i need windows setup.
>>>>>>>>>
>>>>>>>>> let me surely refer the doc and link which u sent but
i need this
>>>>>>>>> to be working ...
>>>>>>>>> can you please help
>>>>>>>>>
>>>>>>>>> regards
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> MANISH DUNANI
>>>>>>>> -THANX
>>>>>>>> +91 9426881954,+91 8460656443
>>>>>>>> manishd207@gmail.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message