hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Irfan Sayed <irfu.sa...@gmail.com>
Subject Re: about replication
Date Tue, 06 Aug 2013 17:29:28 GMT
i have created these dir "wksp_data" and "wksp_name" on both datanode and
namenode
made the respective changes in "hdfs-site.xml" file
formatted the namenode
started the dfs

but still, not able to browse the file system through web browser
please refer below

anything still missing ?
please suggest

[image: Inline image 1]


On Tue, Aug 6, 2013 at 10:35 PM, Irfan Sayed <irfu.sayed@gmail.com> wrote:

> these dir needs to be created on all datanodes and namenodes ?
> further,  hdfs-site.xml needs to be updated on both datanodes and
> namenodes for these new dir?
>
> regards
>
>
>
> On Tue, Aug 6, 2013 at 5:30 PM, Mohammad Tariq <dontariq@gmail.com> wrote:
>
>> Create 2 directories manually corresponding to the values of dfs.name.dir
>> and dfs.data.dir properties and change the permissions of these directories
>> to 755. When you start pushing data into your HDFS, data will start going
>> inside the directory specified by dfs.data.dir and the associated metadata
>> will go inside dfs.name.dir. Remember, you store data in HDFS, but it
>> eventually gets stored in your local/native FS. But you cannot see this
>> data directly on your local/native FS.
>>
>> Warm Regards,
>> Tariq
>> cloudfront.blogspot.com
>>
>>
>> On Tue, Aug 6, 2013 at 5:26 PM, Irfan Sayed <irfu.sayed@gmail.com> wrote:
>>
>>> thanks.
>>> however, i need this to be working on windows environment as project
>>> requirement.
>>> i will add/work on Linux later
>>>
>>> so, now , at this stage , c:\\wksp is the HDFS file system OR do i need
>>> to create it from command line ?
>>>
>>> please suggest
>>>
>>> regards,
>>>
>>>
>>>
>>> On Tue, Aug 6, 2013 at 5:19 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>>
>>>> Hello Irfan,
>>>>
>>>> Sorry for being unresponsive. Got stuck with some imp work.
>>>>
>>>> HDFS webUI doesn't provide us the ability to create file or directory.
>>>> You can browse HDFS, view files, download files etc. But operation like
>>>> create, move, copy etc are not supported.
>>>>
>>>> These values look fine to me.
>>>>
>>>> One suggestion though. Try getting a Linux machine(if possible). Or at
>>>> least use a VM. I personally feel that using Hadoop on windows is always
>>>> messy.
>>>>
>>>> Warm Regards,
>>>> Tariq
>>>> cloudfront.blogspot.com
>>>>
>>>>
>>>> On Tue, Aug 6, 2013 at 5:09 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>
>>>>> thanks.
>>>>> when i browse the file system , i am getting following :
>>>>> i haven't seen any make directory option there
>>>>>
>>>>> i need to create it from command line ?
>>>>> further, in the hdfs-site.xml file , i have given following entries.
>>>>> are they correct ?
>>>>>
>>>>> <property>
>>>>>   <name>dfs.data.dir</name>
>>>>>   <value>c:\\wksp</value>
>>>>>   </property>
>>>>> <property>
>>>>>   <name>dfs.name.dir</name>
>>>>>   <value>c:\\wksp</value>
>>>>>   </property>
>>>>>
>>>>> please suggest
>>>>>
>>>>>
>>>>> [image: Inline image 1]
>>>>>
>>>>>
>>>>> On Tue, Aug 6, 2013 at 12:40 PM, manish dunani <manishd207@gmail.com>wrote:
>>>>>
>>>>>> *You are wrong at this:*
>>>>>>
>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>>> $ ./hadoop dfs -copyFromLocal
>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar /wksp
>>>>>> copyFromLocal: File
>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar does not
exist.
>>>>>>
>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>>> $ ./hadoop dfs -copyFromLocal
>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz /wksp
>>>>>> copyFromLocal: File
>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz does
not exist.
>>>>>>
>>>>>> Because,You had wrote both the paths local and You need not to copy
>>>>>> hadoop into hdfs...Hadoop is already working..
>>>>>>
>>>>>> Just check out in browser by after starting ur single node cluster
:
>>>>>>
>>>>>> localhost:50070
>>>>>>
>>>>>> then go for browse the filesystem link in it..
>>>>>>
>>>>>> If there is no directory then make directory there.
>>>>>> That is your hdfs directory.
>>>>>> Then copy any text file there(no need to copy hadoop there).beacause
>>>>>> u are going to do processing on that data in text file.That's why
hadoop is
>>>>>> used for ,first u need to make it clear in ur mind.Then and then
u will do
>>>>>> it...otherwise not possible..
>>>>>>
>>>>>> *Try this: *
>>>>>>
>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2
>>>>>> $ .bin/hadoop dfs -copyFromLocal /full/local/path/to/ur/file
>>>>>> /hdfs/directory/path
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 11:49 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>>
>>>>>>> thanks. yes , i am newbie.
>>>>>>> however, i need windows setup.
>>>>>>>
>>>>>>> let me surely refer the doc and link which u sent but i need
this to
>>>>>>> be working ...
>>>>>>> can you please help
>>>>>>>
>>>>>>> regards
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> MANISH DUNANI
>>>>>> -THANX
>>>>>> +91 9426881954,+91 8460656443
>>>>>> manishd207@gmail.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message