hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Irfan Sayed <irfu.sa...@gmail.com>
Subject Re: about replication
Date Tue, 06 Aug 2013 17:05:17 GMT
these dir needs to be created on all datanodes and namenodes ?
further,  hdfs-site.xml needs to be updated on both datanodes and namenodes
for these new dir?

regards



On Tue, Aug 6, 2013 at 5:30 PM, Mohammad Tariq <dontariq@gmail.com> wrote:

> Create 2 directories manually corresponding to the values of dfs.name.dir
> and dfs.data.dir properties and change the permissions of these directories
> to 755. When you start pushing data into your HDFS, data will start going
> inside the directory specified by dfs.data.dir and the associated metadata
> will go inside dfs.name.dir. Remember, you store data in HDFS, but it
> eventually gets stored in your local/native FS. But you cannot see this
> data directly on your local/native FS.
>
> Warm Regards,
> Tariq
> cloudfront.blogspot.com
>
>
> On Tue, Aug 6, 2013 at 5:26 PM, Irfan Sayed <irfu.sayed@gmail.com> wrote:
>
>> thanks.
>> however, i need this to be working on windows environment as project
>> requirement.
>> i will add/work on Linux later
>>
>> so, now , at this stage , c:\\wksp is the HDFS file system OR do i need
>> to create it from command line ?
>>
>> please suggest
>>
>> regards,
>>
>>
>>
>> On Tue, Aug 6, 2013 at 5:19 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>
>>> Hello Irfan,
>>>
>>> Sorry for being unresponsive. Got stuck with some imp work.
>>>
>>> HDFS webUI doesn't provide us the ability to create file or directory.
>>> You can browse HDFS, view files, download files etc. But operation like
>>> create, move, copy etc are not supported.
>>>
>>> These values look fine to me.
>>>
>>> One suggestion though. Try getting a Linux machine(if possible). Or at
>>> least use a VM. I personally feel that using Hadoop on windows is always
>>> messy.
>>>
>>> Warm Regards,
>>> Tariq
>>> cloudfront.blogspot.com
>>>
>>>
>>> On Tue, Aug 6, 2013 at 5:09 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>
>>>> thanks.
>>>> when i browse the file system , i am getting following :
>>>> i haven't seen any make directory option there
>>>>
>>>> i need to create it from command line ?
>>>> further, in the hdfs-site.xml file , i have given following entries.
>>>> are they correct ?
>>>>
>>>> <property>
>>>>   <name>dfs.data.dir</name>
>>>>   <value>c:\\wksp</value>
>>>>   </property>
>>>> <property>
>>>>   <name>dfs.name.dir</name>
>>>>   <value>c:\\wksp</value>
>>>>   </property>
>>>>
>>>> please suggest
>>>>
>>>>
>>>> [image: Inline image 1]
>>>>
>>>>
>>>> On Tue, Aug 6, 2013 at 12:40 PM, manish dunani <manishd207@gmail.com>wrote:
>>>>
>>>>> *You are wrong at this:*
>>>>>
>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>> $ ./hadoop dfs -copyFromLocal
>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar /wksp
>>>>> copyFromLocal: File
>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar does not exist.
>>>>>
>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>> $ ./hadoop dfs -copyFromLocal
>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz /wksp
>>>>> copyFromLocal: File
>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz does not
exist.
>>>>>
>>>>> Because,You had wrote both the paths local and You need not to copy
>>>>> hadoop into hdfs...Hadoop is already working..
>>>>>
>>>>> Just check out in browser by after starting ur single node cluster :
>>>>>
>>>>> localhost:50070
>>>>>
>>>>> then go for browse the filesystem link in it..
>>>>>
>>>>> If there is no directory then make directory there.
>>>>> That is your hdfs directory.
>>>>> Then copy any text file there(no need to copy hadoop there).beacause
u
>>>>> are going to do processing on that data in text file.That's why hadoop
is
>>>>> used for ,first u need to make it clear in ur mind.Then and then u will
do
>>>>> it...otherwise not possible..
>>>>>
>>>>> *Try this: *
>>>>>
>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2
>>>>> $ .bin/hadoop dfs -copyFromLocal /full/local/path/to/ur/file
>>>>> /hdfs/directory/path
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Aug 6, 2013 at 11:49 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>
>>>>>> thanks. yes , i am newbie.
>>>>>> however, i need windows setup.
>>>>>>
>>>>>> let me surely refer the doc and link which u sent but i need this
to
>>>>>> be working ...
>>>>>> can you please help
>>>>>>
>>>>>> regards
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> MANISH DUNANI
>>>>> -THANX
>>>>> +91 9426881954,+91 8460656443
>>>>> manishd207@gmail.com
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message