hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: about replication
Date Tue, 20 Aug 2013 07:52:55 GMT
I'm sorry for being unresponsive. Was out of touch for sometime because of
ramzan and eid. Resuming work today.

What's the current status?

Warm Regards,
Tariq
cloudfront.blogspot.com


On Mon, Aug 19, 2013 at 7:18 PM, manish dunani <manishd207@gmail.com> wrote:

> First of all read the concepts ..I hope you will like it..
>
> https://www.frcrc.org/sites/default/files/HadoopTutorialPart1.pdf
>
>
>
> On Mon, Aug 19, 2013 at 9:45 AM, Irfan Sayed <irfu.sayed@gmail.com> wrote:
>
>> please suggest
>>
>> regards
>> irfan
>>
>>
>>
>> On Tue, Aug 13, 2013 at 12:56 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>
>>> hey Tariq,
>>> i am still stuck ..
>>> can you please suggest
>>>
>>> regards
>>> irfan
>>>
>>>
>>>
>>> On Thu, Aug 8, 2013 at 5:56 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>
>>>> please suggest
>>>>
>>>> regards
>>>>
>>>>
>>>>
>>>> On Wed, Aug 7, 2013 at 9:49 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>
>>>>> attachment got quarantined
>>>>> resending in txt format. please rename it to conf.rar
>>>>>
>>>>> regards
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Aug 7, 2013 at 9:41 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>
>>>>>> thanks.
>>>>>>
>>>>>> if i run the jps command on namenode :
>>>>>>
>>>>>> Administrator@DFS-DC /cygdrive/c/Java/jdk1.7.0_25/bin
>>>>>> $ ./jps.exe
>>>>>> 3164 NameNode
>>>>>> 1892 Jps
>>>>>>
>>>>>> same command on datanode :
>>>>>>
>>>>>> Administrator@DFS-1 /cygdrive/c/Java/jdk1.7.0_25/bin
>>>>>> $ ./jps.exe
>>>>>> 3848 Jps
>>>>>>
>>>>>> jps does not list any process for datanode. however, on web browser
i
>>>>>> can see one live data node
>>>>>> please find the attached conf rar file of namenode
>>>>>>
>>>>>> regards
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Aug 7, 2013 at 1:52 AM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>>>>>
>>>>>>> OK. we'll start fresh. Could you plz show me your latest config
>>>>>>> files?
>>>>>>>
>>>>>>> BTW, are your daemons running fine?Use JPS to verify that.
>>>>>>>
>>>>>>> Warm Regards,
>>>>>>> Tariq
>>>>>>> cloudfront.blogspot.com
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Aug 6, 2013 at 10:59 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>>>
>>>>>>>> i have created these dir "wksp_data" and "wksp_name" on both
>>>>>>>> datanode and namenode
>>>>>>>> made the respective changes in "hdfs-site.xml" file
>>>>>>>> formatted the namenode
>>>>>>>> started the dfs
>>>>>>>>
>>>>>>>> but still, not able to browse the file system through web
browser
>>>>>>>> please refer below
>>>>>>>>
>>>>>>>> anything still missing ?
>>>>>>>> please suggest
>>>>>>>>
>>>>>>>> [image: Inline image 1]
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Aug 6, 2013 at 10:35 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>>>>>
>>>>>>>>> these dir needs to be created on all datanodes and namenodes
?
>>>>>>>>> further,  hdfs-site.xml needs to be updated on both datanodes
and
>>>>>>>>> namenodes for these new dir?
>>>>>>>>>
>>>>>>>>> regards
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Aug 6, 2013 at 5:30 PM, Mohammad Tariq <dontariq@gmail.com
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> Create 2 directories manually corresponding to the
values of
>>>>>>>>>> dfs.name.dir and dfs.data.dir properties and change
the permissions of
>>>>>>>>>> these directories to 755. When you start pushing
data into your HDFS, data
>>>>>>>>>> will start going inside the directory specified by
dfs.data.dir and the
>>>>>>>>>> associated metadata will go inside dfs.name.dir.
Remember, you store data
>>>>>>>>>> in HDFS, but it eventually gets stored in your local/native
FS. But you
>>>>>>>>>> cannot see this data directly on your local/native
FS.
>>>>>>>>>>
>>>>>>>>>> Warm Regards,
>>>>>>>>>> Tariq
>>>>>>>>>> cloudfront.blogspot.com
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Aug 6, 2013 at 5:26 PM, Irfan Sayed <irfu.sayed@gmail.com
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> thanks.
>>>>>>>>>>> however, i need this to be working on windows
environment as
>>>>>>>>>>> project requirement.
>>>>>>>>>>> i will add/work on Linux later
>>>>>>>>>>>
>>>>>>>>>>> so, now , at this stage , c:\\wksp is the HDFS
file system OR do
>>>>>>>>>>> i need to create it from command line ?
>>>>>>>>>>>
>>>>>>>>>>> please suggest
>>>>>>>>>>>
>>>>>>>>>>> regards,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Aug 6, 2013 at 5:19 PM, Mohammad Tariq
<
>>>>>>>>>>> dontariq@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello Irfan,
>>>>>>>>>>>>
>>>>>>>>>>>> Sorry for being unresponsive. Got stuck with
some imp work.
>>>>>>>>>>>>
>>>>>>>>>>>> HDFS webUI doesn't provide us the ability
to create file or
>>>>>>>>>>>> directory. You can browse HDFS, view files,
download files etc. But
>>>>>>>>>>>> operation like create, move, copy etc are
not supported.
>>>>>>>>>>>>
>>>>>>>>>>>> These values look fine to me.
>>>>>>>>>>>>
>>>>>>>>>>>> One suggestion though. Try getting a Linux
machine(if
>>>>>>>>>>>> possible). Or at least use a VM. I personally
feel that using Hadoop on
>>>>>>>>>>>> windows is always messy.
>>>>>>>>>>>>
>>>>>>>>>>>> Warm Regards,
>>>>>>>>>>>> Tariq
>>>>>>>>>>>> cloudfront.blogspot.com
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Aug 6, 2013 at 5:09 PM, Irfan Sayed
<
>>>>>>>>>>>> irfu.sayed@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> thanks.
>>>>>>>>>>>>> when i browse the file system , i am
getting following :
>>>>>>>>>>>>> i haven't seen any make directory option
there
>>>>>>>>>>>>>
>>>>>>>>>>>>> i need to create it from command line
?
>>>>>>>>>>>>> further, in the hdfs-site.xml file ,
i have given following
>>>>>>>>>>>>> entries. are they correct ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> <property>
>>>>>>>>>>>>>   <name>dfs.data.dir</name>
>>>>>>>>>>>>>   <value>c:\\wksp</value>
>>>>>>>>>>>>>   </property>
>>>>>>>>>>>>> <property>
>>>>>>>>>>>>>   <name>dfs.name.dir</name>
>>>>>>>>>>>>>   <value>c:\\wksp</value>
>>>>>>>>>>>>>   </property>
>>>>>>>>>>>>>
>>>>>>>>>>>>> please suggest
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> [image: Inline image 1]
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Aug 6, 2013 at 12:40 PM, manish
dunani <
>>>>>>>>>>>>> manishd207@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> *You are wrong at this:*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Administrator@DFS-DC/cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>>>>>>>>>>> $ ./hadoop dfs -copyFromLocal
>>>>>>>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar
/wksp
>>>>>>>>>>>>>> copyFromLocal: File
>>>>>>>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar
does not exist.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Administrator@DFS-DC/cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>>>>>>>>>>>>> $ ./hadoop dfs -copyFromLocal
>>>>>>>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz
/wksp
>>>>>>>>>>>>>> copyFromLocal: File
>>>>>>>>>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz
does not exist.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Because,You had wrote both the paths
local and You need not
>>>>>>>>>>>>>> to copy hadoop into hdfs...Hadoop
is already working..
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Just check out in browser by after
starting ur single node
>>>>>>>>>>>>>> cluster :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> localhost:50070
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> then go for browse the filesystem
link in it..
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If there is no directory then make
directory there.
>>>>>>>>>>>>>> That is your hdfs directory.
>>>>>>>>>>>>>> Then copy any text file there(no
need to copy hadoop
>>>>>>>>>>>>>> there).beacause u are going to do
processing on that data in text
>>>>>>>>>>>>>> file.That's why hadoop is used for
,first u need to make it clear in ur
>>>>>>>>>>>>>> mind.Then and then u will do it...otherwise
not possible..
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Try this: *
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2
>>>>>>>>>>>>>> $ .bin/hadoop dfs -copyFromLocal
/full/local/path/to/ur/file
>>>>>>>>>>>>>> /hdfs/directory/path
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Aug 6, 2013 at 11:49 AM,
Irfan Sayed <
>>>>>>>>>>>>>> irfu.sayed@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> thanks. yes , i am newbie.
>>>>>>>>>>>>>>> however, i need windows setup.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> let me surely refer the doc and
link which u sent but i need
>>>>>>>>>>>>>>> this to be working ...
>>>>>>>>>>>>>>> can you please help
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> regards
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> MANISH DUNANI
>>>>>>>>>>>>>> -THANX
>>>>>>>>>>>>>> +91 9426881954,+91 8460656443
>>>>>>>>>>>>>> manishd207@gmail.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
> --
> Regards
>
> *Manish Dunani*
> *Contact No* : +91 9408329137
> *skype id* : manish.dunani*
> *
>
>
>

Mime
View raw message