hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Irfan Sayed <irfu.sa...@gmail.com>
Subject Re: about replication
Date Wed, 11 Sep 2013 09:26:06 GMT
i do not have any HIVE server host,  then, what should i put over here?? .
if i comment then i guess it throws error of commenting that
can i put the fqdn of namenode for HIVE server host  ?

will it be a really working configuration?

please suggest

regards
irfan



On Tue, Sep 10, 2013 at 5:09 PM, Olivier Renault
<orenault@hortonworks.com>wrote:

> Your cluster-properties.txt should look something like :
>
> #Log directory
> HDP_LOG_DIR=c:\hadoop\logs
>
> #Data directory
> HDP_DATA_DIR=c:\hdp\data
>
> #Hosts
> NAMENODE_HOST=yourmaster.fqdn.com
> JOBTRACKER_HOST=yourmaster.fqdn.com
> HIVE_SERVER_HOST=yourmaster.fqdn.com
> OOZIE_SERVER_HOST=yourmaster.fqdn.com
> TEMPLETON_HOST=yourmaster.fqdn.com
> SLAVE_HOSTS=yourmaster.fqdn.com,youslave.fqdn.com
>
> #Database host
> DB_FLAVOR=derby
> DB_HOSTNAME=yourmaster.fqdn.com
>
>
> #Hive properties
> HIVE_DB_NAME=hive
> HIVE_DB_USERNAME=hive
> HIVE_DB_PASSWORD=hive
>
> #Oozie properties
> OOZIE_DB_NAME=oozie
> OOZIE_DB_USERNAME=oozie
> OOZIE_DB_PASSWORD=oozie
>
> You will need to replace, yourmaster.fqdn.com, yourslave.fqdn.com by your
> servers name. For the time being, I suggest that you do not install HBase,
> Oozie,
>
> regards,
> Olivier
>
>
> On 10 September 2013 07:02, Irfan Sayed <irfu.sayed@gmail.com> wrote:
>
>> please suggest
>>
>> regards
>> irfan
>>
>>
>>
>> On Sat, Sep 7, 2013 at 4:56 PM, Irfan Sayed <irfu.sayed@gmail.com> wrote:
>>
>>> please suggest
>>>
>>> regards
>>> irfan
>>>
>>>
>>>
>>> On Fri, Sep 6, 2013 at 12:16 PM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>
>>>> ok.. now i made some changes and installation went ahead
>>>> but failed in property "HIVE_SERVER_HOST" declaration
>>>> in cluster config file, i have commented this property. if i uncomment
>>>> , then what server address will give ???
>>>>
>>>> i have only two windows machines setup.
>>>> 1: for namenode and another for datanode
>>>>
>>>> please suggest
>>>>
>>>> regards
>>>> irfan
>>>>
>>>>
>>>>
>>>> On Fri, Sep 6, 2013 at 11:42 AM, Irfan Sayed <irfu.sayed@gmail.com>wrote:
>>>>
>>>>> thanks.
>>>>> i installed the latest java in c:\java folder and now no error in log
>>>>> file related to java
>>>>> however, now it is throwing error on not having cluster properties
>>>>> file.
>>>>> in fact i am running/installing hdp from the location where this file
>>>>> exist . still it is throwing error
>>>>>
>>>>> please find the attached
>>>>>
>>>>> [image: Inline image 1]
>>>>>
>>>>> regards
>>>>> irfan
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Sep 6, 2013 at 11:12 AM, Ravi Mummulla (BIG DATA) <
>>>>> ravimu@microsoft.com> wrote:
>>>>>
>>>>>>  Here’s your issue (from the logs you attached earlier):****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> CAQuietExec:  Checking JAVA_HOME is set correctly...****
>>>>>>
>>>>>> CAQuietExec:  Files\Java\jdk1.6.0_31 was unexpected at this time.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> It seems that you installed Java prerequisite in the default path,
>>>>>> which is %PROGRAMFILES% (expands to C:\Program Files in your case). HDP 1.3
>>>>>> does not like spaces in paths, do you need to reinstall Java under c:\java\
>>>>>> or something similar (in a path with no spaces).****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> *From:* Irfan Sayed [mailto:irfu.sayed@gmail.com]
>>>>>> *Sent:* Thursday, September 5, 2013 8:42 PM
>>>>>> *To:* user@hadoop.apache.org
>>>>>> *Subject:* Re: about replication****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> please find the attached.****
>>>>>>
>>>>>> i don't have "c:\HadoopInstallFiles\HadoopSetupTools\hdp-1.3.0.0.winpkg.install.log"
>>>>>> as it is not generated ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Thu, Sep 5, 2013 at 6:09 PM, Olivier Renault <
>>>>>> orenault@hortonworks.com> wrote:****
>>>>>>
>>>>>>  Could you share the log files ( c:\hdp.log,
>>>>>> c:\HadoopInstallFiles\HadoopSetupTools\hdp-1.3.0.0.winpkg.install.log )  as
>>>>>> well as your clusterproperties.txt ?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Thanks, ****
>>>>>>
>>>>>> Olivier****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On 5 September 2013 12:33, Irfan Sayed <irfu.sayed@gmail.com> wrote:*
>>>>>> ***
>>>>>>
>>>>>>  thanks. i followed the user manual for deployment and installed all
>>>>>> pre-requisites ****
>>>>>>
>>>>>> i modified the command and still the issue persist. please suggest **
>>>>>> **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> please refer below ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> [image: Inline image 1]****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Wed, Sep 4, 2013 at 5:13 PM, Olivier Renault <
>>>>>> orenault@hortonworks.com> wrote:****
>>>>>>
>>>>>> The command to install it is msiexec /i msifile /...  ****
>>>>>>
>>>>>> You will find the correct syntax as part of doc. ****
>>>>>>
>>>>>> Happy reading
>>>>>> Olivier ****
>>>>>>
>>>>>> On 4 Sep 2013 12:37, "Irfan Sayed" <irfu.sayed@gmail.com> wrote:****
>>>>>>
>>>>>>  thanks. ****
>>>>>>
>>>>>> i referred the logs and manuals. i modified the clusterproperties
>>>>>> file and then double click on the msi file ****
>>>>>>
>>>>>> however, it still failed.****
>>>>>>
>>>>>> further i started the installation on command line by giving
>>>>>> HDP_LAYOUT=clusterproperties file path, ****
>>>>>>
>>>>>> installation went ahead and it failed for .NET framework 4.0 and VC++
>>>>>> redistributable package dependency   ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> i installed both and started again the installation. ****
>>>>>>
>>>>>> failed again with following error ****
>>>>>>
>>>>>> [image: Inline image 1]****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> when i search for the logs mentioned in the error , i never found
>>>>>> that ****
>>>>>>
>>>>>> please suggest ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Sep 3, 2013 at 12:58 PM, Olivier Renault <
>>>>>> orenault@hortonworks.com> wrote:****
>>>>>>
>>>>>> Correct, you need to define the cluster configuration as part of a
>>>>>> file. You will find some information on the configuration file as part of
>>>>>> the documentation. ****
>>>>>>
>>>>>>
>>>>>> http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.3.0/bk_installing_hdp_for_windows/content/win-getting-ready-6.html
>>>>>> ****
>>>>>>
>>>>>> You should make sure to have also installed the pre requisite. ****
>>>>>>
>>>>>> Thanks
>>>>>> Olivier ****
>>>>>>
>>>>>> On 3 Sep 2013 06:51, "Irfan Sayed" <irfu.sayed@gmail.com> wrote:****
>>>>>>
>>>>>>  thanks. sorry for the long break. actually got involved in some
>>>>>> other priorities****
>>>>>>
>>>>>> i downloaded the installer and while installing i got following error
>>>>>> ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> [image: Inline image 1]****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> do i need to make any configuration prior to installation ??****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Fri, Aug 23, 2013 at 4:10 PM, Olivier Renault <
>>>>>> orenault@hortonworks.com> wrote:****
>>>>>>
>>>>>> Here is the link ****
>>>>>>
>>>>>> http://download.hortonworks.com/products/hdp-windows/****
>>>>>>
>>>>>> Olivier ****
>>>>>>
>>>>>> On 23 Aug 2013 10:55, "Irfan Sayed" <irfu.sayed@gmail.com> wrote:****
>>>>>>
>>>>>>  thanks.****
>>>>>>
>>>>>> i just followed the instructions to setup the pseudo distributed
>>>>>> setup first using the url :
>>>>>> http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html#.UhM8d2T0-4I
>>>>>> ****
>>>>>>
>>>>>>  ****
>>>>>>
>>>>>> i don't think so i am running DN on both machine ****
>>>>>>
>>>>>> please find the attached log****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> hi olivier ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> can you please give me download link ?****
>>>>>>
>>>>>> let me try please ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Fri, Aug 23, 2013 at 1:08 PM, Mohammad Tariq <dontariq@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  Are you running DN on both the machines? Could you please show me
>>>>>> your DN logs?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Also, consider Oliver's suggestion. It's definitely a better option.*
>>>>>> ***
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> Warm Regards,****
>>>>>>
>>>>>> Tariq****
>>>>>>
>>>>>> cloudfront.blogspot.com****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Fri, Aug 23, 2013 at 12:57 PM, Olivier Renault <
>>>>>> orenault@hortonworks.com> wrote:****
>>>>>>
>>>>>> Irfu, ****
>>>>>>
>>>>>> If you want to quickly get Hadoop running on windows platform. You
>>>>>> may want to try our distribution for Windows. You will be able to find the
>>>>>> msi on our website. ****
>>>>>>
>>>>>> Regards
>>>>>> Olivier ****
>>>>>>
>>>>>> On 23 Aug 2013 05:15, "Irfan Sayed" <irfu.sayed@gmail.com> wrote:****
>>>>>>
>>>>>>  thanks. ****
>>>>>>
>>>>>> ok. i think i need to change the plan over here ****
>>>>>>
>>>>>> let me create two environments. 1: totally windows 2: totally Unix***
>>>>>> *
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> because, on windows , anyway i have to try and see how hadoop works *
>>>>>> ***
>>>>>>
>>>>>> on UNIX, it is already known that ,  it is working fine. ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> so, on windows , here is the setup:****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> namenode : windows 2012 R2 ****
>>>>>>
>>>>>> datanode : windows 2012 R2 ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> now, the exact problem is :****
>>>>>>
>>>>>> 1: datanode is not getting started ****
>>>>>>
>>>>>> 2: replication : if i put any file/folder on any datanode , it should
>>>>>> get replicated to all another available datanodes ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Fri, Aug 23, 2013 at 2:42 AM, Mohammad Tariq <dontariq@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  Seriously??You are planning to develop something using Hadoop on
>>>>>> windows. Not a good idea. Anyways, cold you plz show me your log files?I
>>>>>> also need some additional info :****
>>>>>>
>>>>>> -The exact problem which you are facing right now****
>>>>>>
>>>>>> -Your cluster summary(no. of nodes etc)****
>>>>>>
>>>>>> -Your latest configuration files****
>>>>>>
>>>>>> -Your /etc.hosts file****
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> Warm Regards,****
>>>>>>
>>>>>> Tariq****
>>>>>>
>>>>>> cloudfront.blogspot.com****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Fri, Aug 23, 2013 at 1:42 AM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  ok. thanks****
>>>>>>
>>>>>> now, i need to start with all windows setup first as our product will
>>>>>> be based on windows ****
>>>>>>
>>>>>> so, now, please tell me how to resolve the issue ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> datanode is not starting . please suggest ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards,****
>>>>>>
>>>>>> irfan ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Thu, Aug 22, 2013 at 7:56 PM, Mohammad Tariq <dontariq@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  It is possible. Theoretically Hadoop doesn't stop you from doing
>>>>>> that. But it is not a very wise setup.****
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> Warm Regards,****
>>>>>>
>>>>>> Tariq****
>>>>>>
>>>>>> cloudfront.blogspot.com****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Thu, Aug 22, 2013 at 5:01 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  please suggest****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Thu, Aug 22, 2013 at 11:49 AM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  thanks.****
>>>>>>
>>>>>> can i have setup like this :****
>>>>>>
>>>>>> namenode will be on linux (flavour may be RHEL, CentOS, UBuntu etc)**
>>>>>> **
>>>>>>
>>>>>> and datanodes are the combination of any OS (windows , linux , unix
>>>>>> etc )****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> however, my doubt is,  as the file systems of  both the systems (win
>>>>>> and linux ) are different ,  datanodes of these systems can not be part of
>>>>>> single cluster . i have to make windows cluster separate and UNIX cluster
>>>>>> separate ?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Thu, Aug 22, 2013 at 11:26 AM, Arpit Agarwal <
>>>>>> aagarwal@hortonworks.com> wrote:****
>>>>>>
>>>>>> I just noticed you are on Cygwin. IIRC Windows PIDs are not the same
>>>>>> as Cygwin PIDs so that may be causing the discrepancy. I don't know how
>>>>>> well Hadoop works in Cygwin as I have never tried it. Work is in progress
>>>>>> for native Windows support however there are no official releases with
>>>>>> Windows support yet. It may be easier to get familiar with a release<https://www.apache.org/dyn/closer.cgi/hadoop/common/>on Linux if you are new to it.
>>>>>> ****
>>>>>>
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> On Wed, Aug 21, 2013 at 10:05 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  thanks ****
>>>>>>
>>>>>> here is what i did .****
>>>>>>
>>>>>> i stopped all the namenodes and datanodes using ./stop-dfs.sh command
>>>>>> ****
>>>>>>
>>>>>> then deleted all pid files for namenodes and datanodes ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> started dfs again with command : "./start-dfs.sh"****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> when i ran the "Jps" command . it shows****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Administrator@DFS-DC /cygdrive/c/Java/jdk1.7.0_25/bin****
>>>>>>
>>>>>> $ ./jps.exe****
>>>>>>
>>>>>> 4536 Jps****
>>>>>>
>>>>>> 2076 NameNode****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> however, when i open the pid file for namenode then it is not showing
>>>>>> pid as : 4560. on the contrary, it shud show : 2076****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> please suggest ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Thu, Aug 22, 2013 at 9:59 AM, Arpit Agarwal <
>>>>>> aagarwal@hortonworks.com> wrote:****
>>>>>>
>>>>>>  Most likely there is a stale pid file. Something like
>>>>>> \tmp\hadoop-*datanode.pid. You could try deleting it and then restarting
>>>>>> the datanode.
>>>>>>
>>>>>> I haven't read the entire thread so you may have looked at this
>>>>>> already.
>>>>>>
>>>>>> -Arpit****
>>>>>>
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> On Wed, Aug 21, 2013 at 9:22 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  datanode is trying to connect to namenode continuously but fails ***
>>>>>> *
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> when i try to run "jps" command it says :****
>>>>>>
>>>>>> $ ./jps.exe****
>>>>>>
>>>>>> 4584 NameNode****
>>>>>>
>>>>>> 4016 Jps****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> and when i ran the "./start-dfs.sh" then it says :****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> $ ./start-dfs.sh****
>>>>>>
>>>>>> namenode running as process 3544. Stop it first.****
>>>>>>
>>>>>> DFS-1: datanode running as process 4076. Stop it first.****
>>>>>>
>>>>>> localhost: secondarynamenode running as process 4792. Stop it first.*
>>>>>> ***
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> both these logs are contradictory ****
>>>>>>
>>>>>> please find the attached logs ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> should i attach the conf files as well ?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>>  ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Wed, Aug 21, 2013 at 5:28 PM, Mohammad Tariq <dontariq@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  Your DN is still not running. Showing me the logs would be helpful.*
>>>>>> ***
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> Warm Regards,****
>>>>>>
>>>>>> Tariq****
>>>>>>
>>>>>> cloudfront.blogspot.com****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Wed, Aug 21, 2013 at 5:11 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  i followed the url and did the steps mention in that. i have
>>>>>> deployed on the windows platform****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Now, i am able to browse url : http://localhost:50070 (name node )***
>>>>>> *
>>>>>>
>>>>>> however, not able to browse url : http://localhost:50030****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> please refer below****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> [image: Inline image 1]****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> i have modified all the config files as mentioned and formatted the
>>>>>> hdfs file system as well ****
>>>>>>
>>>>>> please suggest ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 20, 2013 at 4:14 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  thanks. i followed this url :
>>>>>> http://blog.sqltrainer.com/2012/01/installing-and-configuring-apache.html
>>>>>> ****
>>>>>>
>>>>>> let me follow the url which you gave for pseudo distributed setup and
>>>>>> then will switch to distributed mode****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 20, 2013 at 3:23 PM, Mohammad Tariq <dontariq@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  You are welcome. Which link have you followed for the
>>>>>> configuration?Your *core-site.xml* is empty. Remove the property *
>>>>>> fs.default.name *from *hdfs-site.xml* and add it to *core-site.xml*.
>>>>>> Remove *mapred.job.tracker* as well. It is required in *
>>>>>> mapred-site.xml*.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> I would suggest you to do a pseudo distributed setup first in order
>>>>>> to get yourself familiar with the process and then proceed to the
>>>>>> distributed mode. You can visit this link<http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html#.UhM8d2T0-4I>if you need some help. Let me know if you face any issue.
>>>>>> ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> HTH****
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> Warm Regards,****
>>>>>>
>>>>>> Tariq****
>>>>>>
>>>>>> cloudfront.blogspot.com****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 20, 2013 at 2:56 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  thanks tariq for response. ****
>>>>>>
>>>>>> as discussed last time, i have sent you all the config files in my
>>>>>> setup . ****
>>>>>>
>>>>>> can you please go through that ?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> please let me know ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 20, 2013 at 1:22 PM, Mohammad Tariq <dontariq@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  I'm sorry for being unresponsive. Was out of touch for sometime
>>>>>> because of ramzan and eid. Resuming work today.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> What's the current status?****
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> Warm Regards,****
>>>>>>
>>>>>> Tariq****
>>>>>>
>>>>>> cloudfront.blogspot.com****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Mon, Aug 19, 2013 at 7:18 PM, manish dunani <manishd207@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  First of all read the concepts ..I hope you will like it..****
>>>>>>
>>>>>>
>>>>>> https://www.frcrc.org/sites/default/files/HadoopTutorialPart1.pdf****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Mon, Aug 19, 2013 at 9:45 AM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  please suggest ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 13, 2013 at 12:56 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  hey Tariq,****
>>>>>>
>>>>>> i am still stuck .. ****
>>>>>>
>>>>>> can you please suggest ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> irfan ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Thu, Aug 8, 2013 at 5:56 AM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  please suggest ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Wed, Aug 7, 2013 at 9:49 AM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  attachment got quarantined ****
>>>>>>
>>>>>> resending in txt format. please rename it to conf.rar ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Wed, Aug 7, 2013 at 9:41 AM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  thanks.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> if i run the jps command on namenode :****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Administrator@DFS-DC /cygdrive/c/Java/jdk1.7.0_25/bin****
>>>>>>
>>>>>> $ ./jps.exe****
>>>>>>
>>>>>> 3164 NameNode****
>>>>>>
>>>>>> 1892 Jps****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> same command on datanode :****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Administrator@DFS-1 /cygdrive/c/Java/jdk1.7.0_25/bin****
>>>>>>
>>>>>> $ ./jps.exe****
>>>>>>
>>>>>> 3848 Jps****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> jps does not list any process for datanode. however, on web browser i
>>>>>> can see one live data node ****
>>>>>>
>>>>>> please find the attached conf rar file of namenode ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Wed, Aug 7, 2013 at 1:52 AM, Mohammad Tariq <dontariq@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  OK. we'll start fresh. Could you plz show me your latest config
>>>>>> files?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> BTW, are your daemons running fine?Use JPS to verify that.****
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> Warm Regards,****
>>>>>>
>>>>>> Tariq****
>>>>>>
>>>>>> cloudfront.blogspot.com****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 10:59 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  i have created these dir "wksp_data" and "wksp_name" on both
>>>>>> datanode and namenode ****
>>>>>>
>>>>>> made the respective changes in "hdfs-site.xml" file ****
>>>>>>
>>>>>> formatted the namenode ****
>>>>>>
>>>>>> started the dfs ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> but still, not able to browse the file system through web browser ***
>>>>>> *
>>>>>>
>>>>>> please refer below ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> anything still missing ?****
>>>>>>
>>>>>> please suggest ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> [image: Inline image 1]****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 10:35 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  these dir needs to be created on all datanodes and namenodes ?****
>>>>>>
>>>>>> further,  hdfs-site.xml needs to be updated on both datanodes and
>>>>>> namenodes for these new dir?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 5:30 PM, Mohammad Tariq <dontariq@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  Create 2 directories manually corresponding to the values of
>>>>>> dfs.name.dir and dfs.data.dir properties and change the permissions of
>>>>>> these directories to 755. When you start pushing data into your HDFS, data
>>>>>> will start going inside the directory specified by dfs.data.dir and the
>>>>>> associated metadata will go inside dfs.name.dir. Remember, you store data
>>>>>> in HDFS, but it eventually gets stored in your local/native FS. But you
>>>>>> cannot see this data directly on your local/native FS.****
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> Warm Regards,****
>>>>>>
>>>>>> Tariq****
>>>>>>
>>>>>> cloudfront.blogspot.com****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 5:26 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  thanks. ****
>>>>>>
>>>>>> however, i need this to be working on windows environment as project
>>>>>> requirement.****
>>>>>>
>>>>>> i will add/work on Linux later ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> so, now , at this stage , c:\\wksp is the HDFS file system OR do i
>>>>>> need to create it from command line ?****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> please suggest****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards,****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 5:19 PM, Mohammad Tariq <dontariq@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  Hello Irfan,****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Sorry for being unresponsive. Got stuck with some imp work.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> HDFS webUI doesn't provide us the ability to create file or
>>>>>> directory. You can browse HDFS, view files, download files etc. But
>>>>>> operation like create, move, copy etc are not supported.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> These values look fine to me.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> One suggestion though. Try getting a Linux machine(if possible). Or
>>>>>> at least use a VM. I personally feel that using Hadoop on windows is always
>>>>>> messy.****
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> Warm Regards,****
>>>>>>
>>>>>> Tariq****
>>>>>>
>>>>>> cloudfront.blogspot.com****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 5:09 PM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  thanks.****
>>>>>>
>>>>>> when i browse the file system , i am getting following :****
>>>>>>
>>>>>> i haven't seen any make directory option there ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> i need to create it from command line ?****
>>>>>>
>>>>>> further, in the hdfs-site.xml file , i have given following entries.
>>>>>> are they correct ? ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> <property>****
>>>>>>
>>>>>>   <name>dfs.data.dir</name>****
>>>>>>
>>>>>>   <value>c:\\wksp</value>****
>>>>>>
>>>>>>   </property>****
>>>>>>
>>>>>> <property>****
>>>>>>
>>>>>>   <name>dfs.name.dir</name>****
>>>>>>
>>>>>>   <value>c:\\wksp</value>****
>>>>>>
>>>>>>   </property>****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> please suggest ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> [image: Inline image 1]****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 12:40 PM, manish dunani <manishd207@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  *You are wrong at this:*****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin****
>>>>>>
>>>>>> $ ./hadoop dfs -copyFromLocal
>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar /wksp****
>>>>>>
>>>>>> copyFromLocal: File
>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar does not exist.
>>>>>> ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin****
>>>>>>
>>>>>> $ ./hadoop dfs -copyFromLocal
>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz /wksp****
>>>>>>
>>>>>> copyFromLocal: File
>>>>>> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz does not exist.
>>>>>> ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Because,You had wrote both the paths local and You need not to copy
>>>>>> hadoop into hdfs...Hadoop is already working..****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Just check out in browser by after starting ur single node cluster :*
>>>>>> ***
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> localhost:50070****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> then go for browse the filesystem link in it..****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> If there is no directory then make directory there.****
>>>>>>
>>>>>> That is your hdfs directory.****
>>>>>>
>>>>>> Then copy any text file there(no need to copy hadoop there).beacause
>>>>>> u are going to do processing on that data in text file.That's why hadoop is
>>>>>> used for ,first u need to make it clear in ur mind.Then and then u will do
>>>>>> it...otherwise not possible..****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> *Try this: *****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2****
>>>>>>
>>>>>> $ .bin/hadoop dfs -copyFromLocal /full/local/path/to/ur/file
>>>>>> /hdfs/directory/path****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> On Tue, Aug 6, 2013 at 11:49 AM, Irfan Sayed <irfu.sayed@gmail.com>
>>>>>> wrote:****
>>>>>>
>>>>>>  thanks. yes , i am newbie.****
>>>>>>
>>>>>> however, i need windows setup.****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> let me surely refer the doc and link which u sent but i need this to
>>>>>> be working ...****
>>>>>>
>>>>>> can you please help****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> regards****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>>  ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> --
>>>>>> MANISH DUNANI
>>>>>> -THANX
>>>>>> +91 9426881954,+91 8460656443****
>>>>>>
>>>>>> manishd207@gmail.com****
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> -- ****
>>>>>>
>>>>>> Regards****
>>>>>>
>>>>>> *Manish Dunani*****
>>>>>>
>>>>>> *Contact No* : +91 9408329137****
>>>>>>
>>>>>> *skype id* : manish.dunani****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>>>> entity to which it is addressed and may contain information that is
>>>>>> confidential, privileged and exempt from disclosure under applicable law.
>>>>>> If the reader of this message is not the intended recipient, you are hereby
>>>>>> notified that any printing, copying, dissemination, distribution,
>>>>>> disclosure or forwarding of this communication is strictly prohibited. If
>>>>>> you have received this communication in error, please contact the sender
>>>>>> immediately and delete it from your system. Thank You.****
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>
>>>>>>
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>>>> entity to which it is addressed and may contain information that is
>>>>>> confidential, privileged and exempt from disclosure under applicable law.
>>>>>> If the reader of this message is not the intended recipient, you are hereby
>>>>>> notified that any printing, copying, dissemination, distribution,
>>>>>> disclosure or forwarding of this communication is strictly prohibited. If
>>>>>> you have received this communication in error, please contact the sender
>>>>>> immediately and delete it from your system. Thank You.****
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>>>> entity to which it is addressed and may contain information that is
>>>>>> confidential, privileged and exempt from disclosure under applicable law.
>>>>>> If the reader of this message is not the intended recipient, you are hereby
>>>>>> notified that any printing, copying, dissemination, distribution,
>>>>>> disclosure or forwarding of this communication is strictly prohibited. If
>>>>>> you have received this communication in error, please contact the sender
>>>>>> immediately and delete it from your system. Thank You.****
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>>>> entity to which it is addressed and may contain information that is
>>>>>> confidential, privileged and exempt from disclosure under applicable law.
>>>>>> If the reader of this message is not the intended recipient, you are hereby
>>>>>> notified that any printing, copying, dissemination, distribution,
>>>>>> disclosure or forwarding of this communication is strictly prohibited. If
>>>>>> you have received this communication in error, please contact the sender
>>>>>> immediately and delete it from your system. Thank You.****
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>>>> entity to which it is addressed and may contain information that is
>>>>>> confidential, privileged and exempt from disclosure under applicable law.
>>>>>> If the reader of this message is not the intended recipient, you are hereby
>>>>>> notified that any printing, copying, dissemination, distribution,
>>>>>> disclosure or forwarding of this communication is strictly prohibited. If
>>>>>> you have received this communication in error, please contact the sender
>>>>>> immediately and delete it from your system. Thank You.****
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>>>> entity to which it is addressed and may contain information that is
>>>>>> confidential, privileged and exempt from disclosure under applicable law.
>>>>>> If the reader of this message is not the intended recipient, you are hereby
>>>>>> notified that any printing, copying, dissemination, distribution,
>>>>>> disclosure or forwarding of this communication is strictly prohibited. If
>>>>>> you have received this communication in error, please contact the sender
>>>>>> immediately and delete it from your system. Thank You.****
>>>>>>
>>>>>>  ** **
>>>>>>
>>>>>>
>>>>>>
>>>>>> ****
>>>>>>
>>>>>> ** **
>>>>>>
>>>>>> -- ****
>>>>>>
>>>>>> Olivier Renault****
>>>>>>
>>>>>> Solution Engineer - Big Data - Hortonworks, Inc.
>>>>>> +44 7500 933 036
>>>>>> orenault@hortonworks.com
>>>>>> www.hortonworks.com****
>>>>>>
>>>>>> **** <http://hortonworks.com/products/hortonworks-sandbox/>
>>>>>>
>>>>>>
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>>>> entity to which it is addressed and may contain information that is
>>>>>> confidential, privileged and exempt from disclosure under applicable law.
>>>>>> If the reader of this message is not the intended recipient, you are hereby
>>>>>> notified that any printing, copying, dissemination, distribution,
>>>>>> disclosure or forwarding of this communication is strictly prohibited. If
>>>>>> you have received this communication in error, please contact the sender
>>>>>> immediately and delete it from your system. Thank You.****<http://hortonworks.com/products/hortonworks-sandbox/>
>>>>>>
>>>>>>  ** ** <http://hortonworks.com/products/hortonworks-sandbox/>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
View raw message