hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: Hadoop Debian Package
Date Sun, 17 Mar 2013 19:15:45 GMT
set these properties in the configuration files present in your /etc
directory. and HADOOP_HOME is the parent directory of the hadoop bin
directory that holds the Hadoop scripts. so, set that accordingly in
.bashrc file.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Mon, Mar 18, 2013 at 12:35 AM, Mohammad Alkahtani
<m.alkahtani@gmail.com>wrote:

> Thank you Mohammad Tariq
>
> Mohammad Alkahtani
> P.O.Box 102275
> Riyadh 11675
> Saudi Arabia
> mobile: 00966 555 33 1717
>
>
> On Sun, Mar 17, 2013 at 10:04 PM, Mohammad Alkahtani <
> m.alkahtani@gmail.com> wrote:
>
>> I tried all of the hadoop home dirs but didn't worke
>>
>> Mohammad Alkahtani
>> P.O.Box 102275
>> Riyadh 11675
>> Saudi Arabia
>> mobile: 00966 555 33 1717
>>
>>
>> On Sun, Mar 17, 2013 at 9:57 PM, Mohammad Alkahtani <
>> m.alkahtani@gmail.com> wrote:
>>
>>> OK what the Hadoop home should be in ubuntu because the binary files in
>>> /usr/bin
>>> the hadoop-env.sh and othe xml file in /etc/hadoop
>>> the conf files in /usr/share/hadoop/templates/conf
>>>
>>> shall I use /usr as hadoop path because it is the dir that contain the
>>> bin files
>>>
>>> Mohammad Alkahtani
>>> P.O.Box 102275
>>> Riyadh 11675
>>> Saudi Arabia
>>> mobile: 00966 555 33 1717
>>>
>>>
>>> On Sun, Mar 17, 2013 at 9:50 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>>
>>>> log out from the user. log in again and see if it works.
>>>>
>>>> Warm Regards,
>>>> Tariq
>>>> https://mtariq.jux.com/
>>>> cloudfront.blogspot.com
>>>>
>>>>
>>>> On Mon, Mar 18, 2013 at 12:18 AM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>>>
>>>>> you can avoid the warning by setting the following prop to true in the
>>>>> hadoop-env.sh file :
>>>>> export HADOOP_HOME_WARN_SUPPRESS=true
>>>>>
>>>>>
>>>>>
>>>>> Warm Regards,
>>>>> Tariq
>>>>> https://mtariq.jux.com/
>>>>> cloudfront.blogspot.com
>>>>>
>>>>>
>>>>> On Mon, Mar 18, 2013 at 12:07 AM, Mohammad Alkahtani <
>>>>> m.alkahtani@gmail.com> wrote:
>>>>>
>>>>>> Thank you Mohammad
>>>>>> I still get the same error with this msg
>>>>>>
>>>>>> localhost: Warning: $HADOOP_HOME is deprecated.
>>>>>> I searched ~/.bashrc but only what I wrote is there.
>>>>>>
>>>>>>
>>>>>> Mohammad Alkahtani
>>>>>> P.O.Box 102275
>>>>>> Riyadh 11675
>>>>>> Saudi Arabia
>>>>>> mobile: 00966 555 33 1717
>>>>>>
>>>>>>
>>>>>> On Sun, Mar 17, 2013 at 9:21 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>>>>>>
>>>>>>> you can do that using these command :
>>>>>>>
>>>>>>> sudo gedit ~/.bashrc
>>>>>>>
>>>>>>> then go to the end of the file and add this line :
>>>>>>> export HADOOP_HOME=/YOUR_FULL_HADOOP_PATH
>>>>>>>
>>>>>>> after that use it to freeze the changes :
>>>>>>> source ~/.bashrc
>>>>>>>
>>>>>>> to check it :
>>>>>>> echo $HADOOP_HOME
>>>>>>>
>>>>>>> This will permanently set your HADOOP_HOME.
>>>>>>>
>>>>>>> HTH
>>>>>>>
>>>>>>>
>>>>>>> Warm Regards,
>>>>>>> Tariq
>>>>>>> https://mtariq.jux.com/
>>>>>>> cloudfront.blogspot.com
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Mar 17, 2013 at 11:46 PM, Mohammad Alkahtani <
>>>>>>> m.alkahtani@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Tariq, Could you please tell me how to set HADOOP_HOME
because
>>>>>>>> I don't find it in the hadoop-env.sh
>>>>>>>>
>>>>>>>> Thank you Shashwat
>>>>>>>> this is the output and it is already configured but hadoop
don't
>>>>>>>> read the configuration from here.
>>>>>>>>
>>>>>>>> /usr/share/maven-repo/org/apache
>>>>>>>> /commons/commons-parent/22/commons-parent-22-site.xml
>>>>>>>> /usr/share/maven-repo/org/apache/commons/commons-parent/debian
>>>>>>>> /commons-parent-debian-site.xml
>>>>>>>> /usr/share/maven-repo/org/apache/apache/10/apache-10-site.xml
>>>>>>>> /usr/share/maven-repo/org/apache/apache/debian/apache-debian-site.
>>>>>>>> xml
>>>>>>>> /usr/share/compiz/composite.xml
>>>>>>>> /usr/share/hadoop/templates/conf/mapred-site.xml
>>>>>>>> /usr/share/hadoop/templates/conf/core-site.xml
>>>>>>>> /usr/share/hadoop/templates/conf/hdfs-site.xml
>>>>>>>>
>>>>>>>> Mohammad Alkahtani
>>>>>>>> P.O.Box 102275
>>>>>>>> Riyadh 11675
>>>>>>>> Saudi Arabia
>>>>>>>> mobile: 00966 555 33 1717
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Mar 17, 2013 at 9:07 PM, shashwat shriparv <
>>>>>>>> dwivedishashwat@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> try
>>>>>>>>> find / -type f -iname "*site.xml"
>>>>>>>>> it will show you where ever those files are..
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ∞
>>>>>>>>> Shashwat Shriparv
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sun, Mar 17, 2013 at 11:34 PM, Mohammad Alkahtani
<
>>>>>>>>> m.alkahtani@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> The problem is I tried I read the configuration file
by changing
>>>>>>>>>> export HADOOP_CONF_DIR=${HADOOP_CONF_
>>>>>>>>>> DIR:-"/usr/shar/hadoop/templates/conf"}
>>>>>>>>>> but I think Hadoop dosen't get the configration from
this dir, I
>>>>>>>>>> trid and searched the system for conf dir the only
dir is this one which I
>>>>>>>>>> changed.
>>>>>>>>>>
>>>>>>>>>> Mohammad Alkahtani
>>>>>>>>>> P.O.Box 102275
>>>>>>>>>> Riyadh 11675
>>>>>>>>>> Saudi Arabia
>>>>>>>>>> mobile: 00966 555 33 1717
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sun, Mar 17, 2013 at 8:57 PM, shashwat shriparv
<
>>>>>>>>>> dwivedishashwat@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Ye its is asking for file:/// instead of hdfs://
just check if
>>>>>>>>>>> it is taking setting configuration from other
location...
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ∞
>>>>>>>>>>> Shashwat Shriparv
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Mar 17, 2013 at 11:07 PM, Luangsay Sourygna
<
>>>>>>>>>>> luangsay@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> What is the version of Hadoop you use?
>>>>>>>>>>>>
>>>>>>>>>>>> Try using fs.defaultFS instead of fs.default.name
(see the
>>>>>>>>>>>> list of all
>>>>>>>>>>>> the deprecated properties here:
>>>>>>>>>>>>
>>>>>>>>>>>> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
>>>>>>>>>>>> ).
>>>>>>>>>>>> I remember I once had a similar error message
and it was due to
>>>>>>>>>>>> the
>>>>>>>>>>>> change in properties names.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Sourygna
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Mar 17, 2013 at 2:32 PM, Mohammad
Alkahtani
>>>>>>>>>>>> <m.alkahtani@gmail.com> wrote:
>>>>>>>>>>>> > Hi to all users of Hadoop,
>>>>>>>>>>>> >
>>>>>>>>>>>> > I installed Hadoop the .deb file on
Ubuntu 12.04 but I might
>>>>>>>>>>>> could not
>>>>>>>>>>>> > configure it right. The conf dir is
under templates in
>>>>>>>>>>>> /usr/shar/hadoop. I
>>>>>>>>>>>> > edit the core-site.xml, mapred-site.xml
files to give
>>>>>>>>>>>> > <property>
>>>>>>>>>>>> > <name>fs.default.name</name>
>>>>>>>>>>>> > <value>hdfs://localhost:9000</value>
>>>>>>>>>>>> > </property>
>>>>>>>>>>>> > and for mapred
>>>>>>>>>>>> > <property>
>>>>>>>>>>>> > <name>mapred.job.tracker</name>
>>>>>>>>>>>> > <value>localhost:9001</value>
>>>>>>>>>>>> > </property>
>>>>>>>>>>>> >
>>>>>>>>>>>> > but i get these errors, I assume that
there is problem,
>>>>>>>>>>>> Hadoop cannot read
>>>>>>>>>>>> > the configuration file.
>>>>>>>>>>>> > I chaned the hadoop-env.sh to
>>>>>>>>>>>> > export
>>>>>>>>>>>> HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/shar/hadoop/templates/conf"}
>>>>>>>>>>>> > but dosen't solve the problem.
>>>>>>>>>>>> >
>>>>>>>>>>>> > ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
>>>>>>>>>>>> > java.lang.IllegalArgumentException:
Does not contain a valid
>>>>>>>>>>>> host:port
>>>>>>>>>>>> > authority: file:/// at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:225)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:347)
>>>>>>>>>>>> > at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:309)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1651)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1590)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1608)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1734)
>>>>>>>>>>>> > at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1751)
>>>>>>>>>>>> >
>>>>>>>>>>>> > ________________________________
>>>>>>>>>>>> >
>>>>>>>>>>>> > FATAL org.apache.hadoop.mapred.JobTracker:
>>>>>>>>>>>> > java.lang.IllegalArgumentException:
Does not contain a valid
>>>>>>>>>>>> host:port
>>>>>>>>>>>> > authority: local at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2312)
at
>>>>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:2070)
at
>>>>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:1889)
at
>>>>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:1883)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:312)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:303)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:298)
>>>>>>>>>>>> > at
>>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4791)
>>>>>>>>>>>> >
>>>>>>>>>>>> > ________________________________
>>>>>>>>>>>> >
>>>>>>>>>>>> > ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>>>>>>>>>>>> > java.lang.IllegalArgumentException:
Does not contain a valid
>>>>>>>>>>>> host:port
>>>>>>>>>>>> > authority: file:/// at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:265)
>>>>>>>>>>>> > at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:536)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410)
>>>>>>>>>>>> > at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1419)
>>>>>>>>>>>> >
>>>>>>>>>>>> > ________________________________
>>>>>>>>>>>> >
>>>>>>>>>>>> > Exception in thread "main"
>>>>>>>>>>>> java.lang.IllegalArgumentException: Does
not
>>>>>>>>>>>> > contain a valid host:port authority:
file:/// at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:225)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:167)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.(SecondaryNameNode.java:135)
>>>>>>>>>>>> > at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:650)
>>>>>>>>>>>> >
>>>>>>>>>>>> > ________________________________
>>>>>>>>>>>> >
>>>>>>>>>>>> > ERROR org.apache.hadoop.mapred.TaskTracker:
Can not start
>>>>>>>>>>>> task tracker
>>>>>>>>>>>> > because java.lang.IllegalArgumentException:
Does not contain
>>>>>>>>>>>> a valid
>>>>>>>>>>>> > host:port authority: local at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130)
at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2312)
at
>>>>>>>>>>>> > org.apache.hadoop.mapred.TaskTracker.(TaskTracker.java:1532)
>>>>>>>>>>>> at
>>>>>>>>>>>> >
>>>>>>>>>>>> org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3906)
>>>>>>>>>>>> >
>>>>>>>>>>>> >
>>>>>>>>>>>> > Regards,
>>>>>>>>>>>> > Mohammad Alkahtani
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message