hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: Error while Creating Table in Hive
Date Wed, 06 Jun 2012 18:25:35 GMT
also change the permissions of these directories to 777.

Regards,
    Mohammad Tariq


On Wed, Jun 6, 2012 at 11:54 PM, Mohammad Tariq <dontariq@gmail.com> wrote:
> create a directory "/home/username/hdfs" (or at some place of your
> choice)..inside this hdfs directory create three sub directories -
> name, data, and temp, then follow these steps :
>
> add following properties in your core-site.xml -
>
> <property>
>          <name>fs.default.name</name>
>          <value>hdfs://localhost:9000/</value>
>        </property>
>
>        <property>
>          <name>hadoop.tmp.dir</name>
>          <value>/home/mohammad/hdfs/temp</value>
>        </property>
>
> then add following two properties in your hdfs-site.xml -
>
> <property>
>                <name>dfs.replication</name>
>                <value>1</value>
>        </property>
>
>        <property>
>                <name>dfs.name.dir</name>
>                <value>/home/mohammad/hdfs/name</value>
>        </property>
>
>        <property>
>                <name>dfs.data.dir</name>
>                <value>/home/mohammad/hdfs/data</value>
>        </property>
>
> finally add this property in your mapred-site.xml -
>
>       <property>
>          <name>mapred.job.tracker</name>
>          <value>hdfs://localhost:9001</value>
>        </property>
>
> NOTE: you can give any name to these directories of your choice, just
> keep in mind you have to give same names as values of
>           above specified properties in your configuration files.
> (give full path of these directories, not just the name of the
> directory)
>
> After this  follow the steps provided in the previous reply.
>
> Regards,
>     Mohammad Tariq
>
>
> On Wed, Jun 6, 2012 at 11:42 PM, Babak Bastan <babakbsn@gmail.com> wrote:
>> thank's Mohammad
>>
>> with this command:
>>
>> babak@ubuntu:~/Downloads/hadoop/bin$ hadoop namenode -format
>>
>> this is my output:
>>
>> 12/06/06 20:05:20 INFO namenode.NameNode: STARTUP_MSG:
>> /************************************************************
>> STARTUP_MSG: Starting NameNode
>> STARTUP_MSG:   host = ubuntu/127.0.1.1
>> STARTUP_MSG:   args = [-format]
>> STARTUP_MSG:   version = 0.20.2
>> STARTUP_MSG:   build =
>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r
>> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
>> ************************************************************/
>> 12/06/06 20:05:20 INFO namenode.FSNamesystem:
>> fsOwner=babak,babak,adm,dialout,cdrom,plugdev,lpadmin,admin,sambashare
>> 12/06/06 20:05:20 INFO namenode.FSNamesystem: supergroup=supergroup
>> 12/06/06 20:05:20 INFO namenode.FSNamesystem: isPermissionEnabled=true
>> 12/06/06 20:05:20 INFO common.Storage: Image file of size 95 saved in 0
>> seconds.
>> 12/06/06 20:05:20 INFO common.Storage: Storage directory
>> /tmp/hadoop-babak/dfs/name has been successfully formatted.
>> 12/06/06 20:05:20 INFO namenode.NameNode: SHUTDOWN_MSG:
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
>> ************************************************************/
>>
>> by this command:
>>
>> babak@ubuntu:~/Downloads/hadoop/bin$ start-dfs.sh
>>
>> this is the out put
>>
>> mkdir: kann Verzeichnis „/home/babak/Downloads/hadoop/bin/../logs“ nicht
>> anlegen: Keine Berechtigung
>>
>> this out put(it's in german and it means no right to make this folder)
>>
>>
>> On Wed, Jun 6, 2012 at 7:59 PM, Mohammad Tariq <dontariq@gmail.com> wrote:
>>>
>>> once we are done with the configuration, we need to format the file
>>> system..use this command to do that-
>>> bin/hadoop namenode -format
>>>
>>> after this, hadoop daemon processes should be started using following
>>> commands -
>>> bin/start-dfs.sh (it'll start NN & DN)
>>> bin/start-mapred.sh (it'll start JT & TT)
>>>
>>> after this use jps to check if everything is alright or point your
>>> browser to localhost:50070..if you further find any problem provide us
>>> with the error logs..:)
>>>
>>> Regards,
>>>     Mohammad Tariq
>>>
>>>
>>> On Wed, Jun 6, 2012 at 11:22 PM, Babak Bastan <babakbsn@gmail.com> wrote:
>>> > were you able to format hdfs properly???
>>> > I did'nt get your question,Do you mean HADOOP_HOME? or where did I
>>> > install
>>> > Hadoop?
>>> >
>>> > On Wed, Jun 6, 2012 at 7:49 PM, Mohammad Tariq <dontariq@gmail.com>
>>> > wrote:
>>> >>
>>> >> if you are getting only this, it means your hadoop is not
>>> >> running..were you able to format hdfs properly???
>>> >>
>>> >> Regards,
>>> >>     Mohammad Tariq
>>> >>
>>> >>
>>> >> On Wed, Jun 6, 2012 at 11:17 PM, Babak Bastan <babakbsn@gmail.com>
>>> >> wrote:
>>> >> > Hi MohammadmI irun jps in my shel I can see this result:
>>> >> > 2213 Jps
>>> >> >
>>> >> >
>>> >> > On Wed, Jun 6, 2012 at 7:44 PM, Mohammad Tariq <dontariq@gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> you can also use "jps" command at your shell to see whether
Hadoop
>>> >> >> processes are running or not.
>>> >> >>
>>> >> >> Regards,
>>> >> >>     Mohammad Tariq
>>> >> >>
>>> >> >>
>>> >> >> On Wed, Jun 6, 2012 at 11:12 PM, Mohammad Tariq <dontariq@gmail.com>
>>> >> >> wrote:
>>> >> >> > Hi Babak,
>>> >> >> >
>>> >> >> >  You have to type it in you web browser..Hadoop provides
us a web
>>> >> >> > GUI
>>> >> >> > that not only allows us to browse through the file system,
but to
>>> >> >> > download the files as well..Apart from that it also provides
a web
>>> >> >> > GUI
>>> >> >> > that can be used to see the status of Jobtracker and
>>> >> >> > Tasktracker..When
>>> >> >> > you run a Hive or Pig job or a Mapreduce job, you can
point your
>>> >> >> > browser to http://localhost:50030 to see the status and
logs of
>>> >> >> > your
>>> >> >> > job.
>>> >> >> >
>>> >> >> > Regards,
>>> >> >> >     Mohammad Tariq
>>> >> >> >
>>> >> >> >
>>> >> >> > On Wed, Jun 6, 2012 at 8:28 PM, Babak Bastan <babakbsn@gmail.com>
>>> >> >> > wrote:
>>> >> >> >> Thank you shashwat for the answer,
>>> >> >> >> where should I type http://localhost:50070?
>>> >> >> >> I typed here: hive>http://localhost:50070 but
nothing as result
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> On Wed, Jun 6, 2012 at 3:32 PM, shashwat shriparv
>>> >> >> >> <dwivedishashwat@gmail.com> wrote:
>>> >> >> >>>
>>> >> >> >>> first type http://localhost:50070 whether this
is opening or not
>>> >> >> >>> and
>>> >> >> >>> check
>>> >> >> >>> how many nodes are available, check some of the
hadoop shell
>>> >> >> >>> commands
>>> >> >> >>>
>>> >> >> >>> from http://hadoop.apache.org/common/docs/r0.18.3/hdfs_shell.html
>>> >> >> >>> run
>>> >> >> >>> example mapreduce task on hadoop take example
from here
>>> >> >> >>>
>>> >> >> >>>
>>> >> >> >>>
>>> >> >> >>> : http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/
>>> >> >> >>>
>>> >> >> >>> if all the above you can do sucessfully means
hadoop is
>>> >> >> >>> configured
>>> >> >> >>> correctly
>>> >> >> >>>
>>> >> >> >>> Regards
>>> >> >> >>> Shashwat
>>> >> >> >>>
>>> >> >> >>>
>>> >> >> >>> On Wed, Jun 6, 2012 at 1:30 AM, Babak Bastan
>>> >> >> >>> <babakbsn@gmail.com>
>>> >> >> >>> wrote:
>>> >> >> >>>>
>>> >> >> >>>> no I'm not working on CDH.Is there a way to
test if my Hadoop
>>> >> >> >>>> works
>>> >> >> >>>> fine
>>> >> >> >>>> or not?
>>> >> >> >>>>
>>> >> >> >>>>
>>> >> >> >>>> On Tue, Jun 5, 2012 at 9:55 PM, Bejoy KS <bejoy_ks@yahoo.com>
>>> >> >> >>>> wrote:
>>> >> >> >>>>>
>>> >> >> >>>>> Hi Babak
>>> >> >> >>>>>
>>> >> >> >>>>> You gotta follow those instructions in
the apace site to set
>>> >> >> >>>>> up
>>> >> >> >>>>> hadoop
>>> >> >> >>>>> from scratch and ensure that hdfs is working
first. You should
>>> >> >> >>>>> be
>>> >> >> >>>>> able to
>>> >> >> >>>>> read and write files to hdfs before you
do your next steps.
>>> >> >> >>>>>
>>> >> >> >>>>> Are you on CDH or apache distribution
of hadoop? If it is CDH
>>> >> >> >>>>> there
>>> >> >> >>>>> are
>>> >> >> >>>>> detailed instructions on Cloudera web
site.
>>> >> >> >>>>>
>>> >> >> >>>>> Regards
>>> >> >> >>>>> Bejoy KS
>>> >> >> >>>>>
>>> >> >> >>>>> Sent from handheld, please excuse typos.
>>> >> >> >>>>> ________________________________
>>> >> >> >>>>> From: Babak Bastan <babakbsn@gmail.com>
>>> >> >> >>>>> Date: Tue, 5 Jun 2012 21:30:22 +0200
>>> >> >> >>>>> To: <user@hive.apache.org>
>>> >> >> >>>>> ReplyTo: user@hive.apache.org
>>> >> >> >>>>> Subject: Re: Error while Creating Table
in Hive
>>> >> >> >>>>>
>>> >> >> >>>>> @Bejoy: I set the fs.default.name in
the core-site.xml and I
>>> >> >> >>>>> did
>>> >> >> >>>>> all
>>> >> >> >>>>> of
>>> >> >> >>>>> thing that was mentioned in the reference
but no effect
>>> >> >> >>>>>
>>> >> >> >>>>> On Tue, Jun 5, 2012 at 8:43 PM, Babak
Bastan
>>> >> >> >>>>> <babakbsn@gmail.com>
>>> >> >> >>>>> wrote:
>>> >> >> >>>>>>
>>> >> >> >>>>>> Ok sorry but that was my Mistake .I
thought it works but no.
>>> >> >> >>>>>> I wrote the command without ; and
then I think It works but
>>> >> >> >>>>>> with
>>> >> >> >>>>>> ;
>>> >> >> >>>>>> at
>>> >> >> >>>>>> the end of command
>>> >> >> >>>>>>
>>> >> >> >>>>>> CREATE TABLE pokes (foo INT, bar STRING);
>>> >> >> >>>>>>
>>> >> >> >>>>>> does'nt work
>>> >> >> >>>>>>
>>> >> >> >>>>>>
>>> >> >> >>>>>> On Tue, Jun 5, 2012 at 8:34 PM, shashwat
shriparv
>>> >> >> >>>>>> <dwivedishashwat@gmail.com>
wrote:
>>> >> >> >>>>>>>
>>> >> >> >>>>>>> inside configuration. all properties
will be inside the
>>> >> >> >>>>>>> configuration
>>> >> >> >>>>>>> tags
>>> >> >> >>>>>>>
>>> >> >> >>>>>>>
>>> >> >> >>>>>>> On Tue, Jun 5, 2012 at 11:53 PM,
Babak Bastan
>>> >> >> >>>>>>> <babakbsn@gmail.com>
>>> >> >> >>>>>>> wrote:
>>> >> >> >>>>>>>>
>>> >> >> >>>>>>>> Thank you so much my friend
your idee works fine(no error)
>>> >> >> >>>>>>>> you
>>> >> >> >>>>>>>> are
>>> >> >> >>>>>>>> the best :)
>>> >> >> >>>>>>>>
>>> >> >> >>>>>>>>
>>> >> >> >>>>>>>> On Tue, Jun 5, 2012 at 8:20
PM, Babak Bastan
>>> >> >> >>>>>>>> <babakbsn@gmail.com>
>>> >> >> >>>>>>>> wrote:
>>> >> >> >>>>>>>>>
>>> >> >> >>>>>>>>> It must be inside the
<configuration></configuration> or
>>> >> >> >>>>>>>>> outside
>>> >> >> >>>>>>>>> this?
>>> >> >> >>>>>>>>>
>>> >> >> >>>>>>>>>
>>> >> >> >>>>>>>>> On Tue, Jun 5, 2012 at
8:15 PM, shashwat shriparv
>>> >> >> >>>>>>>>> <dwivedishashwat@gmail.com>
wrote:
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>> It will be inside
hive/conf
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>> On Tue, Jun 5, 2012
at 11:43 PM, Babak Bastan
>>> >> >> >>>>>>>>>> <babakbsn@gmail.com>
>>> >> >> >>>>>>>>>> wrote:
>>> >> >> >>>>>>>>>>>
>>> >> >> >>>>>>>>>>> Thanks sShashwat,
and where is this hive-site.xml
>>> >> >> >>>>>>>>>>>
>>> >> >> >>>>>>>>>>> On Tue, Jun 5,
2012 at 8:02 PM, shashwat shriparv
>>> >> >> >>>>>>>>>>> <dwivedishashwat@gmail.com>
wrote:
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> set
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> hive.metastore.warehouse.dir
in hive-site.xml
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> <property>
>>> >> >> >>>>>>>>>>>>   <name>hive.metastore.local</name>
>>> >> >> >>>>>>>>>>>>   <value>true</value>
>>> >> >> >>>>>>>>>>>> </property>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> <name>hive.metastore.warehouse.dir</name>
>>> >> >> >>>>>>>>>>>>        
       <value>/home/<your
>>> >> >> >>>>>>>>>>>> username>/hivefolder</value>
>>> >> >> >>>>>>>>>>>>        
       <description>location of default
>>> >> >> >>>>>>>>>>>> database
>>> >> >> >>>>>>>>>>>> for
>>> >> >> >>>>>>>>>>>> the
>>> >> >> >>>>>>>>>>>> warehouse</description>
>>> >> >> >>>>>>>>>>>>        </property>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> On Tue, Jun
5, 2012 at 10:43 PM, Babak Bastan
>>> >> >> >>>>>>>>>>>> <babakbsn@gmail.com>
wrote:
>>> >> >> >>>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>> Hello
Experts ,
>>> >> >> >>>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>> I'm new
in Hive .When try to create a test Table in
>>> >> >> >>>>>>>>>>>>> Hive
>>> >> >> >>>>>>>>>>>>> I
>>> >> >> >>>>>>>>>>>>> get
>>> >> >> >>>>>>>>>>>>> an error.I
want to run this command:
>>> >> >> >>>>>>>>>>>>> CREATE
TABLE Test (DateT STRING, Url STRING, Content
>>> >> >> >>>>>>>>>>>>> STRING);
>>> >> >> >>>>>>>>>>>>> but this
error occured:
>>> >> >> >>>>>>>>>>>>> FAILED:
Error in metadata: MetaException(message:Got
>>> >> >> >>>>>>>>>>>>> exception:
>>> >> >> >>>>>>>>>>>>> java.io.FileNotFoundException
File
>>> >> >> >>>>>>>>>>>>> file:/user/hive/warehouse/test
does not
>>> >> >> >>>>>>>>>>>>> exist.)
>>> >> >> >>>>>>>>>>>>> FAILED:
Execution Error, return code 1 from
>>> >> >> >>>>>>>>>>>>> org.apache.hadoop.hive.ql.exec.DDLTask
>>> >> >> >>>>>>>>>>>>> How can
I solve this Problem?
>>> >> >> >>>>>>>>>>>>> Thank
you so much
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> --
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> ∞
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>> Shashwat Shriparv
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>>
>>> >> >> >>>>>>>>>>>
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>> --
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>> ∞
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>> Shashwat Shriparv
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>>
>>> >> >> >>>>>>>>>
>>> >> >> >>>>>>>>
>>> >> >> >>>>>>>
>>> >> >> >>>>>>>
>>> >> >> >>>>>>>
>>> >> >> >>>>>>> --
>>> >> >> >>>>>>>
>>> >> >> >>>>>>>
>>> >> >> >>>>>>> ∞
>>> >> >> >>>>>>>
>>> >> >> >>>>>>> Shashwat Shriparv
>>> >> >> >>>>>>>
>>> >> >> >>>>>>>
>>> >> >> >>>>>>
>>> >> >> >>>>>
>>> >> >> >>>>
>>> >> >> >>>
>>> >> >> >>>
>>> >> >> >>>
>>> >> >> >>> --
>>> >> >> >>>
>>> >> >> >>>
>>> >> >> >>> ∞
>>> >> >> >>>
>>> >> >> >>> Shashwat Shriparv
>>> >> >> >>>
>>> >> >> >>>
>>> >> >> >>
>>> >> >
>>> >> >
>>> >
>>> >
>>
>>

Mime
View raw message