incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Jurney <russell.jur...@gmail.com>
Subject Re: Booting the metastore?
Date Wed, 13 Jun 2012 01:24:28 GMT
I've started Hive's metastore before, so I'm not sure why it is crashing -
other than my symlink?

hadoop@ip-10-4-115-51:~$ sudo bash -x
/usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh
+ this=/usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh
+ '[' -h /usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh ']'
++ dirname /usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh
+ bin=/usr/local/hcat/share/hcatalog/scripts
++ basename /usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh
+ script=hcat_server_start.sh
++ unset CDPATH
++ cd /usr/local/hcat/share/hcatalog/scripts
++ pwd
+ bin=/usr/local/hcat/share/hcatalog/scripts
+ this=/usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh
++ dirname /usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh
+ export HCAT_HOME=/usr/local/hcat/share/hcatalog/scripts/..
+ HCAT_HOME=/usr/local/hcat/share/hcatalog/scripts/..
+ . /usr/local/hcat/share/hcatalog/scripts/../conf/hcat-env.sh
++ ROOT=/usr/local/hcat
++ DBROOT=/home/hadoop/mysql-connector-java-3.1.14
++ USER=root
++ HADOOP_HOME=/home/hadoop
++ export METASTORE_PORT=9933
++ METASTORE_PORT=9933
+ PID_FILE=/usr/local/hcat/var/log/hcat.pid
+ SLEEP_TIME_AFTER_START=15
+ '[' -s /usr/local/hcat/var/log/hcat.pid ']'
+ for f in '${DBROOT}/mysql-connector-java-*-bin.jar'
+
AUX_CLASSPATH=:/home/hadoop/mysql-connector-java-3.1.14/mysql-connector-java-3.1.14-bin.jar
+ for f in '${ROOT}/lib/*.jar'
+
AUX_CLASSPATH=':/home/hadoop/mysql-connector-java-3.1.14/mysql-connector-java-3.1.14-bin.jar:/usr/local/hcat/lib/*.jar'
+ export
'AUX_CLASSPATH=:/home/hadoop/mysql-connector-java-3.1.14/mysql-connector-java-3.1.14-bin.jar:/usr/local/hcat/lib/*.jar'
+
AUX_CLASSPATH=':/home/hadoop/mysql-connector-java-3.1.14/mysql-connector-java-3.1.14-bin.jar:/usr/local/hcat/lib/*.jar'
+ export HADOOP_HOME=/home/hadoop
+ HADOOP_HOME=/home/hadoop
++ date +%Y%m%d%H%M
+ export 'HADOOP_OPTS= -server -XX:+UseConcMarkSweepGC
-XX:ErrorFile=/usr/local/hcat/var/log/hcat_err_pid%p.log
-Xloggc:/usr/local/hcat/var/log/hcat_gc.log-201206130121 -verbose:gc
-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps'
+ HADOOP_OPTS=' -server -XX:+UseConcMarkSweepGC
-XX:ErrorFile=/usr/local/hcat/var/log/hcat_err_pid%p.log
-Xloggc:/usr/local/hcat/var/log/hcat_gc.log-201206130121 -verbose:gc
-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps'
+ export HADOOP_HEAPSIZE=2048
+ HADOOP_HEAPSIZE=2048
+ PID=15287
+ '[' 15287x == x ']'
+ echo Started metastore server init, testing if initialized correctly...
Started metastore server init, testing if initialized correctly...
+ sleep 15
+ nohup /usr/local/hcat/bin/hive --service metastore
+ ps -p 15287
+ echo 'Metastore startup failed, see /usr/local/hcat/var/log/hcat.err'
Metastore startup failed, see /usr/local/hcat/var/log/hcat.err
+ exit 1


On Tue, Jun 12, 2012 at 4:33 PM, Aniket Mokashi <aniket486@gmail.com> wrote:

> Can you try sudo bash -x /usr/local/hcat/share/
> hcatalog/scripts/hcat_server_start.sh and paste me stderr/stdout.
>
> Thanks,
> Aniket
>
> On Tue, Jun 12, 2012 at 3:48 PM, Russell Jurney <russell.jurney@gmail.com>wrote:
>
>> Thanks for your help, I've gotten a bit further.
>>
>> I did this:
>>
>> [bash]$ wget
>> http://download.nextag.com/apache/incubator/hcatalog/hcatalog-0.4.0-incubating/hcatalog-src-0.4.0-incubating.tar.gz
>> [bash]$ tar -xvzf hcatalog-src-0.4.0-incubating.tar.gz
>> [bash]$ cd hcatalog-src-0.4.0-incubating
>> [bash]$ mkdir var/log
>> [bash]$ export HCAT_HOME=/home/hadoop/hcatalog-src-0.4.0-incubating
>> [bash]$ ant tar -v #will take a long time, be patient
>> [bash]$ cd build/hcatalog-0.4.0-dev
>> [bash]$ cp ../../NOTICE.txt .
>> [bash]$ sudo bash share/hcatalog/scripts/hcat_server_install.sh -r
>> /usr/local/hcat -d /home/hadoop/mysql-connector-java-3.1.14 -h $HADOOP_HOME
>> -p 9933
>> [bash]$ cd
>> [bash]$ sudo cp hive-0.9.0/conf/hive-site.xml
>>  /usr/local/hcat/etc/hcatalog/
>> [bash]$ sudo ln -s /home/hadoop/hive-0.9.0/bin/hive
>> /usr/local/hcat/bin/hive
>> [bash]$ sudo bash
>> /usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh
>>
>>
>> And I now get this:
>>
>> hadoop@ip-10-4-115-51:~$ sudo bash
>> /usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh
>> Started metastore server init, testing if initialized correctly...
>> Metastore startup failed, see /usr/local/hcat/var/log/hcat.err
>> hadoop@ip-10-4-115-51:~$ cat /usr/local/hcat/var/log/hcat.err
>> <empty>
>>
>>
>> I'm going to circle around with the directions again and see if I missed
>> anything.
>>
>>
>> On Mon, Jun 11, 2012 at 8:28 PM, Aniket Mokashi <aniket486@gmail.com>wrote:
>>
>>> The steps are little unclear on the document
>>>
>>> Here is what I would suggest you to try-
>>>
>>>  1. from share location under build/hcatalog-$version, run
>>> share/hcatalog/scripts/hcat_server_install.sh -r /usr/local/hcat -d
>>> /home/hadoop/mysql-connector-java-3.1.14 -h $HADOOP_HOME -p 9933 (I think
>>> it somehow mandates you to run it with that path).
>>> 2. copy your hive-site.xml under /usr/local/hcat/etc/hcatalog/ (or
>>> wherever the hive-default.xml is).
>>> 3. run /usr/local/hcat/share/hcatalog/scripts/hcat_server_start.sh
>>>
>>> In general, /usr/local/hcat can be any directory. install script is
>>> essentially a set of copy commands (along with generating a config.sh file
>>> from template) and you need to configure installation under
>>> /usr/local/hcat.
>>>
>>> (I may have mispelled shell script names)
>>>
>>> Thanks,
>>> Aniket
>>>
>>> On Mon, Jun 11, 2012 at 7:58 PM, Russell Jurney <
>>> russell.jurney@gmail.com> wrote:
>>>
>>>> I've gotten a little further after reading the installation
>>>> instructions at
>>>> http://incubator.apache.org/hcatalog/docs/r0.4.0/install.html and
>>>> patching https://issues.apache.org/jira/browse/HCATALOG-429
>>>>
>>>> Now I am stuck at:
>>>>
>>>> [bash]$ sudo mkdir /usr/local/hcat
>>>> [bash]$ sudo ./scripts/hcat_server_install.sh -r /usr/local/hcat -d
>>>> /home/hadoop/mysql-connector-java-3.1.14 -h $HADOOP_HOME -p 9933
>>>>
>>>> Installing into [/usr/local/hcat]
>>>> cp: cannot stat `./etc/*': No such file or directory
>>>> cp: cannot stat `./libexec/*': No such file or directory
>>>> cp: cannot stat `./sbin/*': No such file or directory
>>>> cp: cannot stat `./share/*': No such file or directory
>>>> ln: creating symbolic link `/usr/local/hcat/share/hcatalog/conf': No
>>>> such file or directory
>>>> ./scripts/hcat_server_install.sh: line 146:
>>>> /usr/local/hcat/etc/hcatalog/hcat-env.sh: No such file or directory
>>>> Installation successful
>>>>
>>>>
>>>> Will patch as needed.
>>>>
>>>> On Mon, Jun 11, 2012 at 6:31 PM, Russell Jurney <
>>>> russell.jurney@gmail.com> wrote:
>>>>
>>>>> I am running HCatalog for the first time.  My first issue was
>>>>> https://issues.apache.org/jira/browse/HCATALOG-428
>>>>>
>>>>> Once I create var/log, and re-run bin/hcat_server.sh start I get:
>>>>>
>>>>> Exception in thread "main" java.lang.NumberFormatException: For input
>>>>> string: ""
>>>>> at
>>>>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
>>>>>  at java.lang.Integer.parseInt(Integer.java:470)
>>>>> at java.lang.Integer.parseInt(Integer.java:499)
>>>>>  at
>>>>> org.apache.hadoop.hive.metastore.HiveMetaStore$HiveMetastoreCli.parse(HiveMetaStore.java:2902)
>>>>> at
>>>>> org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:2913)
>>>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>  at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>  at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>
>>>>>
>>>>> My relevant ENV follows (had to set HIVE_HOME):
>>>>>
>>>>> TERM=xterm-color
>>>>> SHELL=/bin/bash
>>>>> HADOOP_HOME=/home/hadoop
>>>>> OLDPWD=/home/hadoop
>>>>> SSH_TTY=/dev/pts/0
>>>>> USER=hadoop
>>>>> LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/lib:
>>>>> MAIL=/var/mail/hadoop
>>>>>
>>>>> PATH=/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/hadoop/bin
>>>>> HIVE_HOME=/home/hadoop/hive-0.9.0
>>>>> PWD=/home/hadoop/hcatalog-src-0.4.0-incubating
>>>>> JAVA_HOME=/usr/lib/jvm/java-6-sun
>>>>> LANG=en_US.UTF-8
>>>>> SHLVL=1
>>>>> HOME=/home/hadoop
>>>>> LOGNAME=hadoop
>>>>> HADOOP_HOME_WARN_SUPPRESS=true
>>>>>
>>>>> What do I do? :)
>>>>>
>>>>> --
>>>>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
>>>>> datasyndrome.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
>>>> datasyndrome.com
>>>>
>>>
>>>
>>>
>>> --
>>> "...:::Aniket:::... Quetzalco@tl"
>>>
>>
>>
>>
>> --
>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.
>> com
>>
>
>
>
> --
> "...:::Aniket:::... Quetzalco@tl"
>



-- 
Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com

Mime
View raw message