hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastien FLAESCH ...@4js.com>
Subject Re: Compiling Hive ODBC
Date Tue, 30 Apr 2013 14:39:14 GMT
FYI:

hive@orca:~/hive-0.10.0/src$ g++ --version
g++ (Debian 4.4.5-8) 4.4.5

Seb

On 04/30/2013 04:13 PM, Sebastien FLAESCH wrote:
> Making some progress...
>
> After disabling some options to build thrift - because it installs some
> files in the system directory (/usr/lib/php), and I - do not - want that,
> I did following configure for thrift:
>
> ./configure --prefix=/home/hive/thrift-0.9.0 \
> --with-qt4=no \
> --with-csharp=no \
> --with-erlang=no \
> --with-python=no \
> --with-perl=no \
> --with-php=no \
> --with-php_extension=no \
> --with-ruby=no \
> --with-haskell=no \
> --with-go=no \
> --with-d=no
>
> Then I could build thrift and fb303, but not when I try to build Hive ODBC:
>
> hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
> ...
> ...
> ...
> [exec] g++ -Wall -g -fPIC -m32 -DARCH32
> -I/home/hive/thrift-0.9.0/include/thrift
> -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
> -I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
> -o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
> [exec] In file included from
> /usr/local/include/thrift/protocol/TProtocol.h:23,
> [exec] from /usr/local/include/thrift/TProcessor.h:24,
> [exec] from /usr/local/include/thrift/TDispatchProcessor.h:22,
> [exec] from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10,
>
> [exec] from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>
> [exec] /usr/local/include/thrift/transport/TTransport.h:34: error:
> expected constructor, destructor, or type conversion before ‘readAll’
> [exec] /usr/local/include/thrift/transport/TTransport.h:107: error:
> ‘uint32_t’ does not name a type
> [exec] /usr/local/include/thrift/transport/TTransport.h:111: error:
> ‘uint32_t’ does not name a type
> [exec] /usr/local/include/thrift/transport/TTransport.h:124: error:
> ‘uint32_t’ does not name a type
> [exec] /usr/local/include/thrift/transport/TTransport.h:128: error:
> ‘uint32_t’ does not name a type
> [exec] /usr/local/include/thrift/transport/TTransport.h:139: error:
> ‘uint32_t’ does not name a type
>
>
> Any idea?
>
> Thanks
> Seb
>
> On 04/30/2013 11:00 AM, Sebastien FLAESCH wrote:
>> Thank you Carl, but I still have problems to build Hive ODBC:
>>
>> I followed the instruction from this page (I believe the doc is wrong):
>>
>> https://cwiki.apache.org/Hive/hiveodbc.html
>>
>> Where it says:
>>
>> "Build the Hive client by running the following command from
>> HIVE_HOME..."
>>
>> Here is my env:
>>
>> ----------------------------------------------------------
>> hive@orca:~$ cat env-apache.sh
>> JAVA_HOME="/usr/lib/jvm/java-6-sun"
>> export JAVA_HOME
>> JRE_HOME="$JAVA_HOME/jre"
>> export JRE_HOME
>> PATH=$JAVA_HOME/bin:$PATH
>>
>> HADOOP_HOME=/home/hive/hadoop-0.23.7
>> export HADOOP_HOME
>> PATH=$HADOOP_HOME/bin:$PATH
>> export PATH
>>
>> HIVE_HOME=/home/hive/hive-0.10.0
>> export HIVE_HOME
>> PATH=$HIVE_HOME/bin:$PATH
>> export PATH
>>
>> THRIFT_HOME=/home/hive/thrift-0.9.0
>> export THRIFT_HOME
>> ----------------------------------------------------------
>>
>> But:
>>
>> ----------------------------------------------------------
>> hive@orca:~$ cd $HIVE_HOME
>> hive@orca:~/hive-0.10.0$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
>> Buildfile: build.xml does not exist!
>> Build failed
>> ----------------------------------------------------------
>>
>>
>> And when I go to the src directory, it goes further, but then
>> I have missing Thrift header files:
>>
>> ----------------------------------------------------------
>> hive@orca:~/hive-0.10.0$ cd src
>> hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
>> ...
>> compile-cpp:
>> [echo] Project: odbc
>> [exec] mkdir -p /home/hive/hive-0.10.0/src/build/metastore/objs
>> [exec] g++ -Wall -g -fPIC -m32 -DARCH32
>> -I/home/hive/thrift-0.9.0/include/thrift
>> -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
>> -I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
>> -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
>> -I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
>> -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
>>
>> -o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
>> [exec] In file included from
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>>
>>
>> [exec]
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10:39:
>>
>> error: thrift/TDispatchProcessor.h: No such file or directory
>> [exec] In file included from
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:11,
>>
>>
>> [exec] from
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>>
>>
>> [exec]
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:10:27:
>>
>> error: thrift/Thrift.h: No such file or directory
>> [exec]
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:11:42:
>>
>> error: thrift/TApplicationException.h: No such file or directory
>> ...
>> ----------------------------------------------------------
>>
>> In fact there is no such directory:
>>
>> /home/hive/thrift-0.9.0/include/thrift/fb303
>>
>> I see only:
>>
>> ----------------------------------------------------------
>> hive@orca:~/hive-0.10.0/src$ ls /home/hive/thrift-0.9.0/include/thrift
>> c_glib
>> ----------------------------------------------------------
>>
>> According to:
>>
>> http://mail-archives.apache.org/mod_mbox/hive-user/201110.mbox/%3C556325346CA26341B6F0530E07F90D960169F0FD464D@GBGH-EXCH-CMS.sig.ads%3E
>>
>>
>>
>> I need to build fb303 component of thrift...
>>
>> So I tried to build "fb303" in the thrift sources.
>>
>> Note that I ran thrift's configure with a non-system installation target
>> directory:
>>
>> ----------------------------------------------------------
>> hive@orca:~$ cd tmp/thrift-0.9.0/contrib/fb303
>> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ ./configure
>> --prefix=/home/hive/thrift-0.9.0
>> ----------------------------------------------------------
>>
>> But it could not find /usr/local/bin/thrift and
>> /usr/local/include/thrift...
>>
>> So I created links:
>>
>> ----------------------------------------------------------
>> /usr/local/bin/thrift -> /home/hive/thrift-0.9.0/bin/thrift
>> /usr/local/include/thrift -> /home/hive/thrift-0.9.0/include/thrift
>> ----------------------------------------------------------
>>
>> But:
>>
>> ----------------------------------------------------------
>> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ make
>>
>> make all-recursive
>> make[1]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
>> Making all in .
>> make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
>> make[2]: Nothing to be done for `all-am'.
>> make[2]: Leaving directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
>> Making all in cpp
>> make[2]: Entering directory
>> `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
>> make all-am
>> make[3]: Entering directory
>> `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
>> g++ -DPACKAGE_NAME=\"libfb303\" -DPACKAGE_TARNAME=\"libfb303\"
>> -DPACKAGE_VERSION=\"20080209\" -DPACKAGE_STRING=\"libfb303\ 20080209\"
>> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DHAVE_BOOST=/\*\*/
>> -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1
>> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1
>> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1
>> -DHAVE_INTTYPES_H=1 -DHAVE_NETINET_IN_H=1 -I. -I.. -Igen-cpp
>> -I/usr/local/include/thrift -I/usr/include -Wall -O3 -MT
>> FacebookService.o -MD -MP -MF .deps/FacebookService.Tpo -c -o
>> FacebookService.o `test -f 'gen-cpp/FacebookService.cpp' || echo
>> './'`gen-cpp/FacebookService.cpp
>> In file included from gen-cpp/FacebookService.cpp:7:
>> gen-cpp/FacebookService.h:10:39: error: thrift/TDispatchProcessor.h: No
>> such file or directory
>> In file included from gen-cpp/FacebookService.h:11,
>> from gen-cpp/FacebookService.cpp:7:
>>
>> ----------------------------------------------------------
>>
>> ...
>>
>> Seb
>>
>>
>> On 04/29/2013 08:48 PM, Carl Steinbach wrote:
>>> Hi Sebastien,
>>>
>>> Most (if not all) of the ant targets will fail if run from the submodule
>>> directories. You must run the targets from the root source directory
>>> instead.
>>>
>>> Thanks.
>>>
>>> Carl
>>>
>>>
>>> On Fri, Apr 26, 2013 at 6:09 AM, Sebastien FLAESCH <sf@4js.com
>>> <mailto:sf@4js.com>> wrote:
>>>
>>> I have now installed Thrift, but I still get the error with the
>>> "osfamily" task:
>>>
>>> Please help!
>>>
>>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>>> -Dthrift.home=/home/hive/__thrift-0.9.0
>>>
>>>
>>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>>
>>> BUILD FAILED
>>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The following error
>>> occurred while executing this line:
>>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem: failed
>>> to create task or type osfamily
>>> Cause: The name is undefined.
>>> Action: Check the spelling.
>>> Action: Check that any custom tasks/types have been declared.
>>> Action: Check that any <presetdef>/<macrodef> declarations have
>>> taken place.
>>>
>>>
>>> Total time: 0 seconds
>>>
>>> Seb
>>>
>>>
>>> On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
>>>
>>> Go it, need to install Apache Thrift first ... sorry for this mail.
>>> Seb
>>>
>>> On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
>>>
>>> Hi all,
>>>
>>> Just started with an Hadoop / Hive POC to write a
>>>
>>> So far, I have installed:
>>>
>>> So far I have downloaded the Hadoop and Hive packages from:
>>>
>>> http://hadoop.apache.org/__releases.html
>>> <http://hadoop.apache.org/releases.html>
>>> http://hive.apache.org/__releases.html
>>> <http://hive.apache.org/releases.html>
>>>
>>> I took:
>>>
>>> http://hadoop.apache.org/__releases.html#18+April%2C+__2013%3A+Release+0.23.7+__available
>>>
>>>
>>> <http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available>
>>>
>>>
>>>
>>>
>>>
>>> and
>>>
>>> http://hive.apache.org/__releases.html#11+January%2C+__2013%3A+release+0.10.0+__available
>>>
>>>
>>> <http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available>
>>>
>>>
>>>
>>>
>>>
>>>
>>> After setting the env, it seems that hive is working:
>>>
>>> hive@orca:~$ $HIVE_HOME/bin/hive
>>> ...
>>> hive> create table t1 ( k int, s string );
>>> OK
>>>
>>> hive> select count(*) from t1;
>>> Total MapReduce jobs = 1
>>> ...
>>> OK
>>> 0
>>> Time taken: 7.634 seconds
>>>
>>>
>>> No I want to connect through ODBC...
>>>
>>> Following the instructions of this page:
>>>
>>> https://cwiki.apache.org/__confluence/display/Hive/__HiveODBC
>>> <https://cwiki.apache.org/confluence/display/Hive/HiveODBC>
>>>
>>> Section "Hive Client Build/Setup", I tried:
>>>
>>> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
>>> hive@orca:~/hive-0.10.0$ ant compile-cpp
>>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>>> Buildfile: build.xml does not exist!
>>> Build failed
>>>
>>> Then, from the sources directory src/odbc:
>>>
>>> hive@orca:~/hive-0.10.0$ cd src/odbc/
>>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>>
>>> BUILD FAILED
>>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The
>>> following error
>>> occurred while executing this line:
>>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem:
>>> failed to
>>> create task or type osfamily
>>> Cause: The name is undefined.
>>> Action: Check the spelling.
>>> Action: Check that any custom tasks/types have been declared.
>>> Action: Check that any <presetdef>/<macrodef> declarations
>>> have taken
>>> place.
>>>
>>>
>>> Total time: 0 seconds
>>>
>>>
>>> Can someone help or point me to an up to date documentation?
>>>
>>> Thanks!
>>> Seb
>>>
>>>
>>>
>>>
>>
>


Mime
View raw message