hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastien FLAESCH ...@4js.com>
Subject Re: Compiling Hive ODBC
Date Tue, 30 Apr 2013 14:13:12 GMT
Making some progress...

After disabling some options to build thrift - because it installs some
files in the system directory (/usr/lib/php), and I - do not - want that,
I did following configure for thrift:

./configure --prefix=/home/hive/thrift-0.9.0 \
   --with-qt4=no \
   --with-csharp=no \
   --with-erlang=no \
   --with-python=no \
   --with-perl=no \
   --with-php=no \
   --with-php_extension=no \
   --with-ruby=no \
   --with-haskell=no \
   --with-go=no \
   --with-d=no

Then I could build thrift and fb303, but not when I try to build Hive ODBC:

hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
      ...
      ...
      ...
      [exec] g++ -Wall -g -fPIC -m32 -DARCH32 -I/home/hive/thrift-0.9.0/include/thrift -I/home/hive/thrift-0.9.0/include/thrift/fb303
-I/include 
-I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp

-I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp -I/home/hive/hive-0.10.0/src/odbc/src/cpp
-c 
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp -o 
/home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
      [exec] In file included from /usr/local/include/thrift/protocol/TProtocol.h:23,
      [exec]                  from /usr/local/include/thrift/TProcessor.h:24,
      [exec]                  from /usr/local/include/thrift/TDispatchProcessor.h:22,
      [exec]                  from /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10,
      [exec]                  from /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
      [exec] /usr/local/include/thrift/transport/TTransport.h:34: error: expected constructor,
destructor, or type conversion before ‘readAll’
      [exec] /usr/local/include/thrift/transport/TTransport.h:107: error: ‘uint32_t’ does
not name a type
      [exec] /usr/local/include/thrift/transport/TTransport.h:111: error: ‘uint32_t’ does
not name a type
      [exec] /usr/local/include/thrift/transport/TTransport.h:124: error: ‘uint32_t’ does
not name a type
      [exec] /usr/local/include/thrift/transport/TTransport.h:128: error: ‘uint32_t’ does
not name a type
      [exec] /usr/local/include/thrift/transport/TTransport.h:139: error: ‘uint32_t’ does
not name a type


Any idea?

Thanks
Seb

On 04/30/2013 11:00 AM, Sebastien FLAESCH wrote:
> Thank you Carl, but I still have problems to build Hive ODBC:
>
> I followed the instruction from this page (I believe the doc is wrong):
>
> https://cwiki.apache.org/Hive/hiveodbc.html
>
> Where it says:
>
> "Build the Hive client by running the following command from HIVE_HOME..."
>
> Here is my env:
>
> ----------------------------------------------------------
> hive@orca:~$ cat env-apache.sh
> JAVA_HOME="/usr/lib/jvm/java-6-sun"
> export JAVA_HOME
> JRE_HOME="$JAVA_HOME/jre"
> export JRE_HOME
> PATH=$JAVA_HOME/bin:$PATH
>
> HADOOP_HOME=/home/hive/hadoop-0.23.7
> export HADOOP_HOME
> PATH=$HADOOP_HOME/bin:$PATH
> export PATH
>
> HIVE_HOME=/home/hive/hive-0.10.0
> export HIVE_HOME
> PATH=$HIVE_HOME/bin:$PATH
> export PATH
>
> THRIFT_HOME=/home/hive/thrift-0.9.0
> export THRIFT_HOME
> ----------------------------------------------------------
>
> But:
>
> ----------------------------------------------------------
> hive@orca:~$ cd $HIVE_HOME
> hive@orca:~/hive-0.10.0$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
> Buildfile: build.xml does not exist!
> Build failed
> ----------------------------------------------------------
>
>
> And when I go to the src directory, it goes further, but then
> I have missing Thrift header files:
>
> ----------------------------------------------------------
> hive@orca:~/hive-0.10.0$ cd src
> hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
> ...
> compile-cpp:
> [echo] Project: odbc
> [exec] mkdir -p /home/hive/hive-0.10.0/src/build/metastore/objs
> [exec] g++ -Wall -g -fPIC -m32 -DARCH32
> -I/home/hive/thrift-0.9.0/include/thrift
> -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
> -I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
> -o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
> [exec] In file included from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>
> [exec]
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10:39:
> error: thrift/TDispatchProcessor.h: No such file or directory
> [exec] In file included from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:11,
>
> [exec] from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>
> [exec]
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:10:27:
> error: thrift/Thrift.h: No such file or directory
> [exec]
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:11:42:
> error: thrift/TApplicationException.h: No such file or directory
> ...
> ----------------------------------------------------------
>
> In fact there is no such directory:
>
> /home/hive/thrift-0.9.0/include/thrift/fb303
>
> I see only:
>
> ----------------------------------------------------------
> hive@orca:~/hive-0.10.0/src$ ls /home/hive/thrift-0.9.0/include/thrift
> c_glib
> ----------------------------------------------------------
>
> According to:
>
> http://mail-archives.apache.org/mod_mbox/hive-user/201110.mbox/%3C556325346CA26341B6F0530E07F90D960169F0FD464D@GBGH-EXCH-CMS.sig.ads%3E
>
>
> I need to build fb303 component of thrift...
>
> So I tried to build "fb303" in the thrift sources.
>
> Note that I ran thrift's configure with a non-system installation target
> directory:
>
> ----------------------------------------------------------
> hive@orca:~$ cd tmp/thrift-0.9.0/contrib/fb303
> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ ./configure
> --prefix=/home/hive/thrift-0.9.0
> ----------------------------------------------------------
>
> But it could not find /usr/local/bin/thrift and
> /usr/local/include/thrift...
>
> So I created links:
>
> ----------------------------------------------------------
> /usr/local/bin/thrift -> /home/hive/thrift-0.9.0/bin/thrift
> /usr/local/include/thrift -> /home/hive/thrift-0.9.0/include/thrift
> ----------------------------------------------------------
>
> But:
>
> ----------------------------------------------------------
> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ make
>
> make all-recursive
> make[1]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
> Making all in .
> make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
> make[2]: Nothing to be done for `all-am'.
> make[2]: Leaving directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
> Making all in cpp
> make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
> make all-am
> make[3]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
> g++ -DPACKAGE_NAME=\"libfb303\" -DPACKAGE_TARNAME=\"libfb303\"
> -DPACKAGE_VERSION=\"20080209\" -DPACKAGE_STRING=\"libfb303\ 20080209\"
> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DHAVE_BOOST=/\*\*/
> -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1
> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1
> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1
> -DHAVE_INTTYPES_H=1 -DHAVE_NETINET_IN_H=1 -I. -I.. -Igen-cpp
> -I/usr/local/include/thrift -I/usr/include -Wall -O3 -MT
> FacebookService.o -MD -MP -MF .deps/FacebookService.Tpo -c -o
> FacebookService.o `test -f 'gen-cpp/FacebookService.cpp' || echo
> './'`gen-cpp/FacebookService.cpp
> In file included from gen-cpp/FacebookService.cpp:7:
> gen-cpp/FacebookService.h:10:39: error: thrift/TDispatchProcessor.h: No
> such file or directory
> In file included from gen-cpp/FacebookService.h:11,
> from gen-cpp/FacebookService.cpp:7:
>
> ----------------------------------------------------------
>
> ...
>
> Seb
>
>
> On 04/29/2013 08:48 PM, Carl Steinbach wrote:
>> Hi Sebastien,
>>
>> Most (if not all) of the ant targets will fail if run from the submodule
>> directories. You must run the targets from the root source directory
>> instead.
>>
>> Thanks.
>>
>> Carl
>>
>>
>> On Fri, Apr 26, 2013 at 6:09 AM, Sebastien FLAESCH <sf@4js.com
>> <mailto:sf@4js.com>> wrote:
>>
>> I have now installed Thrift, but I still get the error with the
>> "osfamily" task:
>>
>> Please help!
>>
>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>> -Dthrift.home=/home/hive/__thrift-0.9.0
>>
>>
>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>
>> BUILD FAILED
>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The following error
>> occurred while executing this line:
>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem: failed
>> to create task or type osfamily
>> Cause: The name is undefined.
>> Action: Check the spelling.
>> Action: Check that any custom tasks/types have been declared.
>> Action: Check that any <presetdef>/<macrodef> declarations have
>> taken place.
>>
>>
>> Total time: 0 seconds
>>
>> Seb
>>
>>
>> On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
>>
>> Go it, need to install Apache Thrift first ... sorry for this mail.
>> Seb
>>
>> On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
>>
>> Hi all,
>>
>> Just started with an Hadoop / Hive POC to write a
>>
>> So far, I have installed:
>>
>> So far I have downloaded the Hadoop and Hive packages from:
>>
>> http://hadoop.apache.org/__releases.html
>> <http://hadoop.apache.org/releases.html>
>> http://hive.apache.org/__releases.html
>> <http://hive.apache.org/releases.html>
>>
>> I took:
>>
>> http://hadoop.apache.org/__releases.html#18+April%2C+__2013%3A+Release+0.23.7+__available
>>
>> <http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available>
>>
>>
>>
>>
>> and
>>
>> http://hive.apache.org/__releases.html#11+January%2C+__2013%3A+release+0.10.0+__available
>>
>> <http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available>
>>
>>
>>
>>
>>
>> After setting the env, it seems that hive is working:
>>
>> hive@orca:~$ $HIVE_HOME/bin/hive
>> ...
>> hive> create table t1 ( k int, s string );
>> OK
>>
>> hive> select count(*) from t1;
>> Total MapReduce jobs = 1
>> ...
>> OK
>> 0
>> Time taken: 7.634 seconds
>>
>>
>> No I want to connect through ODBC...
>>
>> Following the instructions of this page:
>>
>> https://cwiki.apache.org/__confluence/display/Hive/__HiveODBC
>> <https://cwiki.apache.org/confluence/display/Hive/HiveODBC>
>>
>> Section "Hive Client Build/Setup", I tried:
>>
>> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
>> hive@orca:~/hive-0.10.0$ ant compile-cpp
>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>> Buildfile: build.xml does not exist!
>> Build failed
>>
>> Then, from the sources directory src/odbc:
>>
>> hive@orca:~/hive-0.10.0$ cd src/odbc/
>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>
>> BUILD FAILED
>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The
>> following error
>> occurred while executing this line:
>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem:
>> failed to
>> create task or type osfamily
>> Cause: The name is undefined.
>> Action: Check the spelling.
>> Action: Check that any custom tasks/types have been declared.
>> Action: Check that any <presetdef>/<macrodef> declarations
>> have taken
>> place.
>>
>>
>> Total time: 0 seconds
>>
>>
>> Can someone help or point me to an up to date documentation?
>>
>> Thanks!
>> Seb
>>
>>
>>
>>
>


Mime
View raw message