impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wang, Youwei A" <youwei.a.w...@intel.com>
Subject Symbol not found issue when working on IMPALA-3504
Date Thu, 12 May 2016 13:16:06 GMT
Greetings All. I am working on IMPALA-3504 right now.
https://issues.cloudera.org/browse/IMPALA-3504
This JIRA basically wants to implement a UDF timestamp function. Following is my process when
working on this JIRA:

1. Back-end source code:
I have added an function entry in the file timestamp-functions.h:
static TimestampVal UtcTimestamp(FunctionContext* context);
and added corresponding implementation in the file timestamp-functions.cc.

2. Build back-end:
And after that, I first build impalad binary using make_debug.sh script.
After the impalad binary is generated, I run Linux "strings" command to get the binary signature
of the function like: _ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE.
And this entry is also added to the Impala/common/function-registry/impala_functions.py file
like:
  [['utc_timestamp'], 'TIMESTAMP', [], '_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE'],

3. Overall build:
Then I execute the buildall.sh script to initiate build all front-end, back-end and third-party
tools.

Issue:
When the build process is done, I use the command:"start-impala-cluster.py  -r --build_type=debug"
to start the Impala cluster. However, the impalad process fails to run due to the error which
is copied from Impala/logs/cluster/impalad.INFO:

I0512 20:35:46.621750 31842 init.cc:204] Physical Memory: 11.69 GB
I0512 20:35:46.621752 31842 init.cc:205] OS version: Linux version 4.5.3 (root@debian) (gcc
version 4.9.2 (Debian 4.9.2-10) ) #1 SMP Tue May 10 09:19:36 EDT 2016
Clock: clocksource: 'tsc', clockid_t: CLOCK_MONOTONIC
I0512 20:35:46.621754 31842 init.cc:206] Process ID: 31842
I0512 20:35:47.307698 31842 status.cc:40] Kudu is not supported on this operating system.
    @          0x10de550  impala::Status::Status()
    @          0x10e1984  impala::CheckKuduAvailability()
    @          0x10e18f7  impala::KuduIsAvailable()
    @          0x10d89c1  impala::InitCommonRuntime()
    @          0x1446db1  ImpaladMain()
    @          0x1088720  main
    @     0x7f214e93ab45  (unknown)
    @          0x108859c  (unknown)
I0512 20:35:47.689296 31842 hbase-table-scanner.cc:161] Detected HBase version >= 0.95.2
I0512 20:35:47.700866 31842 GlogAppender.java:123] Logging initialized. Impala: VLOG, All
other: INFO
I0512 20:35:47.702227 31842 JniFrontend.java:125] Authorization is 'DISABLED'.
I0512 20:35:47.702286 31842 JniFrontend.java:127] Java Version Info: Java(TM) SE Runtime Environment
(1.8.0_65-b17)
W0512 20:35:47.871737 31842 HiveConf.java:2721] HiveConf of name hive.access.conf.url does
not exist
I0512 20:35:47.958432 31842 fe-support.cc:201] Could not find symbol '_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE'
in:
Unable to find _ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE
dlerror: /root/Impala/be/build/debug/service/impalad: undefined symbol: _ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE
F0512 20:35:47.959112 31842 frontend.cc:107] java.lang.RuntimeException: Builtin symbol '_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE'[]
not found!
            at com.cloudera.impala.catalog.ScalarFunction.createBuiltin(ScalarFunction.java:79)
            at com.cloudera.impala.catalog.Db.addScalarBuiltin(Db.java:389)
            at com.cloudera.impala.catalog.Db.addScalarBuiltin(Db.java:379)
            at com.cloudera.impala.builtins.ScalarBuiltins.initBuiltins(ScalarBuiltins.java:124)
            at com.cloudera.impala.catalog.BuiltinsDb.initBuiltins(BuiltinsDb.java:58)
            at com.cloudera.impala.catalog.BuiltinsDb.<init>(BuiltinsDb.java:39)
            at com.cloudera.impala.catalog.Catalog.<init>(Catalog.java:97)
            at com.cloudera.impala.catalog.ImpaladCatalog.<init>(ImpaladCatalog.java:92)
            at com.cloudera.impala.service.Frontend.<init>(Frontend.java:170)
            at com.cloudera.impala.service.JniFrontend.<init>(JniFrontend.java:129)
Caused by: com.cloudera.impala.common.AnalysisException: Could not find symbol '_ZN6impala18TimestampFunctions12UtcTimestampEPN10impala_udf15FunctionContextE'
in:
            at com.cloudera.impala.catalog.Function.lookupSymbol(Function.java:423)
            at com.cloudera.impala.catalog.ScalarFunction.createBuiltin(ScalarFunction.java:75)
            ... 9 more

This is really weird since such symbol "ZN6impala18TimestampFunctions12UtcTimestampEPN10impala"
is indeed in the impalad binary and I have double-checked that. Actually I have some successful
experience of developing some UDF before. So this issue really makes me feel confused. I have
removed all built binaries and replay the steps above. This issue still remains there.

Thank you so much for any advice or hint. :)



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message