hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Melick <rmel...@hmc.edu>
Subject Debugging Hive
Date Wed, 17 Nov 2010 07:33:51 GMT
Hi all,

Apologies for the long email.  I'm attempting to connect to the hive
code with a debugger to understand the optimizer.  In this example, I
would like to pause at the beginning of the createIndex function in
DDLTask.  After following the directions on the wiki, I'm stumped as to
why connecting remotely with jdb is not working.  Any help is
appreciated.  I've detailed my process below.

Thanks,
Russell Melick



First, I check out a fresh copy of trunk:
        
> svn checkout http://svn.apache.org/repos/asf/hive/trunk hive

Then, I compile it for debugging, as I found at http://wiki.apache.org/hadoop/Hive/DeveloperGuide#Debugging_Hive_code:

> cd hive
> ant -Djavac.debug=on package

Here are the important variables before I run hive:
> echo $HIVE_DEBUG_PORT
> 8000

> echo $HIVE_DEBUG
> -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y

> echo $HADOOP_OPTS
> -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y

I then try to run one of the unit tests:

> ant test -Dtestcase=TestCliDriver -Dqfile=index_compact_1.q

The test then waits for a debugger to attach:
> [junit] Begin query: index_compact_1.q
>     [junit] Listening for transport dt_socket at address: 8000

However, when I try to attach with jdb, it doesn't work.

> jdb -attach 8000
> Set uncaught java.lang.Throwable
> Set deferred uncaught java.lang.Throwable
> Initializing jdb ...
> > 
> VM Started: No frames on the current call stack
> 
> main[1] stop at DDLTask:317
> Deferring breakpoint DDLTask:317.
> It will be set after the class is loaded.
> main[1] resume
> All threads resumed.
> > 
> The application exited

The unit test does not seem to recognize that it received the connection.  After jdb prints
"application exited", the test says it is listening again, but it didn't confirm the connection.

> [junit] Begin query: index_compact_1.q
> [junit] Listening for transport dt_socket at address: 8000
> [junit] Deleted file:/home/rmelick/hive/build/ql/test/data/warehouse/default__src_src_index__
> [junit] Listening for transport dt_socket at address: 8000
> [junit] Listening for transport dt_socket at address: 8000

I have also attempted to start hiveCli, but I have had no success getting the CLI to start
after attaching the debugger.

This is the only output I get from the cli:
> build/dist/bin/hive
> Unable to determine Hadoop version information.
> 'hadoop version' returned:

jdb doesn't seem to be connecting correctly
> jdb -attach 8000
> Set uncaught java.lang.Throwable
> Set deferred uncaught java.lang.Throwable
> Initializing jdb ...
> > 
> VM Started: No frames on the current call stack
> 
> main[1] stop at DDLTask:317
> Deferring breakpoint DDLTask:317.
> It will be set after the class is loaded.
> main[1] resume
> All threads resumed.
> > 
> The application exited

I've also tried connecting with Eclipse, after generating the eclipse files, importing the
project, and creating the MapRedTask configuration.  It either seems to similarly connect
without the client recognizing it, or throws a connection error.

Again, thanks for any help.



Mime
View raw message