hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joydeep Sen Sarma <jssa...@facebook.com>
Subject RE: Latest trunk breaks logging?
Date Mon, 21 Jun 2010 22:30:22 GMT
543 changes logging for local mode. By default, the logs from local mode map-reduce tasks now
go to a per-query log file that's written to /tmp/<username>/<queryid>.log

This was very much intended to enable local mode to be a bit friendlier to use.

-----Original Message-----
From: Mayank Lahiri [mailto:mayank.lahiri@facebook.com] 
Sent: Monday, June 21, 2010 2:54 PM
To: John Sichi
Cc: hive-dev@hadoop.apache.org
Subject: Re: Latest trunk breaks logging?

Hi Joydeep,

I've confirmed that logging is restored after reverting HIVE-543. The problem is that LOG.warn()
calls from inside UDAFs do not generate any output after applying HIVE-543. For example, passing
an invalid argument to histogram() causes a general one-line exception instead of the diagnostic
HiveException that is supposed to be thrown. This is what HEAD currently returns:

----- snip ------
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
---- end snip ----

Was this intended, or are the logs just being written to some other location post-HIVE-543?

Thanks,
-Mayank


On 6/21/10 1:00 PM, "John Sichi" <jsichi@facebook.com> wrote:

+hive-dev

Looks like Joydeep made some changes to the logging in recently committed HIVE-543; maybe
related?

Apache JIRA is down at the moment but I found the patch here:

http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3c20100616225037.C67DF23888E3@eris.apache.org%3e

Mayank, if you can confirm that this is the cause, we can check with Joydeep on whether or
not this was intentional.

The DataNucleus noise I've been seeing for a while now; I think it's harmless, but go ahead
and create a JIRA issue to get it cleaned up.

JVS

On Jun 21, 2010, at 12:34 PM, Mayank Lahiri wrote:

Hi John,

I just updated trunk and logging seems to be slightly different, and possibly broken. For
one, LOG.warn() messages from inside UDAFs don't show up anywhere, and are not printed to
console. /tmp/mlahiri/hive.log contains a lot of lines like this:

2010-06-21 12:02:01,486 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.core.runtime" but it cannot be resolved.
2010-06-21 12:02:01,487 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.text" but it cannot be resolved.
2010-06-21 12:23:23,576 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.core.resources" but it cannot be resolved.
2010-06-21 12:23:23,578 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.core.runtime" but it cannot be resolved.
2010-06-21 12:23:23,579 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.text" but it cannot be resolved.
2010-06-21 12:27:45,003 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.core.resources" but it cannot be resolved.
2010-06-21 12:27:45,005 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.core.runtime" but it cannot be resolved.
2010-06-21 12:27:45,005 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.text" but it cannot be resolved.
2010-06-21 12:30:13,786 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.core.resources" but it cannot be resolved.
2010-06-21 12:30:13,788 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.core.runtime" but it cannot be resolved.
2010-06-21 12:30:13,789 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c
ore" requires "org.eclipse.text" but it cannot be resolved.
~

Any ideas what I could be doing differently/wrong?

- Mayank



Mime
View raw message