hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Shelukhin <ser...@hortonworks.com>
Subject Re: Execution log for qfile tests
Date Fri, 28 Feb 2014 21:28:36 GMT
That doesn't work for me... I will take a look later.


On Mon, Feb 24, 2014 at 2:11 PM, Brock Noland <brock@cloudera.com> wrote:

> They always show up in ${system:java.io.tmp}/$USER for me. For example:
>
> $ head -5 $TMPDIR/noland/noland*
> ==>
> /var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T//noland/noland_20140221162424_103e159a-4511-42ae-a84a-01cd95820886.log
> <==
> 2014-02-21 16:24:36,807 WARN  common.LogUtils
> (LogUtils.java:logConfigLocation(145)) - hive-site.xml not found on
> CLASSPATH
> 2014-02-21 16:24:36,816 INFO  mr.ExecDriver
> (SessionState.java:printInfo(417)) - Execution log at:
>
> /var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T//noland/noland_20140221162424_103e159a-4511-42ae-a84a-01cd95820886.log
> 2014-02-21 16:24:36,974 WARN  conf.Configuration
> (Configuration.java:loadProperty(2345)) -
>
> file:/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T/noland/hive_2014-02-21_16-24-32_176_6585530032622480558-3/-local-10002/jobconf.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval;  Ignoring.
> 2014-02-21 16:24:36,985 WARN  conf.Configuration
> (Configuration.java:loadProperty(2345)) -
>
> file:/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T/noland/hive_2014-02-21_16-24-32_176_6585530032622480558-3/-local-10002/jobconf.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts;  Ignoring.
> 2014-02-21 16:24:37,071 INFO  log.PerfLogger
> (PerfLogger.java:PerfLogBegin(97)) - <PERFLOG method=deserializePlan
> from=org.apache.hadoop.hive.ql.exec.Utilities>
>
> ==>
> /var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T//noland/noland_20140221162424_5629b8d1-77f3-4180-a000-220c0e551812.log
> <==
> 2014-02-21 16:24:27,004 WARN  common.LogUtils
> (LogUtils.java:logConfigLocation(145)) - hive-site.xml not found on
> CLASSPATH
> 2014-02-21 16:24:27,013 INFO  mr.ExecDriver
> (SessionState.java:printInfo(417)) - Execution log at:
>
> /var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T//noland/noland_20140221162424_5629b8d1-77f3-4180-a000-220c0e551812.log
> 2014-02-21 16:24:27,175 WARN  conf.Configuration
> (Configuration.java:loadProperty(2345)) -
>
> file:/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T/noland/hive_2014-02-21_16-24-20_980_1410307595308328042-2/-local-10002/jobconf.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval;  Ignoring.
> 2014-02-21 16:24:27,186 WARN  conf.Configuration
> (Configuration.java:loadProperty(2345)) -
>
> file:/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T/noland/hive_2014-02-21_16-24-20_980_1410307595308328042-2/-local-10002/jobconf.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts;  Ignoring.
> 2014-02-21 16:24:27,279 INFO  log.PerfLogger
> (PerfLogger.java:PerfLogBegin(97)) - <PERFLOG method=deserializePlan
> from=org.apache.hadoop.hive.ql.exec.Utilities>
>
> On Mon, Feb 24, 2014 at 4:00 PM, Sergey Shelukhin
> <sergey@hortonworks.com> wrote:
> > That doesn't contain logs from local MR tasks for me...
> >
> > So all there is is
> >
> > 2014-02-24 13:50:35,434 INFO  mr.MapredLocalTask
> > (MapredLocalTask.java:execute(254)) - Executing:
> > /Users/sergey/git/hive/itests/qtest/../../testutils/hadoop jar
> >
> /Users/sergey/.m2/repository/org/apache/hive/hive-exec/0.13.0-SNAPSHOT/hive-exec-0.13.0-SNAPSHOT.jar
> > org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
> >
> file:/Users/sergey/git/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-24_13-50-35_104_1296981405740899980-1/-local-10006/plan.xml
> > -jobconffile
> >
> file:/Users/sergey/git/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-24_13-50-35_104_1296981405740899980-1/-local-10007/jobconf.xml
> > ...
> > 2014-02-24 13:50:41,680 ERROR exec.Task
> (SessionState.java:printError(533))
> > - Execution failed with exit status: 2
> > 2014-02-24 13:50:41,680 ERROR exec.Task
> (SessionState.java:printError(533))
> > - Obtaining error information
> > 2014-02-24 13:50:41,681 ERROR exec.Task
> (SessionState.java:printError(533))
> > -
> > Task failed!
> > Task ID:
> >   Stage-6
> >
> > Logs:
> >
> > 2014-02-24 13:50:41,681 ERROR exec.Task
> (SessionState.java:printError(533))
> > - /Users/sergey/git/hive/itests/qtest/target/tmp/log/hive.log
> > 2014-02-24 13:50:41,681 ERROR mr.MapredLocalTask
> > (MapredLocalTask.java:execute(270)) - Execution failed with exit status:
> 2
> >
> > and I can't find the logs anywhere... do you know where they are if
> > anywhere?
> >
> >
> > On Mon, Feb 17, 2014 at 6:51 PM, Lefty Leverenz <leftyleverenz@gmail.com
> >wrote:
> >
> >> Sweet.  Thanks Brock.
> >>
> >> -- Lefty
> >>
> >>
> >> On Mon, Feb 17, 2014 at 6:43 PM, Brock Noland <brock@cloudera.com>
> wrote:
> >>
> >> > Good call! I added this here:
> >> > https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ
> >> >
> >> > On Mon, Feb 17, 2014 at 8:24 PM, Lefty Leverenz <
> leftyleverenz@gmail.com
> >> >
> >> > wrote:
> >> > > Should this be mentioned somewhere in the Hive wiki, or is just a
> >> Hadoop
> >> > > doc issue?
> >> > >
> >> > > -- Lefty
> >> > >
> >> > >
> >> > > On Mon, Feb 17, 2014 at 6:21 PM, Brock Noland <brock@cloudera.com>
> >> > wrote:
> >> > >
> >> > >> It should be in /tmp/$USER/ on linux or $TMPDIR/$USER/ on a mac.
> >> > >>
> >> > >> On Mon, Feb 17, 2014 at 6:48 PM, Prasanth Jayachandran
> >> > >> <pjayachandran@hortonworks.com> wrote:
> >> > >> > Hi All
> >> > >> >
> >> > >> > Can some point me to the location of hadoop exec log when
I run a
> >> > >> particular qfile test?
> >> > >> >
> >> > >> > I can see from hive.log that the job failed with following
error
> >> > >> > 2014-02-17 16:38:52,774 ERROR exec.Task
> >> > >> (SessionState.java:printError(524)) -
> >> > >> > Task failed!
> >> > >> > Task ID:
> >> > >> >   Stage-1
> >> > >> >
> >> > >> > Logs:
> >> > >> >
> >> > >> > But this does not provide the location of hadoop execution
log
> >> (local
> >> > >> job runner logs). Is the location of log file for execution log
> >> > >> configurable? I tried copying my custom hive-exec-log4j.properties
> to
> >> > >> <hive-src>/data/conf directory but that doesn't help.
> >> > >> >
> >> > >> > Thanks
> >> > >> > Prasanth Jayachandran
> >> > >> >
> >> > >> >
> >> > >> > --
> >> > >> > CONFIDENTIALITY NOTICE
> >> > >> > NOTICE: This message is intended for the use of the individual
or
> >> > entity
> >> > >> to
> >> > >> > which it is addressed and may contain information that is
> >> > confidential,
> >> > >> > privileged and exempt from disclosure under applicable law.
If
> the
> >> > reader
> >> > >> > of this message is not the intended recipient, you are hereby
> >> notified
> >> > >> that
> >> > >> > any printing, copying, dissemination, distribution, disclosure
or
> >> > >> > forwarding of this communication is strictly prohibited.
If you
> have
> >> > >> > received this communication in error, please contact the
sender
> >> > >> immediately
> >> > >> > and delete it from your system. Thank You.
> >> > >>
> >> > >>
> >> > >>
> >> > >> --
> >> > >> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
> >> > >>
> >> >
> >> >
> >> >
> >> > --
> >> > Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
> >> >
> >>
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message