Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Received-SPF: pass (nike.apache.org: domain of brock@cloudera.com designates
 209.85.216.172 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAHXxaiChWJbvCfo6nsGBoVVOTjRLHzOoQ8j-N59FawaRxW5F=A@mail.gmail.com>
References: <9DAF2DF4-7829-4368-9FD6-5C3B049D9A1A@hortonworks.com>
 <CAFukC=6iVCONTN8RydOFE_Ev74AKZL+cnacQWTLKS-AJ7waA5A@mail.gmail.com>
 <CACEcauUhLusNsA0wQsnSc+mYkRCfrip1ESjV7BH-Qk0UQWYvrw@mail.gmail.com>
 <CAFukC=6iPQsuAxJdWzGfPQZybVcgaYSWg3ku8RT9UHATQgmEmg@mail.gmail.com>
 <CACEcauVh41vHCedL93j_OPOe3Fe9j1D5RMD9CK7U_NWc=YoqAA@mail.gmail.com>
 <CAHXxaiChWJbvCfo6nsGBoVVOTjRLHzOoQ8j-N59FawaRxW5F=A@mail.gmail.com>
From: Brock Noland <brock@cloudera.com>
Date: Mon, 24 Feb 2014 16:11:18 -0600
Message-ID: 
 <CAFukC=5fQFk5i2JmtF=9tzYLzA2cKA00qJu6MpT8YyfvszpCfw@mail.gmail.com>
Subject: Re: Execution log for qfile tests
To: "dev@hive.apache.org" <dev@hive.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

They always show up in ${system:java.io.tmp}/$USER for me. For example:

$ head -5 $TMPDIR/noland/noland*
==> /var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T//noland/noland_20140221162424_103e159a-4511-42ae-a84a-01cd95820886.log
<==
2014-02-21 16:24:36,807 WARN  common.LogUtils
(LogUtils.java:logConfigLocation(145)) - hive-site.xml not found on
CLASSPATH
2014-02-21 16:24:36,816 INFO  mr.ExecDriver
(SessionState.java:printInfo(417)) - Execution log at:
/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T//noland/noland_20140221162424_103e159a-4511-42ae-a84a-01cd95820886.log
2014-02-21 16:24:36,974 WARN  conf.Configuration
(Configuration.java:loadProperty(2345)) -
file:/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T/noland/hive_2014-02-21_16-24-32_176_6585530032622480558-3/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-02-21 16:24:36,985 WARN  conf.Configuration
(Configuration.java:loadProperty(2345)) -
file:/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T/noland/hive_2014-02-21_16-24-32_176_6585530032622480558-3/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-02-21 16:24:37,071 INFO  log.PerfLogger
(PerfLogger.java:PerfLogBegin(97)) - <PERFLOG method=deserializePlan
from=org.apache.hadoop.hive.ql.exec.Utilities>

==> /var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T//noland/noland_20140221162424_5629b8d1-77f3-4180-a000-220c0e551812.log
<==
2014-02-21 16:24:27,004 WARN  common.LogUtils
(LogUtils.java:logConfigLocation(145)) - hive-site.xml not found on
CLASSPATH
2014-02-21 16:24:27,013 INFO  mr.ExecDriver
(SessionState.java:printInfo(417)) - Execution log at:
/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T//noland/noland_20140221162424_5629b8d1-77f3-4180-a000-220c0e551812.log
2014-02-21 16:24:27,175 WARN  conf.Configuration
(Configuration.java:loadProperty(2345)) -
file:/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T/noland/hive_2014-02-21_16-24-20_980_1410307595308328042-2/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-02-21 16:24:27,186 WARN  conf.Configuration
(Configuration.java:loadProperty(2345)) -
file:/var/folders/6l/2kf3r2pj1t176h2nhdwfpk1r0000gp/T/noland/hive_2014-02-21_16-24-20_980_1410307595308328042-2/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-02-21 16:24:27,279 INFO  log.PerfLogger
(PerfLogger.java:PerfLogBegin(97)) - <PERFLOG method=deserializePlan
from=org.apache.hadoop.hive.ql.exec.Utilities>

On Mon, Feb 24, 2014 at 4:00 PM, Sergey Shelukhin
<sergey@hortonworks.com> wrote:
> That doesn't contain logs from local MR tasks for me...
>
> So all there is is
>
> 2014-02-24 13:50:35,434 INFO  mr.MapredLocalTask
> (MapredLocalTask.java:execute(254)) - Executing:
> /Users/sergey/git/hive/itests/qtest/../../testutils/hadoop jar
> /Users/sergey/.m2/repository/org/apache/hive/hive-exec/0.13.0-SNAPSHOT/hive-exec-0.13.0-SNAPSHOT.jar
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan
> file:/Users/sergey/git/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-24_13-50-35_104_1296981405740899980-1/-local-10006/plan.xml
> -jobconffile
> file:/Users/sergey/git/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-24_13-50-35_104_1296981405740899980-1/-local-10007/jobconf.xml
> ...
> 2014-02-24 13:50:41,680 ERROR exec.Task (SessionState.java:printError(533))
> - Execution failed with exit status: 2
> 2014-02-24 13:50:41,680 ERROR exec.Task (SessionState.java:printError(533))
> - Obtaining error information
> 2014-02-24 13:50:41,681 ERROR exec.Task (SessionState.java:printError(533))
> -
> Task failed!
> Task ID:
>   Stage-6
>
> Logs:
>
> 2014-02-24 13:50:41,681 ERROR exec.Task (SessionState.java:printError(533))
> - /Users/sergey/git/hive/itests/qtest/target/tmp/log/hive.log
> 2014-02-24 13:50:41,681 ERROR mr.MapredLocalTask
> (MapredLocalTask.java:execute(270)) - Execution failed with exit status: 2
>
> and I can't find the logs anywhere... do you know where they are if
> anywhere?
>
>
> On Mon, Feb 17, 2014 at 6:51 PM, Lefty Leverenz <leftyleverenz@gmail.com>wrote:
>
>> Sweet.  Thanks Brock.
>>
>> -- Lefty
>>
>>
>> On Mon, Feb 17, 2014 at 6:43 PM, Brock Noland <brock@cloudera.com> wrote:
>>
>> > Good call! I added this here:
>> > https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ
>> >
>> > On Mon, Feb 17, 2014 at 8:24 PM, Lefty Leverenz <leftyleverenz@gmail.com
>> >
>> > wrote:
>> > > Should this be mentioned somewhere in the Hive wiki, or is just a
>> Hadoop
>> > > doc issue?
>> > >
>> > > -- Lefty
>> > >
>> > >
>> > > On Mon, Feb 17, 2014 at 6:21 PM, Brock Noland <brock@cloudera.com>
>> > wrote:
>> > >
>> > >> It should be in /tmp/$USER/ on linux or $TMPDIR/$USER/ on a mac.
>> > >>
>> > >> On Mon, Feb 17, 2014 at 6:48 PM, Prasanth Jayachandran
>> > >> <pjayachandran@hortonworks.com> wrote:
>> > >> > Hi All
>> > >> >
>> > >> > Can some point me to the location of hadoop exec log when I run a
>> > >> particular qfile test?
>> > >> >
>> > >> > I can see from hive.log that the job failed with following error
>> > >> > 2014-02-17 16:38:52,774 ERROR exec.Task
>> > >> (SessionState.java:printError(524)) -
>> > >> > Task failed!
>> > >> > Task ID:
>> > >> >   Stage-1
>> > >> >
>> > >> > Logs:
>> > >> >
>> > >> > But this does not provide the location of hadoop execution log
>> (local
>> > >> job runner logs). Is the location of log file for execution log
>> > >> configurable? I tried copying my custom hive-exec-log4j.properties to
>> > >> <hive-src>/data/conf directory but that doesn't help.
>> > >> >
>> > >> > Thanks
>> > >> > Prasanth Jayachandran
>> > >> >
>> > >> >
>> > >> > --
>> > >> > CONFIDENTIALITY NOTICE
>> > >> > NOTICE: This message is intended for the use of the individual or
>> > entity
>> > >> to
>> > >> > which it is addressed and may contain information that is
>> > confidential,
>> > >> > privileged and exempt from disclosure under applicable law. If the
>> > reader
>> > >> > of this message is not the intended recipient, you are hereby
>> notified
>> > >> that
>> > >> > any printing, copying, dissemination, distribution, disclosure or
>> > >> > forwarding of this communication is strictly prohibited. If you have
>> > >> > received this communication in error, please contact the sender
>> > >> immediately
>> > >> > and delete it from your system. Thank You.
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
>> > >>
>> >
>> >
>> >
>> > --
>> > Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
>> >
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.


-- 
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org