kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakob Stengård <jakob.steng...@gmail.com>
Subject Re: Tez
Date Mon, 02 Mar 2015 18:08:39 GMT
I'll file a bug in jira.
For your information i also happen to know that tez boosted map-reduce
doesn't work either, but there, the problem seems to have to do with the
map reduce jobs them selves failing.

In this case however, from what i've read on git hub about the "Can't get
cube source record size",
error", it could possibly be a problem with that tez doesn't output a
string on the form:
"HDFS Read: (\d+) HDFS Write"

However, it could be something else, it could also be my logging
configuration for tez.

/J



On Mon, Mar 2, 2015 at 6:54 PM, Adunuthula, Seshu <sadunuthula@ebay.com>
wrote:

> Did not read your logs fully. The culprit seems to be
>
>
> 1. [QuartzScheduler_Worker-7]:[2015-03-02
> 16:09:24,737][ERROR][com.kylinolap.job.flow.JobFlowListener.handleException
> (JobFlowListener.java:312)] - Can't get cube source record size.
>
> source_records_size may have to be computed differently when using Tez as
> the Hive engine. I will let Yang/Luke answer this when they come back
> onlineŠ
>
>
>
>
> On 3/2/15, 9:30 AM, "Adunuthula, Seshu" <sadunuthula@ebay.com> wrote:
>
> >Jakob,
> >
> >Could you go ahead and file a JIRA with the stack trace at
> >https://issues.apache.org/jira/browse/KYLIN/
> >
> >Also include the App Master and Map/Reduce Task logs for the Tasks that
> >failed.
> >
> >Regards
> >Seshu
> >
> >On 3/2/15, 8:12 AM, "Jakob Stengård" <jakob.stengard@gmail.com> wrote:
> >
> >>Correction: That, isn't the last job, it's step 9.
> >>
> >>/J
> >>
> >>On Mon, Mar 2, 2015 at 5:10 PM, Jakob Stengård <jakob.stengard@gmail.com
> >
> >>wrote:
> >>
> >>> Well, this only happens with
> >>> <property>
> >>>   <name>hive.execution.engine</name>
> >>>   <value>tez</value>
> >>> </property>
> >>>
> >>> If i use mr, it works fine.
> >>>
> >>>
> >>> From what i can see form the job log, none of the previous map-reduce
> >>>jobs
> >>> fail, but the build fails at the last step.
> >>> Log from the last mr job for example:
> >>> http://pastebin.com/jNWSRKLi
> >>>
> >>>
> >>> Is tez supposed to work? Because in that case, maybe it's just my
> >>>setup.
> >>>
> >>> /Jakob
> >>>
> >>>
> >>>
> >>> On Mon, Mar 2, 2015 at 4:34 PM, Santosh Akhilesh <
> >>> santoshakhilesh@gmail.com> wrote:
> >>>
> >>>> This doesn't seems to be complete kylin.log. Do you have any exception
> >>>> related to MR counter in map reduce logs? If yes then this needs to
be
> >>>> checked because kylin will need the MR counter to calculate the cube
> >>>>size.
> >>>> On Mon, 2 Mar 2015 at 8:49 pm, Jakob Stengård
> >>>><jakob.stengard@gmail.com>
> >>>> wrote:
> >>>>
> >>>> > It breaks on the last step: "Load HFile to HBase Table":
> >>>> >
> >>>> > Kylin.log:
> >>>> > http://pastebin.com/3kAQmJ3u
> >>>> >
> >>>> > On Mon, Mar 2, 2015 at 2:27 PM, Adunuthula, Seshu
> >>>><sadunuthula@ebay.com
> >>>> >
> >>>> > wrote:
> >>>> >
> >>>> > > Jakob,
> >>>> > >
> >>>> > > Could you please send out the complete stack trace with Tez
> >>>>enabled?
> >>>> > >
> >>>> > > Regards
> >>>> > > Seshu
> >>>> > >
> >>>> > > On 3/2/15, 4:51 AM, "Jakob Stengård" <jakob.stengard@gmail.com>
> >>>> wrote:
> >>>> > >
> >>>> > > >Hi
> >>>> > > >Will kylin suport hive+tez in v0.7.x?
> >>>> > > >
> >>>> > > >I've tried using tez with hive, but currently the cube
build
> >>>>fails
> >>>> with
> >>>> > > >"Can't get cube source record size", because kylin expects
> >>>>certain
> >>>> > console
> >>>> > > >output from map-reduce, and tez doesn't print this.
> >>>> > > >
> >>>> > > >BG.
> >>>> > > >Jakob Stengard
> >>>> > >
> >>>> > >
> >>>> >
> >>>>
> >>>
> >>>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message