kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chao Long <chao.long0...@gmail.com>
Subject Re: Re: FactDistinctColumnsJob.run throw exception:IllegalStateException
Date Mon, 15 Apr 2019 02:49:05 GMT
You can take a look at this commit:
https://github.com/apache/kylin/commit/b780759c6641f0dfbd57f91d4f3d568b8dbb46f6

On Sat, Apr 13, 2019 at 5:58 PM bubugao0809 <bubugao0809@163.com> wrote:

> The bypassing seems a little tricky, especially in a auto building
> production env, in which one can not manually reload meta every time job
> failed.
> The issue can severely damege the robustness of kylin cluster.
> Is there any clue to fix the issue "It seems caused by the broadcaster
> doesn't broadcast metadata to the job engine in time." ?
> I found the snippet from source code :
>
> CubeSegment segment = cube.getSegmentById(segmentID);
> if (segment == null) {
>     logger.error("Failed to find {} in cube {}", segmentID, cube);
>     for (CubeSegment s : cube.getSegments()) {
>         logger.error(s.getName() + " with status " + s.getStatus());
>     }
>     throw new IllegalStateException();
> }
>
>
>
>
> At 2019-04-13 12:42:08, "Chao Long" <chao.long0101@gmail.com> wrote:
>
> It seems caused by the broadcaster doesn't broadcast metadata to the job
> engine in time. There is a workaround you can try: reload the metadata and
> resume the failed job.
>
> There are two related issues:
> https://issues.apache.org/jira/browse/KYLIN-3740
> https://issues.apache.org/jira/browse/KYLIN-3752
>
> On Fri, Apr 12, 2019 at 11:56 PM venkatesh Mora <
> venkatesh.mora@scanbuy.com> wrote:
>
>> I am running into the same. Can someone respond on this. I opened this a
>> couple of days ago.
>>
>> On Apr 11, 2019, at 9:31 PM, bubugao0809 <bubugao0809@163.com> wrote:
>>
>>
>> Hi all,
>>      I am using kylin-2.2.0, and some of my job constantly failed at the
>> third stage of "Extract Fact Table Distinct Columns", with exception
>> below:
>>
>> java.lang.IllegalStateException
>> 	at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:116)
>> 	at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:129)
>> 	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
>> 	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
>> 	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)
>> 	at org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:171)
>> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> 	at java.lang.Thread.run(Thread.java:748)
>>
>> result code:2
>>
>>
>>

Mime
View raw message