kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-3406) When the /execute_output/ metadata file sinking to HDFS is deleted, the monitoring page does not display any tasks.
Date Thu, 06 Dec 2018 02:04:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710891#comment-16710891
] 

ASF GitHub Bot commented on KYLIN-3406:
---------------------------------------

hit-lacus opened a new pull request #373: KYLIN-3406 Ignore execute output file lose
URL: https://github.com/apache/kylin/pull/373
 
 
   ## Root Cause
   When any single file of Kylin's metadata miss in HDFS, the "Monitor" Page will failed to
display all job info and kylin process will throw FileNotFoundException as above. The page
has a warning message.
   ## Fix solution
   The better way should be : miss file should only affect specific task. And if only execute
output miss, task should be success, otherwise that should be a fatal exeception. You have
to fix it manually.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> When the /execute_output/ metadata file sinking to HDFS is deleted, the monitoring page
does not display any tasks.
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: KYLIN-3406
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3406
>             Project: Kylin
>          Issue Type: Bug
>          Components: Metadata
>    Affects Versions: v2.3.1
>            Reporter: yujie.sun
>            Assignee: XiaoXiang Yu
>            Priority: Major
>             Fix For: v2.6.0
>
>         Attachments: Lost Execute Output.png, Step which lost execute output.png, image-2018-12-05-13-22-42-456.png,
image-2018-12-05-18-45-32-923.png
>
>
> *{color:#24292e}Issue description:{color}*
> When a single file of Hbase metadata is greater than a certain value, the file is sunk
into HDFS for storage. When files in execute_output/ are deleted in HDFS, the monitoring page
will not display any tasks.
> *{color:#24292e}Reproduction step:{color}*
> {color:#24292e}1.Use Hbase as metadata storage{color}
> {color:#24292e}2.Modify the hbase configuration ,configure "hbase.client.keyvalue.maxsize"
into a small value, such as 524288.{color}
> {color:#24292e}3.Create a new build task, after the build task finished,find the /execute_output
hbase metadata path on HDFS, /\{working_dir}/\{metadata_url}/resources/execute_output/...,
and delete one of the files.{color}
> {color:#24292e}4.At this time,{color:#333333}the monitoring page will not display any
tasks.{color}{color}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message