kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shaofeng SHI (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KYLIN-3653) After kylin configured with hive data source with beeline, build failed if two jobs for creating flat table submitted at same time
Date Fri, 02 Nov 2018 11:03:00 GMT

     [ https://issues.apache.org/jira/browse/KYLIN-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shaofeng SHI resolved KYLIN-3653.
---------------------------------
    Resolution: Fixed

> After kylin configured with hive data source with beeline, build failed if two jobs for
creating flat table submitted at same time
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KYLIN-3653
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3653
>             Project: Kylin
>          Issue Type: Bug
>          Components: Job Engine
>    Affects Versions: v2.4.1
>            Reporter: nichunen
>            Assignee: nichunen
>            Priority: Major
>             Fix For: v2.6.0
>
>
> The root cause is in org.apache.kylin.common.util.HiveCmdBuilder#build
> {code:java}
>         case BEELINE:
>             String tmpHqlPath = null;
>             StringBuilder hql = new StringBuilder();
>             try {
>                 tmpHqlPath = "/tmp/" + System.currentTimeMillis() + ".hql";
>                 for (String statement : statements) {
>                     hql.append(statement);
>                     hql.append("\n");
>                 }
>                 String createFileCmd = String.format(CREATE_HQL_TMP_FILE_TEMPLATE, tmpHqlPath,
hql);
>                 buf.append(createFileCmd);
>                 buf.append("\n");
>                 buf.append(beelineShell);
>                 buf.append(" ");
>                 buf.append(beelineParams);
>                 buf.append(parseProps());
>                 buf.append(" -f ");
>                 buf.append(tmpHqlPath);
>                 buf.append(";ret_code=$?;rm -f ");
>                 buf.append(tmpHqlPath);
>                 buf.append(";exit $ret_code");
>             } finally {
>                 if (tmpHqlPath != null && logger.isDebugEnabled()) {
>                     logger.debug("The SQL to execute in beeline: \n" + hql);
>                 }
>             }
>             break;
> {code}
> A temp hql file will be created when creating flat table, the file name is the current
time stamp. But if two jobs triggered with same time, there will be only one file, and jobs
may be failed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message