kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chenwen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KYLIN-3465) kylin Kafka stream incremental build error
Date Tue, 24 Jul 2018 06:42:00 GMT
chenwen created KYLIN-3465:
------------------------------

             Summary:  kylin Kafka stream incremental build error
                 Key: KYLIN-3465
                 URL: https://issues.apache.org/jira/browse/KYLIN-3465
             Project: Kylin
          Issue Type: Bug
          Components: Environment , Streaming
    Affects Versions: v2.4.0
            Reporter: chenwen
         Attachments: image-2018-07-24-14-34-38-102.png

h1. Development environment:
 * Centos 7
 * Hadoop 2.7.2
 * Hbase 1.2.5
 * Hive 1.2.2
 * Kylin-2.4.0-bin-hbase1x
 * Kafka 0.10.2.2

I built a cluster consisting of three machines that build the cube from the Kafka stream,
but each time a new incremental build is made, the following error is displayed.

 
{{2018-07-24 14:27:09,269 WARN [Scheduler 268704591 Job faf6db81-bc76-4b25-b838-8ca993e889d5-602]
config.KafkaConsumerProperties:142 : KYLIN_CONF property was not set, will seek KYLIN_HOME
env variable 2018-07-24 14:27:09,295 ERROR [Scheduler 268704591 Job faf6db81-bc76-4b25-b838-8ca993e889d5-602]
hadoop.KafkaFlatTableJob:131 : error in KafkaFlatTableJob java.lang.NullPointerException at
org.apache.kylin.source.kafka.hadoop.KafkaFlatTableJob.setupMapper(KafkaFlatTableJob.java:143)
at org.apache.kylin.source.kafka.hadoop.KafkaFlatTableJob.run(KafkaFlatTableJob.java:118)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:131)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:307)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 2018-07-24 14:27:09,296 ERROR [Scheduler 268704591
Job faf6db81-bc76-4b25-b838-8ca993e889d5-602] common.MapReduceExecutable:138 : error execute
MapReduceExecutable\{id=faf6db81-bc76-4b25-b838-8ca993e889d5-00, name=Save data from Kafka,
state=RUNNING} java.lang.NullPointerException at org.apache.kylin.source.kafka.hadoop.KafkaFlatTableJob.setupMapper(KafkaFlatTableJob.java:143)
at org.apache.kylin.source.kafka.hadoop.KafkaFlatTableJob.run(KafkaFlatTableJob.java:118)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:131)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:307)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 2018-07-24 14:27:09,302 INFO [Scheduler 268704591
Job faf6db81-bc76-4b25-b838-8ca993e889d5-602] execution.ExecutableManager:411 : job id:faf6db81-bc76-4b25-b838-8ca993e889d5-00
from RUNNING to ERROR 2018-07-24 14:27:09,310 INFO [Scheduler 268704591 Job faf6db81-bc76-4b25-b838-8ca993e889d5-602]
execution.ExecutableManager:411 : job id:faf6db81-bc76-4b25-b838-8ca993e889d5 from RUNNING
to ERROR 2018-07-24 14:27:09,310 DEBUG [Scheduler 268704591 Job faf6db81-bc76-4b25-b838-8ca993e889d5-602]
execution.AbstractExecutable:310 : no need to send email, user list is empty }}
My current solution is to re-start the service of the job node every time, and build the incremental
build again.
h1. Kylin.properties

kylin.server.mode=job
kylin.server.cluster-servers=99.48.1.2:7070,99.48.1.3:7070

kylin.server.mode=query
kylin.server.cluster-servers=99.48.1.2:7070,99.48.1.3:7070

kylin.server.mode=query
kylin.server.cluster-servers=99.48.1.2:7070,99.48.1.3:7070

99.48.1.1:7070 job
99.48.1.2:7070 query
99.48.1.3:7070 query

Thanks



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message