kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Yang <liy...@apache.org>
Subject Re: Nullpointer Exception in Kylin while checking the status of cube
Date Fri, 25 Dec 2015 02:19:52 GMT
It's likely that Kylin did extract from hive but pulled 0 records out of
it. Click the 'parameters' icon (like a key) in the first job step "Create
Intermediate Flat Hive Table" and find the exact command that Kylin used to
extract from hive. Like

.....
CREATE EXTERNAL TABLE IF NOT EXISTS
kylin_intermediate_xxxxx_20150101000000_20150927000000
.....
INSERT OVERWRITE TABLE
kylin_intermediate_xxxxx_20150101000000_20150927000000 SELECT ....
.....

Double check the command does return some records.


On Thu, Dec 17, 2015 at 2:43 PM, Mehak Mehta <mehak.mehta@stonybrook.edu>
wrote:

> Hi Li,
>
> Again thanks for your reply. I resolved that issue and I am able to build
> the cube.
> But after completion of building it. After building it my Cube size and
> Source file is shown as 0.
> As shown below.
>
> [image: Inline image 1]
>
> How can I check/debug this problem.
> Can you help me with this.
>
> I have checked logs and Kylin is retrieving data from Hive tables.
>
> Thanks & Regards,
> Mehak
>
>
>
> On Mon, Dec 14, 2015 at 3:29 AM, Li Yang <liyang@apache.org> wrote:
>
>> Pls check https://issues.apache.org/jira/browse/KYLIN-953
>>
>>
>> On Fri, Dec 11, 2015 at 8:25 PM, Mehak Mehta <mehak.mehta@stonybrook.edu>
>> wrote:
>>
>>> Hi,
>>>
>>> Thanks Li Yang for your reply.
>>>
>>> I solved that previous issue.
>>> Now my cube is building but it always fails at 70% with the following
>>> error:
>>>
>>> 2015-12-11 07:12:37,021 INFO  [pool-4-thread-1]
>>> mapreduce.HFileOutputFormat2: bulkload locality sensitive enabled
>>> 2015-12-11 07:12:37,021 INFO  [pool-4-thread-1]
>>> mapreduce.HFileOutputFormat2: Looking up current regions for table
>>> KYLIN_23ZG714AUU
>>> 2015-12-11 07:12:37,029 INFO  [pool-4-thread-1]
>>> mapreduce.HFileOutputFormat2: Configuring 1 reduce partitions to match
>>> current region count
>>> [pool-4-thread-1]:[2015-12-11 07:12:37,030][ERROR]*[org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:95)]
>>> - error in CubeHFileJob*
>>> *java.lang.IllegalArgumentException: Can not create a Path from a null
>>> string*
>>> * at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
>>> * at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
>>> * at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
>>> * at
>>> org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
>>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
>>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
>>> * at
>>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>>> * at java.lang.Thread.run(Thread.java:745)*
>>> *usage: CubeHFileJob*
>>> * -cubename <name>            Cube name. For exmaple, flat_item_cube*
>>> * -htablename <htable name>   HTable name*
>>> * -input <path>               Input path*
>>> * -jobname <name>             Job name. For exmaple,*
>>> *                             Kylin_Cuboid_Builder-clsfd_v2_Step_22-D)*
>>>  -output <path>              Output path
>>> [pool-4-thread-1]:[2015-12-11
>>> 07:12:37,034][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.cleanupTempConfFile(AbstractHadoopJob.java:340)]
>>> - tempMetaFileString is :
>>> file:///data/shared/mehak/hadoop/kylin/./bin/../tomcat/temp/kylin_job_meta4467707123004651766/meta
>>> *[pool-4-thread-1]:[2015-12-11
>>> 07:12:37,040][ERROR][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:116)]
>>> - error execute
>>> MapReduceExecutable{id=dbf394d0-5a18-4299-b191-c7bc0dc9b941-09,
>>> name=Convert Cuboid Data to HFile, state=RUNNING}*
>>> *java.lang.IllegalArgumentException: Can not create a Path from a null
>>> string*
>>> * at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
>>> * at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
>>> * at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
>>> * at
>>> org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
>>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
>>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
>>> * at
>>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>>> * at java.lang.Thread.run(Thread.java:745)*
>>> [pool-4-thread-1]:[2015-12-11
>>> 07:12:37,045][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>> - Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
>>> (Store kylin_metadata@hbase)
>>> [pool-4-thread-1]:[2015-12-11
>>> 07:12:37,051][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>> - Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
>>> (Store kylin_metadata@hbase)
>>> [pool-4-thread-1]:[2015-12-11
>>> 07:12:37,053][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)]
>>> - job id:dbf394d0-5a18-4299-b191-c7bc0dc9b941-09 from RUNNING to ERROR
>>>
>>> I cannot see any errors in hadoop, hive or hbase logs. It seems to be
>>> some configuration issue.
>>> Kindly help, I have tried a lot configurations but it doesn't work.
>>>
>>> Thanks & Regards,
>>> Mehak Mehta
>>>
>>>
>>>
>>> On Thu, Dec 10, 2015 at 4:42 AM, Li Yang <liyang@apache.org> wrote:
>>>
>>>> Kylin depends on a hadoop config "yarn.resourcemanager.webapp.address"
>>>> to find resource manager and come up a ws URL to query job status.
>>>>
>>>> You can check if the config presents in site xmls on Kylin classpath.
>>>>
>>>> Or if troubleshooting Hadoop is too painful, a simpler way is adding a
>>>> line in conf/kylin.properties.
>>>>
>>>> kylin.job.yarn.app.rest.check.status.url=
>>>> https://YOUR_RM_AND_PORT/ws/v1/cluster/apps/${job_id}?anonymous=true
>>>>
>>>> On Tue, Dec 8, 2015 at 11:46 AM, Mehak Mehta <
>>>> mehak.mehta@stonybrook.edu> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am getting the following error while building my cube using browser
>>>>> console on my Hivetable.
>>>>> I am using default architecture
>>>>>
>>>>> hadoop-2.6.0 (cluster of 10 nodes)
>>>>> apache-hive-0.14.0 (using default metastore)
>>>>> apache-kylin-1.1.1 (on single node)
>>>>> hbase-0.98.16 (on single node as pseudo cluster)
>>>>>
>>>>> Note I have recompiled hadoop (with Snappy library) for my system
>>>>> (Centos 6.5 64 bit).
>>>>> It starts building the cube but fails while checking its status with
>>>>> the following *Kylin logs,* can you please help me:
>>>>>
>>>>> 015-12-05 03:34:35,779 INFO  [Thread-36] mapred.LocalJobRunner:
>>>>> OutputCommitter is
>>>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
>>>>> 2015-12-05 03:34:35,848 INFO  [Thread-36] mapred.LocalJobRunner:
>>>>> Waiting for map tasks
>>>>> 2015-12-05 03:34:35,849 INFO  [LocalJobRunner Map Task Executor #0]
>>>>> mapred.LocalJobRunner: Starting task:
>>>>> attempt_local1931418873_0001_m_000000_0
>>>>> 2015-12-05 03:34:35,890 INFO  [LocalJobRunner Map Task Executor #0]
>>>>> mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
>>>>> 2015-12-05 03:34:35,897 INFO  [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: Processing split:
>>>>> org.apache.hive.hcatalog.mapreduce.HCatSplit@7d6447d
>>>>> 2015-12-05 03:34:35,921 INFO  [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: Map output collector class =
>>>>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer
>>>>> 2015-12-05 03:34:35,963 INFO  [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
>>>>> 2015-12-05 03:34:35,963 INFO  [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: mapreduce.task.io.sort.mb: 100
>>>>> 2015-12-05 03:34:35,963 INFO  [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: soft limit at 83886080
>>>>> 2015-12-05 03:34:35,963 INFO  [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: bufstart = 0; bufvoid = 104857600
>>>>> 2015-12-05 03:34:35,963 INFO  [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: kvstart = 26214396; length = 6553600
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:35,966][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)]
>>>>> - Going to buffer response body of large or unknown size. Using
>>>>> getResponseBodyAsStream instead is recommended.
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:35,967][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:92)]
>>>>> - Job job_local1931418873_0001 get status check result.
>>>>>
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:35,994][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)]
>>>>> - error check status
>>>>> *java.lang.NullPointerException*
>>>>> *        at
>>>>> org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)*
>>>>> *        at
>>>>> org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)*
>>>>> *        at
>>>>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:137)*
>>>>> *        at
>>>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>>>> *        at
>>>>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>>>>> *        at
>>>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>>>> *        at
>>>>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>>>>> *        at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>>>>> *        at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>>>>> *        at java.lang.Thread.run(Thread.java:745)*
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:35,998][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>>> (Store kylin_metadata@hbase)
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:36,003][DEBUG][org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:94)]
>>>>> - Counters: 0
>>>>> 2015-12-05 03:34:36,010 WARN  [pool-4-thread-2] mapreduce.Counters:
>>>>> Group FileSystemCounters is deprecated. Use
>>>>> org.apache.hadoop.mapreduce.FileSystemCounter instead
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:36,013][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>>> (Store kylin_metadata@hbase)
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:36,018][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>>> (Store kylin_metadata@hbase)
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:36,023][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>>> (Store kylin_metadata@hbase)
>>>>>
>>>>>
>>>>>
>>>>> I couldn't see any error in my Hive logs or Hadoop logs, but there
>>>>> were some logs related to this in *Hbase Logs:*
>>>>>
>>>>> 2015-12-05 03:29:43,208 DEBUG [region-location-2]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a
>>>>> *2015-12-05 03:29:43,208 DEBUG [region-location-2]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>>> *2015-12-05 03:30:01,382 INFO
>>>>>  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>>> server.NIOServerCnxnFactory: Accepted socket connection from
>>>>> /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>>>> *2015-12-05 03:30:01,387 INFO
>>>>>  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>>> server.ZooKeeperServer: Client attempting to establish new session at
>>>>> /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>>>> *2015-12-05 03:30:01,411 INFO  [SyncThread:0] server.ZooKeeperServer:
>>>>> Established session 0x15171398d090008 with negotiated timeout 40000 for
>>>>> client /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>>>> *2015-12-05 03:30:03,761 INFO
>>>>>  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>>> server.NIOServerCnxnFactory: Accepted socket connection from
>>>>> /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>>>> *2015-12-05 03:30:03,761 INFO
>>>>>  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>>> server.ZooKeeperServer: Client attempting to establish new session at
>>>>> /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>>>> *2015-12-05 03:30:03,763 INFO  [SyncThread:0] server.ZooKeeperServer:
>>>>> Established session 0x15171398d090009 with negotiated timeout 40000 for
>>>>> client /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>>>> *2015-12-05 03:30:03,816 INFO  [ProcessThread(sid:0 cport:-1):]
>>>>> server.PrepRequestProcessor: Got user-level KeeperException when processing
>>>>> sessionid:0x15171398d090009 type:create cxid:0x1 zxid:0x52d txntype:-1
>>>>> reqpath:n/a Error Path:/kylin/job_engine/lock/kylin_metadata
>>>>> Error:KeeperErrorCode = NoNode for /kylin/job_engine/lock/kylin_metadata*
>>>>> *2015-12-05 03:30:43,210 DEBUG [region-location-3]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>>>>> *2015-12-05 03:30:43,211 DEBUG [region-location-3]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>>>>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>>>
>>>>> But I couldn't figure out the cause of the problem, can anybody please
>>>>> help me in resolving this.
>>>>> I have been struggling with this for sometime now.
>>>>>
>>>>> --
>>>>> Thanks & Regards
>>>>> Mehak Mehta
>>>>> Masters - Computer Science
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks & Regards
>>> Mehak Mehta
>>> Masters - Computer Science
>>>
>>
>>
>
>
> --
> Thanks & Regards
> Mehak Mehta
> Masters - Computer Science
>

Mime
View raw message