Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D9F36200CBA for ; Mon, 3 Jul 2017 11:02:01 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id D857D160BEC; Mon, 3 Jul 2017 09:02:01 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2A1EB160BE4 for ; Mon, 3 Jul 2017 11:02:01 +0200 (CEST) Received: (qmail 36844 invoked by uid 500); 3 Jul 2017 09:02:00 -0000 Mailing-List: contact user-help@kylin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@kylin.apache.org Delivered-To: mailing list user@kylin.apache.org Received: (qmail 36834 invoked by uid 99); 3 Jul 2017 09:02:00 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jul 2017 09:02:00 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id C3FFEC0915 for ; Mon, 3 Jul 2017 09:01:59 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=6.31 tests=[RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id H6ID_rw_k0n1 for ; Mon, 3 Jul 2017 09:01:55 +0000 (UTC) Received: from out20-85.mail.aliyun.com (out20-85.mail.aliyun.com [115.124.20.85]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id D51B95F341 for ; Mon, 3 Jul 2017 09:01:54 +0000 (UTC) X-Alimail-AntiSpam: AC=CONTINUE;BC=0.07449293|-1;FP=12095126679044549418|1|1|1|0|-1|-1|-1;HT=e01l10421;MF=zhx@czkj1010.com;NM=1;PH=DS;RN=1;RT=1;SR=0;TI=SMTPD_---.8L.yiCJ_1499072505; Received: from 192.168.3.233(mailfrom:zhx@czkj1010.com ip:114.241.197.98) by smtp.aliyun-inc.com(10.147.41.187); Mon, 03 Jul 2017 17:01:45 +0800 From: =?gb2312?B?1cXRpA==?= Content-Type: text/plain; charset=gb2312 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: StorageCleanup error Message-Id: <84F441A5-7EF5-4342-904B-6F537208D779@czkj1010.com> Date: Mon, 3 Jul 2017 17:01:45 +0800 To: user@kylin.apache.org X-Mailer: Apple Mail (2.3273) archived-at: Mon, 03 Jul 2017 09:02:02 -0000 Hi My test hadoop storage is almost full, so I use StorageCleanupJob to = clear the unused data, but got the following error, please help analyze = it and give me some suggestions, My kylin has upgraded from 1.6 to 2.0 = now. Thanks. 2017-07-03 16:33:36,111 DEBUG [main MetadataManager:384]: Loaded 36 = SourceTable EXD(s) 2017-07-03 16:33:36,111 DEBUG [main MetadataManager:529]: Reloading = DataModel from folder = kylin_metadata(key=3D'/model_desc')@kylin_metadata@hbase 2017-07-03 16:33:36,184 ERROR [main MetadataManager:538]: Error to load = DataModel at /model_desc/test_model.json java.lang.IllegalStateException: Error to load = /model_desc/test_model.json at = org.apache.kylin.metadata.MetadataManager.reloadDataModelDescAt(MetadataMa= nager.java:558) at = org.apache.kylin.metadata.MetadataManager.reloadAllDataModel(MetadataManag= er.java:536) at = org.apache.kylin.metadata.MetadataManager.init(MetadataManager.java:284) at = org.apache.kylin.metadata.MetadataManager.(MetadataManager.java:113)= at = org.apache.kylin.metadata.MetadataManager.getInstance(MetadataManager.java= :83) at = org.apache.kylin.job.dao.ExecutableDao.(ExecutableDao.java:76) at = org.apache.kylin.job.dao.ExecutableDao.getInstance(ExecutableDao.java:57) at = org.apache.kylin.job.execution.ExecutableManager.(ExecutableManager.= java:55) at = org.apache.kylin.job.execution.ExecutableManager.getInstance(ExecutableMan= ager.java:64) at = org.apache.kylin.storage.hbase.util.StorageCleanupJob.(StorageClea= nupJob.java:78) Caused by: java.lang.IllegalArgumentException: Column not found by = KYLIN_SALES.USER_ID at = org.apache.kylin.metadata.model.DataModelDesc.findColumn(DataModelDesc.jav= a:240) at = org.apache.kylin.metadata.model.ModelDimensionDesc.init(ModelDimensionDesc= .java:66) at = org.apache.kylin.metadata.model.DataModelDesc.initDimensionsAndMetrics(Dat= aModelDesc.java:381) at = org.apache.kylin.metadata.model.DataModelDesc.init(DataModelDesc.java:302)= at = org.apache.kylin.metadata.MetadataManager.reloadDataModelDescAt(MetadataMa= nager.java:554) ... 9 more 2017-07-03 16:33:36,194 DEBUG [main MetadataManager:543]: Loaded 8 = DataModel(s) =A1=AD=A1=AD 2017-07-03 16:33:43,335 INFO [main StorageCleanupJob:283]: Checking = table OK 2017-07-03 16:33:43,335 INFO [main StorageCleanupJob:283]: Checking = table Time taken: 1.566 seconds 2017-07-03 16:33:43,335 INFO [main StorageCleanupJob:283]: Checking = table OK 2017-07-03 16:33:43,335 INFO [main StorageCleanupJob:283]: Checking = table = kylin_intermediate_cube_dim_order_one_6b1de133_a10b_4bb6_86a7_f249e6d0c418= Exception in thread "main" java.lang.RuntimeException: error execute = org.apache.kylin.storage.hbase.util.StorageCleanupJob at = org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplicati= on.java:42) at = org.apache.kylin.storage.hbase.util.StorageCleanupJob.main(StorageCleanupJ= ob.java:362) Caused by: java.lang.NullPointerException at = org.apache.kylin.common.util.ClassUtil.forRenamedClass(ClassUtil.java:84) at = org.apache.kylin.common.util.ClassUtil.forName(ClassUtil.java:76) at = org.apache.kylin.job.execution.ExecutableManager.parseTo(ExecutableManager= .java:439) at = org.apache.kylin.job.execution.ExecutableManager.getJob(ExecutableManager.= java:129) at = org.apache.kylin.storage.hbase.util.StorageCleanupJob.isTableInUse(Storage= CleanupJob.java:347) at = org.apache.kylin.storage.hbase.util.StorageCleanupJob.cleanUnusedIntermedi= ateHiveTable(StorageCleanupJob.java:302) at = org.apache.kylin.storage.hbase.util.StorageCleanupJob.execute(StorageClean= upJob.java:153) at = org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplicati= on.java:37) ... 1 more 2017-07-03 16:33:43,342 INFO [Thread-0 = ConnectionManager$HConnectionImplementation:2259]: Closing master = protocol: MasterService 2017-07-03 16:33:43,368 INFO [Thread-0 = ConnectionManager$HConnectionImplementation:1830]: Closing zookeeper = sessionid=3D0x25cbe7e555d9dc5 2017-07-03 16:33:43,371 INFO [Thread-0 ZooKeeper:684]: Session: = 0x25cbe7e555d9dc5 closed 2017-07-03 16:33:43,371 INFO [main-EventThread ClientCnxn:512]: = EventThread shut down Best regards.