kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: Apache Kylin - Hbase Cube Name Space
Date Thu, 23 Nov 2017 14:33:20 GMT
Hi Manoj,

This feature is not released yet, see:
https://issues.apache.org/jira/browse/KYLIN-2846
But the patch is available; you can build a new Kylin binary package by
your own with the patches.

2017-11-23 21:49 GMT+08:00 Kumar, Manoj H <manoj.h.kumar@jpmorgan.com>:

> Its pointing to Default name – How we can change the Habse Namespace so
> that we can build the code successfully. Pls. advise. It has failed at
> Create HT Table step.
>
>
>
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
> permissions (user=a_rcmo_nd@NAEAST.AD.JPMORGANCHASE.COM, scope=default,
> params=[namespace=*default*,table=default:KYLIN_
> S1W12SO8AC,family=F1],action=CREATE)
>
>
>
> Regards,
>
> Manoj
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* Wednesday, November 22, 2017 8:53 AM
> *To:* user
> *Subject:* Re: How to Update the Cube Structure
>
>
>
> The intermediate hive table will be cleaned in the last step "Hive
> Cleanup" automatically.
>
>
>
> If your job is failed in the between, then the last step won't be executed
> by Kylin. In this case, you can manually clean them with the the storage
> <https://kylin.apache.org/docs21/howto/howto_cleanup_storage.html> clean
> up cli <https://kylin.apache.org/docs21/howto/howto_cleanup_storage.html>.
>
>
>
> 2017-11-22 11:13 GMT+08:00 Kumar, Manoj H <manoj.h.kumar@jpmorgan.com>:
>
> Np Mind, I got confused with Cube’s Partition column. Once Cube is build
> out, Can we clean Hive Intermediate flat table? Also, How can we know which
> flat belongs particular cube? How to schedule the cube build process –
> taking incremental data from hive source table
>
>
>
> Regards,
>
> Manoj
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* Wednesday, November 22, 2017 6:52 AM
>
>
> *To:* user
> *Subject:* Re: How to Update the Cube Structure
>
>
>
> Hi Manoj,
>
>
>
> The intermediate flat table is not partitioned, always; it has all files
> in one folder; I'm wondering what's the purpose to make it partitioned?
>
>
>
> 2017-11-22 1:12 GMT+08:00 Kumar, Manoj H <manoj.h.kumar@jpmorgan.com>:
>
> Thanks… one quick question – How to make intermediate flat table
> partitioned one?? Do we have any parameter in kylin.properties?
>
>
>
> Regards,
>
> Manoj
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* Tuesday, November 21, 2017 8:02 PM
>
>
> *To:* user
> *Subject:* Re: How to Update the Cube Structure
>
>
>
> This document can help:
>
>
>
> https://kylin.apache.org/docs21/howto/howto_optimize_build.html
>
>
>
>
>
> 2017-11-21 21:36 GMT+08:00 Kumar, Manoj H <manoj.h.kumar@jpmorgan.com>:
>
> Thanks for the inputs.. It make sense..
>
>
>
> As now I am running on Dev Cluster & need to know optimize parameter for
> Kylin so that Cube building process runs fast. Can you pls. tell what are
> properties I need to change in kylin.properties file – To optimize the cube
> build process.. I have 180 millions of data on fact table.
>
>
>
> Regards,
>
> Manoj
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* Tuesday, November 21, 2017 3:23 PM
> *To:* user
> *Subject:* Re: How to Update the Cube Structure
>
>
>
> To be safe, you can clone the existing to a new cube, make changes on the
> new cube, and then rebuild it.
>
>
>
> 2017-11-21 14:53 GMT+08:00 Kumar, Manoj H <manoj.h.kumar@jpmorgan.com>:
>
> In case of Data Source table has DDL changes – either new Column added or
> Existing Column data type changed. How Can we do Cube data model update &
> reflecting new column data to Cube? Pls. advise.
>
>
>
> Regards,
>
> Manoj
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* Monday, November 20, 2017 7:03 PM
> *To:* user
> *Subject:* Re: Apache Kyine - kylin.env.hdfs-working-dir
>
>
>
> It seems that the script still try to create "/kylin" on HDFS, not in the
> configured path.
>
>
>
> Please double check and make sure this line is not commented:
>
>
>
> kylin.env.hdfs-working-dir=/tenants/rft/rcmo/kylin
>
>
>
> 2017-11-20 14:00 GMT+08:00 Kumar, Manoj H <manoj.h.kumar@jpmorgan.com>:
>
> I am installing kyline 2.10 on Dev Cluster(Cloudera) . Although I have
> given 777 permission on kylin Folder but still giving error. Can you pls.
> help on this? Whats missing here?
>
>
>
> -bash-4.1$ hdfs dfs -mkdir /tenants/rft/rcmo/kylin
>
> -bash-4.1$ hdfs dfs -chmod 777 /tenants/rft/rcmo/kylin
>
>
>
> Kyline properties.conf - #kylin.env.hdfs-working-dir=/
> tenants/rft/rcmo/kylin
>
>
>
> -bash-4.1$ ./check-env.sh
>
> Retrieving hadoop conf dir...
>
> KYLIN_HOME is set to /apps/rft/rcmo/apps/kylin/
> apache-kylin-2.1.0-bin-cdh57
>
> mkdir: Permission denied: user=a_rcmo_nd, access=WRITE,
> inode="/":hdfs:supergroup:drwxr-xr-x
>
> Failed to create /kylin. Please make sure the user has right to access
> /kylin
>
>
>
> Regards,
>
> Manoj
>
>
>
> This message is confidential and subject to terms at: http://
> www.jpmorgan.com/emaildisclaimer including on confidentiality, legal
> privilege, viruses and monitoring of electronic messages. If you are not
> the intended recipient, please delete this message and notify the sender
> immediately. Any unauthorized use is strictly prohibited.
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi 史少锋
>
>
>
> This message is confidential and subject to terms at: http://
> www.jpmorgan.com/emaildisclaimer including on confidentiality, legal
> privilege, viruses and monitoring of electronic messages. If you are not
> the intended recipient, please delete this message and notify the sender
> immediately. Any unauthorized use is strictly prohibited.
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi 史少锋
>
>
>
> This message is confidential and subject to terms at: http://
> www.jpmorgan.com/emaildisclaimer including on confidentiality, legal
> privilege, viruses and monitoring of electronic messages. If you are not
> the intended recipient, please delete this message and notify the sender
> immediately. Any unauthorized use is strictly prohibited.
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi 史少锋
>
>
>
> This message is confidential and subject to terms at: http://
> www.jpmorgan.com/emaildisclaimer including on confidentiality, legal
> privilege, viruses and monitoring of electronic messages. If you are not
> the intended recipient, please delete this message and notify the sender
> immediately. Any unauthorized use is strictly prohibited.
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi 史少锋
>
>
>
> This message is confidential and subject to terms at: http://
> www.jpmorgan.com/emaildisclaimer including on confidentiality, legal
> privilege, viruses and monitoring of electronic messages. If you are not
> the intended recipient, please delete this message and notify the sender
> immediately. Any unauthorized use is strictly prohibited.
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi 史少锋
>
>
>
> This message is confidential and subject to terms at: http://
> www.jpmorgan.com/emaildisclaimer including on confidentiality, legal
> privilege, viruses and monitoring of electronic messages. If you are not
> the intended recipient, please delete this message and notify the sender
> immediately. Any unauthorized use is strictly prohibited.
>



-- 
Best regards,

Shaofeng Shi 史少锋

Mime
View raw message