kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: Error - Step Name: Load HFile to HBase Table
Date Sat, 04 Nov 2017 00:12:29 GMT
Hi Vikash,

When Kylin doing the bulkload, did you see HBase has any difficulty to
upload or transfer the files?

What's the size of the HFile? They can be checked on HDFS, in the output
folder of previous "Convert to HFile" step.

Besides, is your HBase using the same HDFS as the Hadoop MR?

Were you able to build a small cube, for example, the sample cube of Kylin?
If sample cube is okay, that means the environment should be okay. You can
take a try if haven't.

Please provide more info for investigation, thanks.

2017-11-04 1:16 GMT+08:00 <Vikash.Tandon@barclaycard.co.uk>:

> Hi,
>
>
>
> Please can I get help / steps to resolve Step 10 issue while building the
> cube.
>
> I checked Hbase health and all looks good. Log is not giving me any
> insight about error “common.HadoopShellExecutable:64 : error execute
> HadoopShellExecutable{id=16ceb8e6-84cd-4163-b9bc-ad6e3ad87fa8-10,
> name=Load HFile to HBase Table”.
>
>
>
> Am I missing something obvious to check?
>
>
>
> Please comment.
>
>
>
> Regards,
>
> Vikas
>
>
>
> *From:* Tandon, Vikash : Barclaycard
> *Sent:* 03 November 2017 11:02
> *To:* user@kylin.apache.org
> *Cc:* shaofengshi@apache.org
> *Subject:* RE: Error - Step Name: Load HFile to HBase Table
>
>
>
> Hi ShaoFeng,
>
>
>
> Below are details.
>
>
>
> ·         Hadoop Version | CDH 5.11.1
>
> ·         HBase health status | Looks good
>
> ·         hbase-site.xml is on Kylin's classpath.
>
>
>
> It’s keep failing on Step 10 i.e. where Load HFile to Hbase table.
>
> Note – Step 1 and Step 2 are quite heavy in size i.e. 2 GB and 6 GB but I
> am not sure whether it’s linked with this issue.
>
>
>
> Please let me know if need any other details.
>
>
>
> Thanks,
>
> Vikas
>
>
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* 02 November 2017 12:52
> *To:* user
> *Subject:* Re: Error - Step Name: Load HFile to HBase Table
>
>
>
> This mail originated from outside our organisation -
> shaofengshi@apache.org
>
> Hi Vikash,
>
>
>
> What's your Hadoop version? The bulk load step usually is very fast. If
> HBase is running well, it may indicate Kylin wasn't able to connect with
> HBase; Please check HBase healthy status, and ensure the proper
> hbase-site.xml is on Kylin's classpath (put it to $KYLIN_HOME/conf will
> ensure it can be read).
>
>
>
> Besides, check HBase master and region server's log to see whether there
> is anything abnormal. Hope this can help.
>
>
>
> 2017-11-02 19:15 GMT+08:00 <Vikash.Tandon@barclaycard.co.uk>:
>
> Hi Roberto,
>
>
>
> Thanks for your note.
>
>
>
> I can see hbase is running and Kylin is installed on the same machine as
> Hbase.
>
> I can’t find anything on Kylin log. Can you point me what exactly I need
> to see in log.
>
>
>
> Also, what you did to correct this issue.
>
>
>
> Thanks
>
>
>
> *From:* Roberto Tardío [mailto:roberto.tardio@stratebi.com]
> *Sent:* 01 November 2017 18:13
> *To:* user@kylin.apache.org
> *Subject:* Re: Error - Step Name: Load HFile to HBase Table
>
>
>
> This mail originated from outside our organisation -
> roberto.tardio@stratebi.com
>
> Hi Vikash,
>
> Have you checked if Hbase is running? Kylin is instaled on the same
> machine of Hbase? Please give more info and Kylin.log in order to help you
> better.
>
> Some days ago we have a similar issue due to Hbase health and
> configuration.
>
> Regards,
>
>
>
> El 01/11/2017 a las 18:27, Vikash.Tandon@barclaycard.co.uk escribió:
>
> Hi,
> I am trying to build cube and it failed on Step 11 Step Name: Load HFile
> to HBase Table.
> Below is log. please can you advise how we can correct it.
>
> I am on Kylin 2.1
> =====================================
> java.io.IOException: Retry attempted 10 times without completing, bailing
> out
>         at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.
> doBulkLoad(LoadIncrementalHFiles.java:446)
>         at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.
> doBulkLoad(LoadIncrementalHFiles.java:339)
>         at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(
> LoadIncrementalHFiles.java:1063)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>         at org.apache.kylin.storage.hbase.steps.BulkLoadJob.run(
> BulkLoadJob.java:77)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>         at org.apache.kylin.engine.mr.common.HadoopShellExecutable.
> doWork(HadoopShellExecutable.java:62)
>         at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:125)
>         at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(
> DefaultChainedExecutable.java:65)
>         at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:125)
>         at org.apache.kylin.job.impl.threadpool.DefaultScheduler$
> JobRunner.run(DefaultScheduler.java:141)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
>
> result code:2
>
>
>
>
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it from your
> system. Do not copy, use, disclose or otherwise act upon any part of this
> email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays Group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays Group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays Group is personal to the
> sender and is not given or endorsed by the Barclays Group.
>
> Barclaycard is a trading name of Barclays Bank PLC. Registered in England.
> Registered No: 1026167. Registered Office: 1 Churchill Place, London, E14
> 5HP. Barclays Bank PLC is authorised by the Prudential Regulation Authority
> and regulated by the Financial Conduct Authority and the Prudential
> Regulation Authority (Financial Services Register No. 122702).
>
>
>
> --
>
> *Roberto Tardío Olmos*
>
> *Senior Big Data & Business Intelligence Consultant*
>
> Avenida de Brasil, 17
> <https://maps.google.com/?q=Avenida+de+Brasil,+17&entry=gmail&source=g>,
> Planta 16.28020 Madrid
>
> Fijo: 91.788.34.10
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it from your
> system. Do not copy, use, disclose or otherwise act upon any part of this
> email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays Group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays Group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays Group is personal to the
> sender and is not given or endorsed by the Barclays Group.
>
> Barclaycard is a trading name of Barclays Bank PLC. Registered in England.
> Registered No: 1026167. Registered Office: 1 Churchill Place, London, E14
> 5HP. Barclays Bank PLC is authorised by the Prudential Regulation Authority
> and regulated by the Financial Conduct Authority and the Prudential
> Regulation Authority (Financial Services Register No. 122702).
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi 史少锋
>
>
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it from your
> system. Do not copy, use, disclose or otherwise act upon any part of this
> email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays Group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays Group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays Group is personal to the
> sender and is not given or endorsed by the Barclays Group.
>
> Barclaycard is a trading name of Barclays Bank PLC. Registered in England.
> Registered No: 1026167. Registered Office: 1 Churchill Place, London, E14
> 5HP. Barclays Bank PLC is authorised by the Prudential Regulation Authority
> and regulated by the Financial Conduct Authority and the Prudential
> Regulation Authority (Financial Services Register No. 122702).
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it from your
> system. Do not copy, use, disclose or otherwise act upon any part of this
> email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays Group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays Group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays Group is personal to the
> sender and is not given or endorsed by the Barclays Group.
>
> Barclaycard is a trading name of Barclays Bank PLC. Registered in England.
> Registered No: 1026167. Registered Office: 1 Churchill Place, London, E14
> 5HP. Barclays Bank PLC is authorised by the Prudential Regulation Authority
> and regulated by the Financial Conduct Authority and the Prudential
> Regulation Authority (Financial Services Register No. 122702).
>



-- 
Best regards,

Shaofeng Shi 史少锋

Mime
View raw message