kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From roger shi <rogershijich...@hotmail.com>
Subject 答复: 答复: 答复: Kylin Build Cube failing with the following error
Date Fri, 14 Apr 2017 09:57:40 GMT
Please follow the link to generate diagnose package. It helps to identify the root cause.


http://kylin.apache.org/blog/2016/06/10/diagnosis-tool/

Apache Kylin | Diagnosis Tool Introduction<http://kylin.apache.org/blog/2016/06/10/diagnosis-tool/>
kylin.apache.org
Diagnosis Tool Introduction. Jun 10, 2016 • Dong Li. Introduction. Since Apache Kylin 1.5.2,
there’s a diagnosis tool on Web UI, which aims to help Kylin admins ...




________________________________
发件人: ashishjainm <ashishjainm@gmail.com>
发送时间: 2017年4月14日 17:39:04
收件人: dev@kylin.apache.org
主题: Re: 答复: 答复: Kylin Build Cube failing with the following error

any suggestions on the changes of the properties and corresponding values.
also, the sample cube with Kylin builds without any problems. then why this
small cube build fails again and again, beats me.

On Fri, Apr 14, 2017 at 2:21 AM, Roger Shi [via Apache Kylin] <
ml-node+s74782n7656h93@n6.nabble.com> wrote:

> Yarn resource allocation is not that enough in CDH quickstart version 5.8.
> Here "enough" doesn't only refer to total resource, but also map and reduce
> memory configuration. You could have a try to configure them properly.
> ________________________________
> 发件人: ashishjainm <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=7656&i=0>>
> 发送时间: 2017年4月14日 16:44:36
> 收件人: [hidden email] <http:///user/SendEmail.jtp?type=node&node=7656&i=1>
> 主题: Re: 答复: Kylin Build Cube failing with the following error
>
> Thanks Roger.
> Below step is failing:
> #3 Step Name: Extract Fact Table Distinct Columns
> Duration: 1.62 mins Waiting: 23 seconds
>
> I am running on CDH distribution quickstart vm version 5.8
>
> On Thu, Apr 13, 2017 at 11:57 PM, Roger Shi [via Apache Kylin] <
> [hidden email] <http:///user/SendEmail.jtp?type=node&node=7656&i=2>>
> wrote:
>
> > Which step throws this error? From the log, it seems hadoop is opening
> > local file. Please check the default FileSystem of HDFS and also check
> the
> > hadoop job's memory configuration. There might be some hint.
> > ________________________________
> > 发件人: ashishjainm <[hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=7647&i=0>>
> > 发送时间: 2017年4月14日 13:53:58
> > 收件人: [hidden email] <http:///user/SendEmail.jtp?type=node&node=7647&i=1>
>
> > 主题: Kylin Build Cube failing with the following error
> >
> > Hello Team,
> >
> > I am new to Kylin and just created the sample cube. however, when i
> create
> > a
> > model with my own data and this is a very small data set with 10 rows in
> > the
> > fact table. The process is failing with the following error message. I
> am
> > using the CDH quickstart VM.
> >
> > May be this is a Yarn related issue and i wanted to know an optimum
> > configuration that i need to set for me to be able to build a sample
> cube
> > of
> > my own.
> >
> > 82252_0024_01_000003/meta/cube_desc
> > 2017-04-13 22:42:34,902 INFO [main]
> > org.apache.kylin.metadata.project.ProjectManager: Initializing
> > ProjectManager with metadata url
> > /yarn/nm/usercache/root/appcache/application_1492137382252_0024/container_
>
> > 1492137382252_0024_01_000003/meta
> > 2017-04-13 22:42:35,008 INFO [main]
> > org.apache.kylin.measure.MeasureTypeFactory: Checking custom measure
> > types
> > from kylin config
> > 2017-04-13 22:42:35,011 INFO [main]
> > org.apache.kylin.measure.MeasureTypeFactory: registering
> > COUNT_DISTINCT(hllc), class
> > org.apache.kylin.measure.hllc.HLLCMeasureType$Factory
> > 2017-04-13 22:42:35,028 INFO [main]
> > org.apache.kylin.measure.MeasureTypeFactory: registering
> > COUNT_DISTINCT(bitmap), class
> > org.apache.kylin.measure.bitmap.BitmapMeasureType$Factory
> > 2017-04-13 22:42:35,038 INFO [main]
> > org.apache.kylin.measure.MeasureTypeFactory: registering TOP_N(topn),
> > class
> > org.apache.kylin.measure.topn.TopNMeasureType$Factory
> > 2017-04-13 22:42:35,042 INFO [main]
> > org.apache.kylin.measure.MeasureTypeFactory: registering RAW(raw),
> class
> > org.apache.kylin.measure.raw.RawMeasureType$Factory
> > 2017-04-13 22:42:35,046 INFO [main]
> > org.apache.kylin.measure.MeasureTypeFactory: registering
> > EXTENDED_COLUMN(extendedcolumn), class
> > org.apache.kylin.measure.extendedcolumn.ExtendedColumnMeasureType$Factory
>
> > 2017-04-13 22:42:35,049 INFO [main]
> > org.apache.kylin.measure.MeasureTypeFactory: registering
> > PERCENTILE(percentile), class
> > org.apache.kylin.measure.percentile.PercentileMeasureType$Factory
> > 2017-04-13 22:42:35,155 INFO [main] org.apache.kylin.cube.CubeDescManager:
>
> >
> > Loaded 1 Cube(s)
> > 2017-04-13 22:42:35,159 INFO [main] org.apache.kylin.cube.CubeManager:
> > Reloaded cube spend_cube being CUBE[name=spend_cube] having 1 segments
> > 2017-04-13 22:42:35,159 INFO [main] org.apache.kylin.cube.CubeManager:
> > Loaded 1 cubes, fail on 0 cubes
> > 2017-04-13 22:42:38,384 INFO [main] org.apache.hadoop.mapred.MapTask:
> > Starting flush of map output
> > 2017-04-13 22:42:41,220 FATAL [main] org.apache.hadoop.mapred.YarnChild:
>
> > Error running child : java.lang.OutOfMemoryError: GC overhead limit
> > exceeded
> >         at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300)
>
> >
> >         at java.lang.StringCoding.encode(StringCoding.java:344)
> >         at java.lang.String.getBytes(String.java:916)
> >         at java.io.FileOutputStream.open(Native Method)
> >         at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> >         at
> > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:216)
>
> >
> >         at
> > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:209)
>
> >
> >         at
> > org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:307)
>
> >
> >         at
> > org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:295)
>
> >
> >         at
> > org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:281)
>
> >
> >         at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:901)
> >         at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:863)
> >         at
> > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1843)
>
> >
> >         at
> > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1508)
>
> >         at
> > org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:723)
>
> >
> >         at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
>
> >
> >         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
>
> >
> >         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at javax.security.auth.Subject.doAs(Subject.java:415)
> >         at
> > org.apache.hadoop.security.UserGroupInformation.doAs(
> > UserGroupInformation.java:1693)
> >         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> >
> > 2017-04-13 22:42:41,332 INFO [main]
> > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask
> > metrics
> > system...
> > 2017-04-13 22:42:41,333 INFO [main]
> > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics
> system
> > stopped.
> > 2017-04-13 22:42:41,333 INFO [main]
> > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics
> system
> > shutdown complete.
> >
> > --
> > View this message in context: http://apache-kylin.74782.x6.
> > nabble.com/Kylin-Build-Cube-failing-with-the-following-error-tp7646.html
> > Sent from the Apache Kylin mailing list archive at Nabble.com.
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> > http://apache-kylin.74782.x6.nabble.com/Kylin-Build-Cube-
> > failing-with-the-following-error-tp7646p7647.html
> > To unsubscribe from Kylin Build Cube failing with the following error,
> click
> > here
> > <
> > .
> > NAML
> > <http://apache-kylin.74782.x6.nabble.com/template/
> NamlServlet.jtp?macro=macro_viewer&id=instant_html%
> 21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> BasicNamespace-nabble.view.web.template.NabbleNamespace-
> nabble.view.web.template.NodeNamespace&breadcrumbs=
> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> >
>
>
>
> --
>
>
>
> [image: --]
>
> Ashish Jain
> [image: http://]about.me/ashishjainm
> <http://about.me/ashishjainm?promo=email_sig>
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Kylin-Build-Cube-failing-with-the-following-
> error-tp7646p7653.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-kylin.74782.x6.nabble.com/Kylin-Build-Cube-
> failing-with-the-following-error-tp7646p7656.html
> To unsubscribe from Kylin Build Cube failing with the following error, click
> here
> <http://apache-kylin.74782.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=7646&code=YXNoaXNoamFpbm1AZ21haWwuY29tfDc2NDZ8MTk3MTk2MjI2>
> .
> NAML
> <http://apache-kylin.74782.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>



--



[image: --]

Ashish Jain
[image: http://]about.me/ashishjainm
<http://about.me/ashishjainm?promo=email_sig>


--
View this message in context: http://apache-kylin.74782.x6.nabble.com/Kylin-Build-Cube-failing-with-the-following-error-tp7646p7658.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message