kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "George Ni (Chunen Ni)" <chunen...@kyligence.io>
Subject Re: kylin web UI become too slow since update to 2.0 beta
Date Fri, 10 Mar 2017 13:48:23 GMT
Hi Tianshuo,

I have tried to reproduce the case. I run kylin  2.0 beta in a standalone HDP, and made close
to 400 building jobs. However, listing the jobs is still quite fast with less than one second
waiting time.

Have you checked your hbase cluster’s status, is there any regionserver lost? Or is there
any network delay between your kylin server and Hadoop cluster?

If not, can you check kylin.log to see any error or exception after clicking the monitor button?

Best regards,
Thanks

Chun’en Ni(倪春恩)
Mail: chunen.ni@kyligence.io<mailto:kaige.liu@kyligence.io>
Shanghai Kyligence Information Technology Co., Ltd
上海市浦东新区亮秀路112号Y1座405室


发件人: hongbin ma <mahongbin@apache.org>
日期: 2017年3月10日 星期五 上午11:38
至: "George Ni (Chunen Ni)" <chunen.ni@kyligence.io>
主题: Fwd: Re: kylin web UI become too slow since update to 2.0 beta

chunen can you take a look?


---------- Forwarded message ----------
From: 赵天烁 <zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>>
Date: Thu, Mar 9, 2017 at 10:59 AM
Subject: Re: Re: kylin web UI become too slow since update to 2.0 beta
To: mahongbin <mahongbin@apache.org<mailto:mahongbin@apache.org>>


@Override
protected List<RawResource> getAllResourcesImpl(String folderPath, long timeStart, long
timeEndExclusive) throws IOException {
    FilterList filter = generateTimeFilterList(timeStart, timeEndExclusive);
    final List<RawResource> result = Lists.newArrayList();
    try {
        visitFolder(folderPath, filter, new FolderVisitor() {
            @Override
            public void visit(String childPath, String fullPath, Result hbaseResult) throws
IOException {
                // is a direct child (not grand child)?
                if (childPath.equals(fullPath))
                    result.add(new RawResource(getInputStream(childPath, hbaseResult), getTimestamp(hbaseResult)));
            }
        });
    } catch (IOException e) {
        for (RawResource rawResource : result) {
            IOUtils.closeQuietly(rawResource.inputStream);
        }
        throw e;
    }
    return result;
}
I think I found one of the stuck point,as those code above,this is HBaseResourceStore.getAllResourcesImpl
which is called by  jobService.listAllJobs. it list all resources then do a sublist in mem.In
my case, getAllresources will return like 500 records,which will take a minute. I just want
to see those recent jobs about 10~15record is enough,I think there is no need to get all jobs.
________________________________
赵天烁
Kevin Zhao
zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>

珠海市魅族科技有限公司
MEIZU Technology Co., Ltd.
广东省珠海市科技创新海岸魅族科技楼
MEIZU Tech Bldg., Technology & Innovation Coast
Zhuhai, 519085, Guangdong, China
meizu.com<http://meizu.com>

From: hongbin ma<mailto:mahongbin@apache.org>
Date: 2017-03-08 14:12
To: 赵天烁<mailto:zhaotianshuo@meizu.com>
Subject: Re: Re: kylin web UI become too slow since update to 2.0 beta
hi tianshuo

Since it's not a standard HDP/CHD environment, we'll have to establish such environment from
a clean linux. I suppose it will take some time for us to setup. If you have some tutorial
to help us setting up the env please send us.

On Wed, Mar 8, 2017 at 9:30 AM, 赵天烁 <zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>>
wrote:
hadoop version is apache 2.4.6,hive 1.2.1,hbase 1.2.1,these env are the same when we deploy
kylin 1.6.x,havn't came to this issue before update to 2.0

________________________________
赵天烁
Kevin Zhao
zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>

珠海市魅族科技有限公司
MEIZU Technology Co., Ltd.
广东省珠海市科技创新海岸魅族科技楼
MEIZU Tech Bldg., Technology & Innovation Coast
Zhuhai, 519085, Guangdong, China
meizu.com<http://meizu.com>

From: hongbin ma<mailto:mahongbin@apache.org>
Date: 2017-03-08 09:25
To: 赵天烁<mailto:zhaotianshuo@meizu.com>
Subject: Re: Re: kylin web UI become too slow since update to 2.0 beta
sorry i didn't ask clearly, what is your hadoop version? CDH, HDP or apache? we need to build
same env

On Wed, Mar 8, 2017 at 9:09 AM, 赵天烁 <zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>>
wrote:
any update on this issue?

________________________________
赵天烁
Kevin Zhao
zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>

珠海市魅族科技有限公司
MEIZU Technology Co., Ltd.
广东省珠海市科技创新海岸魅族科技楼
MEIZU Tech Bldg., Technology & Innovation Coast
Zhuhai, 519085, Guangdong, China
meizu.com<http://meizu.com>

From: hongbin ma<mailto:mahongbin@apache.org>
Date: 2017-03-07 11:42
To: user.kylin<mailto:user@kylin.apache.org>
CC: Guosheng Lu<mailto:guosheng.lu@kyligence.io>
Subject: Re: Re: kylin web UI become too slow since update to 2.0 beta
Hi tianshuo

Can you please show us the detailed steps to reproduce the issue?
for example, is this issue reproduecable in a brand new sandbox? what sandbox are you using?
 what's the CPU/memory settings?
Given the your sandbox settings, are you able to reproduce with our provided sample cube?

we want to reproduce the issue ASAP

On Tue, Mar 7, 2017 at 11:24 AM, 赵天烁 <zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>>
wrote:
[cid:image001.jpg@01D299E8.08ED2F70]
[cid:image002.jpg@01D299E8.08ED2F70]
crash msg:
{"code":"999","data":null,"msg":"java.lang.RuntimeException: org.apache.kylin.job.exception.PersistentException:
java.io<http://java.io>.IOException: Blocklist for /user/kylin/kylin_metadata/resources/execute_output/3a06ea90-ba4a-4098-b226-eee1bf85b233-02
has changed!","stacktrace":"org.apache.kylin.rest.exception.In<http://apache.kylin.rest.exception.In>ternalErrorException:
java.lang.RuntimeException: org.apache.kylin.job.exception.PersistentException: java.io<http://java.io>.IOException:
Blocklist for /user/kylin/kylin_metadata/resources/execute_output/3a06ea90-ba4a-4098-b226-eee1bf85b233-02
has changed!\n\tat org.apache.kylin.rest.controller.JobController.list(JobController.java:76)\n\tat
sun.reflect.GeneratedMethodAccessor135.invoke(Unknown Source)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
java.lang.reflect.Method.invoke(Method.java:606)\n\tat org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)\n\tat
org.springframework.web.method.support.InvocableHandlerMethod.in<http://port.InvocableHandlerMethod.in>vokeForRequest(InvocableHandlerMethod.java:136)\n\tat
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)\n\tat
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743)\n\tat
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672)\n\tat
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82)\n\tat
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933)\n\tat
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867)\n\tat
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951)\n\tat
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:842)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:622)\n\tat
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827)\n\tat
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)\n\tat
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)\n\tat
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)\n\tat
org.apache.cat<http://org.apache.cat>alina.core.ApplicationFilterChain.do<http://core.ApplicationFilterChain.do>Filter(ApplicationFilterChain.java:165)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)\n\tat
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)\n\tat
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.authentication.An<http://security.web.authentication.An>onymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)\n\tat
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)\n\tat
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)\n\tat
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)\n\tat
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343)\n\tat
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260)\n\tat
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)\n\tat
org.apache.catalina.core.ApplicationFilterChain.do<http://core.ApplicationFilterChain.do>Filter(ApplicationFilterChain.java:165)\n\tat
com.th<http://com.th>etransactioncompany.cors.CORSFilter.do<http://tioncompany.cors.CORSFilter.do>Filter(CORSFilter.java:209)\n\tat
com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)\n\tat
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)\n\tat
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)\n\tat
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)\n\tat org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:474)\n\tat
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)\n\tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)\n\tat
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)\n\tat
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)\n\tat
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783)\n\tat org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)\n\tat
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:789)\n\tat
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1437)\n\tat
org.apache.tomcat.util.net<http://org.apache.tomcat.util.net>.SocketProcessorBase.run(SocketProcessorBase.java:49)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused
by: java.lang.RuntimeException: org.apache.kylin.job.exception.PersistentException: java.io<http://java.io>.IOException:
Blocklist for /user/kylin/kylin_metadata/resources/execute_output/3a06ea90-ba4a-4098-b226-eee1bf85b233-02
has changed!\n\tat org.apache.kylin.job.execution.ExecutableManager.getAllOutputs(ExecutableManager.java:178)\n\tat
org.apache.kylin.rest.se<http://org.apache.kylin.rest.se>rvice.JobService.listCubeJobInstance(JobService.java:189)\n\tat
org.apache.kylin.rest.service.JobService.listAllJobs(JobService.java:162)\n\tat org.apache.kylin.rest.service.JobS<http://apache.kylin.rest.service.JobS>ervice.listAllJobs(JobService.java:144)\n\tat
org.apache.kylin.rest.service.JobService$$FastClassBySpringCGLIB$$83a44b2a.invoke(<generated>)\n\tat
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)\n\tat org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:629)\n\tat
org.apache.kylin.rest.service.JobService$$EnhancerBySpringCGLIB$$89d6b932.listAllJobs(<generated>)\n\tat
org.apache.kylin.rest.controller.JobController.list(JobController.java:73)\n\t... 72 more\nCaused
by: org.apache.kylin.job.exception.PersistentException: java.io<http://java.io>.IOException:
Blocklist for /user/kylin/kylin_metadata/resources/execute_output/3a06ea90-ba4a-4098-b226-eee1bf85b233-02
has changed!\n\tat org.apache.kylin.job.dao.ExecutableDao.ge<http://kylin.job.dao.ExecutableDao.ge>tJobOutputs(ExecutableDao.java:120)\n\tat
org.apache.kylin.job.execution.ExecutableManager.getAllOutputs(ExecutableManager.java:170)\n\t...
80 more\nCaused by: java.io<http://java.io>.IOException: Blocklist for /user/kylin/kylin_metadata/resources/execute_output/3a06ea90-ba4a-4098-b226-eee1bf85b233-02
has changed!\n\tat org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:277)\n\tat
org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:231)\n\tat org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:908)\n\tat
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:560)\n\tat org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:790)\n\tat
org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:837)\n\tat java.io.DataInputStream.re<http://java.io.DataInputStream.re>ad(DataInputStream.java:149)\n\tat
java.io<http://java.io>.DataInputStream.read(DataInputStream.java:149)\n\tat com.fasterxml.jackson.co<http://com.fasterxml.jackson.co>re.json.ByteSourceJsonBootstrapper.ensureLoaded(ByteSourceJsonBootstrapper.java:489)\n\tat
com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.detectEncoding(ByteSourceJsonBootstrapper.java:126)\n\tat
com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.constructParser(ByteSourceJsonBootstrapper.java:215)\n\tat
com.fasterxml.jackson.core.JsonFactory._createParser(JsonFactory.java:1240)\n\tat com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:802)\n\tat
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2796)\n\tat org.apache.kylin.common.util.JsonUtil.readValue(JsonUtil.java:62)\n\tat
org.apache.kylin.common.persistence.JsonSerializer.deserialize(JsonSerializer.java:40)\n\tat
org.apache.kylin.common.persistence.ResourceStore.getAllResources(ResourceStore.java:189)\n\tat
org.apache.kylin.job.dao.ExecutableDao.getJobOutputs(ExecutableDao.java:117)\n\t... 81 more\n","exception":"java.lang.RuntimeException:
org.apache.kylin.job.exception.PersistentException: java.io<http://java.io>.IOException:
Blocklist for /user/kylin/kylin_metadata/resources/execute_output/3a06ea90-ba4a-4098-b226-eee1bf85b233-02
has changed!","url":"http://10.3.141.83:7070/kylin/api/jobs"}

some time it success:
[cid:image003.jpg@01D299E8.08ED2F70]
________________________________
赵天烁
Kevin Zhao
zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>

珠海市魅族科技有限公司
MEIZU Technology Co., Ltd.
广东省珠海市科技创新海岸魅族科技楼
MEIZU Tech Bldg., Technology & Innovation Coast
Zhuhai, 519085, Guangdong, China
meizu.com<http://meizu.com>

From: Guosheng Lu<mailto:guosheng.lu@kyligence.io>
Date: 2017-03-07 10:58
To: 赵天烁<mailto:zhaotianshuo@meizu.com>
CC: user<mailto:user@kylin.apache.org>
Subject: Re: kylin web UI become too slow since update to 2.0 beta
Thx! Tianshuo:
We can not reproduce, so could you help us  to debug in your  chrome network  to watch if
 ajax  request  cost  you more  time !

Best Regards!
Bob Lu|陆国圣
http://kyligence.io

发件人: 赵天烁 <zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>>
日期: 2017年3月7日 星期二 上午10:44
至: luguosheng <guosheng.lu@kyligence.io<mailto:guosheng.lu@kyligence.io>>
抄送: user <user@kylin.apache.org<mailto:user@kylin.apache.org>>
主题: Re: Re: kylin web UI become too slow since update to 2.0 beta

1.chrome v46
2.4example when I create a new cube ,after go through all the step ,when I click save.....it
stuck,edit cube is the same thing.
another one is when I click the monitor tab ,or the refresh button in that tab. it will stuck
in loading jobs....
PS: when I call the  rest API to create cube or get job status,it also stuck just like when
I do the same thing in web UI
[cid:image004.jpg@01D299E8.08ED2F70]

________________________________
赵天烁
Kevin Zhao
zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>

珠海市魅族科技有限公司
MEIZU Technology Co., Ltd.
广东省珠海市科技创新海岸魅族科技楼
MEIZU Tech Bldg., Technology & Innovation Coast
Zhuhai, 519085, Guangdong, China
meizu.com<http://meizu.com>

From: Guosheng Lu<mailto:guosheng.lu@kyligence.io>
Date: 2017-03-07 10:18
To: 赵天烁<mailto:zhaotianshuo@meizu.com>
CC: user@kylin.apache.org<mailto:user@kylin.apache.org>
Subject: Re: kylin web UI become too slow since update to 2.0 beta
Hi ,
I want to know some details
1.which browser you use ?
2.and in what situation it becomes slow? form submit  or just when you open web page


Best Regards!
Bob Lu|陆国圣
http://kyligence.io

发件人: 赵天烁 <zhaotianshuo@meizu.com<mailto:zhaotianshuo@meizu.com>>
答复: "user@kylin.apache.org<mailto:user@kylin.apache.org>" <user@kylin.apache.org<mailto:user@kylin.apache.org>>
日期: 2017年3月7日 星期二 上午10:06
至: user <user@kylin.apache.org<mailto:user@kylin.apache.org>>
主题: kylin web UI become too slow since update to 2.0 beta

cube



--
Regards,

Bin Mahone | 马洪宾



--
Regards,

Bin Mahone | 马洪宾



--
Regards,

Bin Mahone | 马洪宾



--
Regards,

Bin Mahone | 马洪宾
Mime
View raw message