kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billy Liu <billy...@apache.org>
Subject Re: Auto Merge Not Working on Kylin 2.3.1
Date Tue, 10 Apr 2018 23:59:42 GMT
Hi Ketan,

As you mentioned, 7070 works but 7072 does not work. Could you check
if both instances are pointing to the same
"kylin.server.cluster-servers"

With Warm regards

Billy Liu


2018-04-10 23:41 GMT+08:00 ketan dikshit <kdcool6932@yahoo.com.invalid>:
> Hi Billy,
> Yes that’s what I did, I enabled remote debugging for the running instance and checked:
>
> kylin-2.3/kylin/server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java
> has method: updateOnNewSegmentReady(String cubeName) that actually invokes 2 methods
on the event
> keepCubeRetention(cubeName);
> mergeCubeSegment(cubeName);
> This method is actually invoked by a spring bean
> /kylin-2.3/kylin/server-base/src/main/java/org/apache/kylin/rest/service/CacheService.java
> via a Broadcaster.Listener cacheListener;
>
> That binds to the thread:
> @Override
> public void onEntityChange(Broadcaster broadcaster, String entity, Event event, String
cacheKey)
> throws IOException {
> logger.info("Initializing Event-BroadCast-Thread”); //was added by me, never reached
>     if ("cube".equals(entity) && event == Event.UPDATE) { //added a debug point
here, never reached
>         final String cubeName = cacheKey;
>         new Thread() { // do not block the event broadcast thread
>             public void run() {
>                 try {
>                     Thread.sleep(1000);
>                     cubeService.updateOnNewSegmentReady(cubeName);
>                 } catch (Throwable ex) {
>                     logger.error("Error in updateOnNewSegmentReady()", ex);
>                 }
>             }
>         }.start();
>     }
> }
> This method is not being triggered, I have added a debug point as well as logger in here,
neither logger.error("Error in updateOnNewSegmentReady()", ex) is happening;
> is shown in logs. So either this thread is becoming hung(hanging thread issue) or something
else;
>
> One thing that I would like to mention here is that, On the same node we are running
2 Kylin instances with exactly same code base, one on 7070(DEV) other on 7072(TEST) different
JVM’s of course(port was changed using kylin-replace-port-util).
>
> Important: the listener thread is getting fired on 7070(DEV) and the auto merge and retention
is working fine but not on 7072(TEST); Both instances are using exactly same kylin.properties
but pointing to different metadata of their own.
>
> If you have any insights that would be great, will spend some more time into debugging
this hanging thread issue.
>
> Thanks,
> Ketan@Exponential
>
>> On 10-Apr-2018, at 6:31 PM, Billy Liu <billyliu@apache.org> wrote:
>>
>> Hello Ketan
>>
>> If the merge is not triggered, there should be some logs saying why
>> the merge is ignored. That log is not exception or warn, but INFO you
>> what the system is expecting to do.
>>
>> With Warm regards
>>
>> Billy Liu
>>
>>
>> 2018-04-10 15:34 GMT+08:00 ketan dikshit <kdcool6932@yahoo.com.invalid>:
>>> Hi Team,
>>> I am working with Kylin 2.3.1 for last few days, have seen that Kylin auto-merge
functionality is not working.
>>> I have not set the volatile range(default=0), so should be default behaviour.
>>>
>>> Also not able to see any error in logs as well. So basically its not being triggered.
>>> Here is the commit_sha ID: 1e322992b8011d3b430321599d6da762c1a5e6b9
>>>
>>> Could you please point me how to debug or resolve the same, what could be the
potential errors/blockers in here ?
>>>
>>> And Here are my kylin.properties,(nothing special here)
>>>
>>> kylin.web.timezone=US/Pacific
>>> kylin.metadata.url=kylin2.3.1Meta@hbase
>>> kylin.storage.url=hbase
>>> kylin.env.hdfs-working-dir=/tmp/kylin-2.3.1-DEV
>>> kylin.engine.mr.reduce-input-mb=300
>>> kylin.server.mode=all
>>> kylin.env=TEST
>>> kylin.job.max-concurrent-jobs=10
>>> kylin.engine.mr.yarn-check-interval-seconds=10
>>> kylin.storage.hbase.table-name-prefix=KYL_TEST_
>>> kylin.source.hive.database-for-flat-table=kylin
>>> kylin.storage.hbase.compression-codec=lz4
>>> kylin.storage.hbase.region-cut-gb=3
>>> kylin.storage.hbase.min-region-count=1
>>> kylin.storage.hbase.max-region-count=5000
>>> kylin.storage.partition.max-scan-bytes=16106127360
>>> kylin.storage.hbase.coprocessor-mem-gb=6
>>> kylin.security.profile=testing
>>> kylin.query.cache-enabled=true
>>> kylin.query.cache-threshold-duration=500
>>> kylin.query.cache-threshold-scan-count=10240
>>> kylin.storage.hbase.scan-cache-rows=4096
>>>
>>> Any help would be appreciated.
>>>
>>> Thanks,
>>> Ketan@Exponential
>>>
>>>
>

Mime
View raw message