kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Iñigo Martínez <>
Subject Re: Upgrade from 2.4.1 to 2.6.1
Date Tue, 19 Mar 2019 10:01:27 GMT
Hi Shao Feng.

Yesterday I deployed a new 2.6.1 instance with both mysql and hbase as
metastore. Since I've only built test cube, everything runs smoothly.
I'm going to test today restoring metadata from 2.4.1 to see if we can
proceed with an in place migration. Using mysql as backend will probably
help is in order to get more stability with HBase. We have made huge
improvements tuning hbase, but still some issues as present and having
thousands of builds per month dumping logs in kylin_metadata table is not
very good.

El mar., 19 mar. 2019 a las 8:48, ShaoFeng Shi (<>)

> Hello Inigo,
> This is a good question.
> The MySQL metadata store was introduced in Kylin 2.5, as preparation for
> the no-HBase deployment. There is no evidence to say the MySQL meta store
> will have better performance, or be more stable. But at least, when HBase
> has a problem, Kylin service won't be impacted.
> To migrate to MySQL meta store, you can just 1) dump all metadata to local
> disk; 2) change Kylin configuration to use MySQL meta store; 3) restore
> metadata from local disk. MySQL will use two tables to persist the
> metadata, one for static resources (project, cube, etc), the other for job
> outputs. But this is transparent to end user.
> Best regards,
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email:
> Apache Kylin FAQ:
> Join Kylin user mail group:
> Join Kylin dev mail group:
> Na Zhai <> 于2019年3月17日周日 下午9:28写道:
>> Hi, Iñigo Martínez.
>> If you meet too many problems with HBase, you can try to use MySQL
>> instead. Hope this can help you.
>> In HBase,
>> there is one metadata_table. In MySQL, there are two metadata_tables. So I
>> think you can not migrate metadata from HBase to MySQL directly.
>> 发送自 Windows 10 版邮件 <>应用
>> ------------------------------
>> *发件人:* Iñigo Martínez <>
>> *发送时间:* Thursday, March 14, 2019 10:40:27 PM
>> *收件人:*
>> *主题:* Upgrade from 2.4.1 to 2.6.1
>> Now that 2.6.1 is launched, we are planning an upgrade (with a transition
>> period and an extensive test plan) from 2.4.1. I have a few questions for
>> developers and / or system administrators.
>> - Do you recommend use mysql as kylin metadata storage? Until now, most
>> of our problems are related with hbase since we rebuild cubes very
>> frequently and kylin_metadata table grows a lot. Do you notice performance
>> loss or gain? Is it stable enough?
>> - If using mysql as metadata backend, can we export / import metadata in
>> order to migrate from hbase to mysql without having rebuild existing cubes?
>> Thanks in advance.


Iñigo Martínez
Systems Manager

  [image: Telecoming - Make it digital]
[image: 5000_empresas]
[image: 1000_empresas]

Paseo de la Castellana, 95. Torre Europa, pl 16. 28046 Madrid, Spain | <>

  Este correo electrónico y sus archivos adjuntos están dirigidos
únicamente a la(s) dirección(es) indicada(s) anteriormente. El carácter
confidencial, personal e intransferible del mismo está protegido
legalmente. Cualquier publicación, reproducción, distribución o
retransmisión no autorizada, ya sea completa o en parte, se encuentra
prohibida. Si ha recibido este mensaje por equivocación, notifíquelo
inmediatamente a la persona que lo ha enviado y borre el mensaje original
junto con sus ficheros anexos sin leerlo ni grabarlo en modo alguno.

View raw message