kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From BELLIER Jean-luc <jean-luc.bell...@rte-france.com>
Subject RE: Problem starting Kylin Web interface
Date Tue, 06 Feb 2018 15:33:58 GMT

Hello ,


1.       I did the following (in this order) :


·         Stop the kylin process (kylin.sh stop)

·         Rename the kylin.log and kylin.out files in $KYLIN_HOME/logs

·         Checked the environment (check-env.sh)

·         Start the kylin process (kylin.sh start)

So I get new kylin.out and kylin.log files. Apparentlty they seem to be OK. The version is
the http://apache.mindstudios.com/kylin/apache-kylin-2.2.0/apache-kylin-2.2.0-bin-cdh57.tar.gz.


2.       I did another test :
I cleaned everything, changed my version of Kylin (http://mirror.ibcp.fr/pub/apache/kylin/apache-kylin-2.2.0/apache-kylin-2.2.0-bin-hbase1x.tar.gz
instead of http://apache.mindstudios.com/kylin/apache-kylin-2.2.0/apache-kylin-2.2.0-bin-cdh57.tar.gz),
put it in a separate directory and changed the value of $KYLIN_HOME.

I got strange messages like this one :
2018-02-06 15:24:49,857 INFO  [main] Configuration.deprecation:1175 : hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
Exception in thread "main" java.lang.IllegalArgumentException: Failed to find metadata store
by url: kylin_metadata@hbase
        at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:89)
        at org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:101)
        at org.apache.kylin.common.persistence.ResourceTool.copy(ResourceTool.java:165)
       at org.apache.kylin.common.persistence.ResourceTool.copy(ResourceTool.java:196)
        at org.apache.kylin.common.persistence.ResourceTool.copy(ResourceTool.java:190)
        at org.apache.kylin.common.persistence.ResourceTool.main(ResourceTool.java:84)
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Get.setCheckExistenceOnly(Z)Lorg/apache/hadoop/hbase/client/Get;
        at org.apache.kylin.storage.hbase.HBaseResourceStore.internalGetFromHTable(HBaseResourceStore.java:377)
        at org.apache.kylin.storage.hbase.HBaseResourceStore.getFromHTable(HBaseResourceStore.java:363)
        at org.apache.kylin.storage.hbase.HBaseResourceStore.existsImpl(HBaseResourceStore.java:116)
        at org.apache.kylin.common.persistence.ResourceStore.exists(ResourceStore.java:144)
        at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:84)
        ... 5 more
2018-02-06 15:24:50,039 INFO  [Thread-1] client.ConnectionManager$HConnectionImplementation:2259
: Closing master protocol: MasterService
2018-02-06 15:24:50,046 INFO  [Thread-1] client.ConnectionManager$HConnectionImplementation:1830
: Closing zookeeper sessionid=0x36165799fcb08ad
2018-02-06 15:24:50,049 INFO  [Thread-1] zookeeper.ZooKeeper:684 : Session: 0x36165799fcb08ad
closed
2018-02-06 15:24:50,049 INFO  [main-EventThread] zookeeper.ClientCnxn:512 : EventThread shut
down

However, I can create tables in HBase, so it is not a problem of rights.

So I came back to my first version of Kylin. All the folders and bases are back to their state.
In the Web interface, the models are still loading indefinitely, but I could not also see
any Hive table, and cannot load them.
Could there be the cause of the problem ?

Thank you in advance for your help.

Best regards,
Jean-Luc.


De : ShaoFeng Shi [mailto:shaofengshi@apache.org]
Envoyé : mardi 6 février 2018 13:30
À : user <user@kylin.apache.org>
Objet : Re: Problem starting Kylin Web interface

All looks good; Can't believe that it can not create the folder on HDFS. Please check more.

2018-02-06 18:58 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france.com<mailto:jean-luc.bellier@rte-france.com>>:
Hello,

In the /etc/hadoop/conf folder, I can see under the fs.defaultFS property the following value
: hdfs://<cloudera-manager_name>:8020. The file indicates that the content was automatically
generated by Cloudera Manager. Are there other points to check in this file ?

According to the trace of $KYLIN_HOME/bin/find-hadoop-conf-dir.sh –v below :
kylin_hadoop_conf_dir is /etc/hadoop/conf
So I presume it is correct. The file called ‘yarn-site.xml’ is also in this folder.

I hope this helps.

Best regards,
Jean-Luc.

De : ShaoFeng Shi [mailto:shaofengshi@apache.org<mailto:shaofengshi@apache.org>]
Envoyé : mardi 6 février 2018 11:39

À : user <user@kylin.apache.org<mailto:user@kylin.apache.org>>
Objet : Re: Problem starting Kylin Web interface

 Kylin has located the Hadoop config folder as "/etc/hadoop/conf", is this the right location?
Please check whether the core-site.xml under this folder has the proper configuration, for
example, 'fs.defaultFS' points to the HDFS.

2018-02-06 16:58 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france.com<mailto:jean-luc.bellier@rte-france.com>>:
Hello,

I have cleaned all then environments (HBase, Hive) and restarted the sample.sh script again.
In the output messages, I can see several times the following :

2018-02-06 09:43:21,034 INFO  [main] util.ZookeeperDistributedLock:154 : 4058@node2.hades.rte-france.com<mailto:4058@node2.hades.rte-france.com>
acquired lock at /kylin/kylin_metadata/create_htable/kylin_metadata/lock
2018-02-06 09:43:21,050 DEBUG [main] hbase.HBaseConnection:309 : Creating HTable 'kylin_metadata'
2018-02-06 09:43:21,083 INFO  [main] Configuration.deprecation:1175 : hadoop.native.lib is
deprecated. Instead, use io.native.lib.available

However, my HBase table (‘kylin_metadata’) is created and contains 20 rows. I suppose
it is correct.

I have a question : in the KYLIN Interface, where are the models loaded from ? I could help
to understand the problem.

Thank you again for your help. Have a good ay.

Best regards,
Jean-Luc.


De : BELLIER Jean-luc
Envoyé : mardi 6 février 2018 09:12
À : user <user@kylin.apache.org<mailto:user@kylin.apache.org>>
Objet : RE: Problem starting Kylin Web interface

Hello,

I thank you for your feedback and the information about the kylin.properties file, which gives
lots of information.

To answer your question, here is the result of the ./bin/find-hadoop-conf-dir.sh –v command

bellierjlu@node2:~$ $KYLIN_HOME/bin/find-hadoop-conf-dir.sh -v
Turn on verbose mode.
Retrieving hadoop conf dir...
/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hbase/bin/../conf is not valid hadoop
dir conf because yarn-site.xml is missing
/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hbase/bin/.. is not valid hadoop dir
conf because yarn-site.xml is missing
kylin_hadoop_conf_dir is /etc/hadoop/conf
bellierjlu@node2:~$

Could this be the cause of the problem ?

Thank you in advance for your help.

Best regards,
Jean-Luc.

De : ShaoFeng Shi [mailto:shaofengshi@apache.org]
Envoyé : mardi 6 février 2018 01:45
À : user <user@kylin.apache.org<mailto:user@kylin.apache.org>>
Objet : Re: Problem starting Kylin Web interface

Hi Jean-Luc,

Kylin's metadata is persisted in HBase, the metadata table is "kylin_metadata" by default,
which is also configurable in conf/kylin.properties with key kylin.metadata.url;
The HDFS path '/kylin/kylin_metadata' is for the cube and other files; Before you build the
first cube, the folder is empty I think.

If sample.sh couldn't create the /tmp/kylin folder on HDFS; If the folder wasn't create, Kylin
might not find the Hadoop config properly. You can run:


./bin/find-hadoop-conf-dir.sh -v



to see what it prints.




2018-02-05 23:24 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france.com<mailto:jean-luc.bellier@rte-france.com>>:
Hello,

I am sorry to come back again to you, but I am really stuck. I restarted all the services
and agents on my Cloudera cluster, so it is OK now.
I noticed that some HDFS directories were empty, especially /tmp/kylin. When I have a look
on the code of sample.sh, I can see that this script should create sthe subfolder /tmp/kylin/sample_cube/data.
These data are used to create the Hive tables and are removed after the creation of these
tables.  Let call the node N1

I have another machine on which Kylin is installed and I succeeded in launching sample.sh.
On this machine, I can see a HDFS folder called /kylin/kylin_metadata with, I presume, the
metadata of the cube stored in it.

I cannot succeed in creating this folder on N1. I removed the /tmp/kylin folder, the hbase
table and the Hive tables.
I really do not know what to do to restart the creation of the project from scratch.
I also checked the HDFS space using the command ‘hdfs dfs –df’ and it tells me a usage
of 2%.

Could you please guide me in how to achieve it. There may be something I did wrong, but I
really do not what. Is my way of doing correct ? Are there  other files to remove (temporary
files, for example), and in this case, where are they located ?

Any help would be greatly appreciated.

Thank you in advance for your help. Have a good day.

Best regards,
Jean-Luc.

De : BELLIER Jean-luc
Envoyé : vendredi 2 février 2018 18:54
À : user <user@kylin.apache.org<mailto:user@kylin.apache.org>>
Objet : RE: Problem starting Kylin Web interface

Hello,

Thanks for the suggestion. I tried on another machine with Firefox and Chrome, but I get the
same result as before. I also rebuilt the project ‘sample’ from scratch, with no change.
My cluster seems to have problems on its nodes, especially memory which is nearly full on
most nodes.
I suspect this could be one of the cause of the problem : no available memory and swap space
also nearly full.

What do you think of this assumption ?

I will see that with my Cloudera manager next week, as I am not a specialist at all.

Have a good week-end.

Best regards,
Jean-Luc.


De : ShaoFeng Shi [mailto:shaofengshi@apache.org]
Envoyé : vendredi 2 février 2018 14:59

À : user <user@kylin.apache.org<mailto:user@kylin.apache.org>>
Objet : Re: Problem starting Kylin Web interface

Please try Chrome; Most developers are using Chrome; And if there is an error, you can press
F12 to check the HTTP request/response, there should be more information.

2018-02-02 21:54 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france.com<mailto:jean-luc.bellier@rte-france.com>>:
Hello again,

I forgot to mention that I use IE or Secured Firefox as Web browsers. I have the same issue
with both. Could there be the cause of the problem ?

Best regards
Jean-Luc.

De : BELLIER Jean-luc
Envoyé : vendredi 2 février 2018 14:48
À : user <user@kylin.apache.org<mailto:user@kylin.apache.org>>
Objet : RE: Problem starting Kylin Web interface

Hello

Thank you for your answer.
I did the following :

•         rename my current kylin.log and kylin.out files to keep a trace

•         restarted the commands ‘kylin.sh start’ and ‘sample.sh’
I cannot see anything special in the new .log or .out files. The Hive tables exist, and the
kylin_metadata also. I can scan it through Hbase shell command. It gives me 44  rows.

However, the problem is still the same with the Kylin Interface : Models loading indefinitely,
no table displayed, and nothing in the ‘System’ section. When I try to refresh the server
config and the server environement, I still have the message ‘Oops ! Failed to take action.
I also have this message as soon as I click to the ‘System’ button.

Should I rebuild the project from scratch to be sure that everything is correctly created,
or should I investigate somewhere else (services, node states, …) ?

I send in attachement the now kylin.log and kylin.out files
Thank you in advance for your help. Have a good day.

Best regards,
Jean-Luc.

De : ShaoFeng Shi [mailto:shaofengshi@apache.org]
Envoyé : vendredi 2 février 2018 12:06
À : user <user@kylin.apache.org<mailto:user@kylin.apache.org>>
Objet : Re: Problem starting Kylin Web interface

Hi Jean-luc,

The kylin.log seems okay, the service is started and the job scheduler is running up.

The kylin.out has some error about missing ".keystore", but by default if you accessing kylin
via http, it shouldn't need to load keystore. Please try visit http://host:7070/kylin again
to see whether there is any issue in the web console.

2018-02-01 23:17 GMT+08:00 BELLIER Jean-luc <jean-luc.bellier@rte-france.com<mailto:jean-luc.bellier@rte-france.com>>:
Hello,

I am new in Kylin. I would like to test it for a POC for my company. I opened a ticket yesterday
in the issues mail box through Jira, but I am not sure it is the best place.
Here is the reference of the ticket : https://issues.apache.org/jira/browse/KYLIN-3218

I am using  the version apache-kylin-2.2.0-bin-cdh57, with Linux. I installed the software
on a client node. I successfully started the process and created the cube given in the example
project.
Apparently, no mistake appears when starting the kylin.sh shell or generating the cube using
sample.sh.

When I launch the web interface using <nodename>:7070/kylin, I cannot load the models
(the cursor is turning indefinitely).
By looking at the kylin.out log, I could see that the process could not find the $KYLIN_HOME/tomcat/conf/.keystore
file. So I created it manually, and relaunched the cube generation and the ‘kylin.sh start’
command.
Now I face to other mistakes :


•         Exception in thread "Thread-14" java.lang.RuntimeException: Error while peeking
at /kylin/kylin_metadata/job_engine/global_job_engine_lock

•         SEVERE: The web application [/kylin] appears to have started a thread named [TGT
Renewer for bellierjlu@HADES.RTE-FRANCE.COM<mailto:bellierjlu@HADES.RTE-FRANCE.COM>]
but has failed to stop it. This is very likely to create a memory leak. This one appears several
times

•         SEVERE: The web application [/kylin] created a ThreadLocal with key of type [java.lang.ThreadLocal]
(value [java.lang.ThreadLocal@10cb8259]) and a value of type [org.apache.kylin.rest.msg.Message]
(value [org.apache.kylin.rest.msg.Message@267ae210]) but failed to remove it when the web
application was stopped. Threads are going to be renewed over time to try and avoid a probable
memory leak. This one appears several times


Apparently, something else is missing, or I did something wrong, but I do not know what. I
would also like to know what will be the impact of the two ‘SEVERE’ issues above and how
I could correct it.


I attach the last version of kylin.out file. Any help would be kindly appreciated.



Thank you in advance for your help. Have a good day.



Best regards.

Jean-Luc.




"Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé
et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce
document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder
à sa destruction.

This message is solely intended for the use of the individual or entity to which it is addressed
and may contain information that is privileged or confidential. If you have received this
communication by error, please notify us immediately by electronic mail, do not disclose it
and delete the original message."



--
Best regards,

Shaofeng Shi 史少锋



"Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé
et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce
document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder
à sa destruction.

This message is solely intended for the use of the individual or entity to which it is addressed
and may contain information that is privileged or confidential. If you have received this
communication by error, please notify us immediately by electronic mail, do not disclose it
and delete the original message."



--
Best regards,

Shaofeng Shi 史少锋



"Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé
et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce
document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder
à sa destruction.

This message is solely intended for the use of the individual or entity to which it is addressed
and may contain information that is privileged or confidential. If you have received this
communication by error, please notify us immediately by electronic mail, do not disclose it
and delete the original message."



--
Best regards,

Shaofeng Shi 史少锋



"Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé
et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce
document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder
à sa destruction.

This message is solely intended for the use of the individual or entity to which it is addressed
and may contain information that is privileged or confidential. If you have received this
communication by error, please notify us immediately by electronic mail, do not disclose it
and delete the original message."



--
Best regards,

Shaofeng Shi 史少锋



"Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé
et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce
document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder
à sa destruction.

This message is solely intended for the use of the individual or entity to which it is addressed
and may contain information that is privileged or confidential. If you have received this
communication by error, please notify us immediately by electronic mail, do not disclose it
and delete the original message."



--
Best regards,

Shaofeng Shi 史少锋



"Ce message est destiné exclusivement aux personnes ou entités auxquelles il est adressé
et peut contenir des informations privilégiées ou confidentielles. Si vous avez reçu ce
document par erreur, merci de nous l'indiquer par retour, de ne pas le transmettre et de procéder
à sa destruction.

This message is solely intended for the use of the individual or entity to which it is addressed
and may contain information that is privileged or confidential. If you have received this
communication by error, please notify us immediately by electronic mail, do not disclose it
and delete the original message."
Mime
View raw message