kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yu feng <olaptes...@gmail.com>
Subject Re: kylin's compression problem when I comment the related items
Date Wed, 24 Feb 2016 02:14:32 GMT
you should check some file like 'conf/kylin_job_conf.xml', those 'set key =
value' read from this file.

2016-02-23 17:54 GMT+08:00 Mars J <xujiao.mycafe@gmail.com>:

> Hello,
>     I ran into a problem as the subject. Details as follow:
>
> version of my envr
> OS , CentOS 7
> Apache Hadoop 2.7.2
> Apache Hive 1.2.1
> Apache HBase 1.1.3
> Apache Kylin 1.2
>
> I ran the ${KYLIN_HOME}/bin/sample.sh and build it,but the build progress
> is Error.The log print error caused by RuntimeException: native snappy
> library not available: this version of libhadoop was built without snappy
> support.
>
> I have already edit the conf/kylin.properties to set value to 'false' even
>  delete all items which include compress,but the error log also shows the
> information below:(at the same time, I have ignore the related items in
> $HADOOP_HOME/etc/hadoop/mapred-site.xml)
>
> SET mapreduce.job.split.metainfo.maxsize=-1;
> SET mapred.compress.map.output=true;
> SET mapred.map.output.compression.codec=org.apache.hadoop.io.compress.SnappyCodec;
> SET mapred.output.compress=true;
> SET mapred.output.compression.codec=org.apache.hadoop.io.compress.SnappyCodec;
> SET mapred.output.compression.type=BLOCK;
> SET mapreduce.job.max.split.locations=2000;
> SET dfs.replication=2;
> SET hive.merge.mapfiles=true;
> SET hive.merge.mapredfiles=true;
> SET hive.merge.size.per.task=268435456;
> SET hive.support.concurrency=false;
> SET hive.exec.compress.output=true;
> SET hive.auto.convert.join.noconditionaltask = true;
> SET hive.auto.convert.join.noconditionaltask.size = 300000000;
>
> Why it execute the set statement after I ignore the compress items like
> mapred.map.output.compress etc.
>
> Is that some configurations I havn't done yet?
>

Mime
View raw message