Return-Path: X-Original-To: apmail-kylin-user-archive@minotaur.apache.org Delivered-To: apmail-kylin-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BBF8A18CF6 for ; Wed, 24 Feb 2016 02:14:40 +0000 (UTC) Received: (qmail 68494 invoked by uid 500); 24 Feb 2016 02:14:40 -0000 Delivered-To: apmail-kylin-user-archive@kylin.apache.org Received: (qmail 68436 invoked by uid 500); 24 Feb 2016 02:14:40 -0000 Mailing-List: contact user-help@kylin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@kylin.apache.org Delivered-To: mailing list user@kylin.apache.org Received: (qmail 68426 invoked by uid 99); 24 Feb 2016 02:14:40 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Feb 2016 02:14:40 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 36661C00EA for ; Wed, 24 Feb 2016 02:14:40 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.179 X-Spam-Level: * X-Spam-Status: No, score=1.179 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id LSpzgbtH5g04 for ; Wed, 24 Feb 2016 02:14:39 +0000 (UTC) Received: from mail-lb0-f174.google.com (mail-lb0-f174.google.com [209.85.217.174]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 840635FAFF for ; Wed, 24 Feb 2016 02:14:38 +0000 (UTC) Received: by mail-lb0-f174.google.com with SMTP id x4so2426524lbm.0 for ; Tue, 23 Feb 2016 18:14:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=r6xwANhrgmjxGCntpqkw7TyykbZtOFyAdYsKV075spg=; b=bbzwZuwlJz8gZ04C/UprSwS6D4NKOupLSMbZOXo34gbM51E3ac8TYrO3xyuQPQK4AI 9NskFJ/EI96StRO9Wobx/p2h6NbAQw4W78hzN2UIDiDMCitwxPzB7ev+ZsZ0WCR5qCkQ /XgK+BJmYDqUnxUm2Gh8UkmvWF2eD4+iQ/0J0Uky/cPtR30cnMSrRqrLCirsrrY97cDF EdL+vuQ9wrV8jkzHitQjQKfWYj0kfsWX1rLaHJNbi14E9ejLJ3oh+62Wp/amZxBGU7M7 S4VMas5WbAy6HwjxkAPZBFOzHj/LTN2c+HrkWd7VXhAXMXzikvZCM/8CYPuK4EJs35Lk HIHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=r6xwANhrgmjxGCntpqkw7TyykbZtOFyAdYsKV075spg=; b=T2FqWsihkhmYsK/yXgcbsiex1ZhQJWniKNFaSzR2cP3RtRLsFX1j64nCTridX/j9Yl bvEx/o6iJStxt+qFnroeNEpzBKt6veT8MOLM69tldkh1j99evIv2CnAcorO0UTtmBjlX jZINAchkAk1sY57M6h/L4l6FUXK6ri53KJhbSsI5IaT9Hn12N/YHvFmsjXqWEIG9VN2U MNyGE9UlprZfo4LpZ6ZRwoixkI6TPvccWVN3hTY4PwDkcdypK5P9HVfk04TYoxhupjEF d6xklrJb8a5zg7tFycOPBFsITonCtNMUlM6K54oLN6Whhj25Npmd+pDwH2IgkHl8bYg0 mbbQ== X-Gm-Message-State: AG10YOQwAOQp8xNY0OA3KOKLnMHuhFp+idlwFZAPG3XxEU8WbmHGtWCc7ec85NoHmb/K+8No1+4eF8oepSzdVQ== MIME-Version: 1.0 X-Received: by 10.112.198.162 with SMTP id jd2mr11287056lbc.53.1456280072455; Tue, 23 Feb 2016 18:14:32 -0800 (PST) Received: by 10.25.144.18 with HTTP; Tue, 23 Feb 2016 18:14:32 -0800 (PST) In-Reply-To: References: Date: Wed, 24 Feb 2016 10:14:32 +0800 Message-ID: Subject: Re: kylin's compression problem when I comment the related items From: yu feng To: user@kylin.apache.org Content-Type: multipart/alternative; boundary=001a11c349d6698caa052c7aa099 --001a11c349d6698caa052c7aa099 Content-Type: text/plain; charset=UTF-8 you should check some file like 'conf/kylin_job_conf.xml', those 'set key = value' read from this file. 2016-02-23 17:54 GMT+08:00 Mars J : > Hello, > I ran into a problem as the subject. Details as follow: > > version of my envr > OS , CentOS 7 > Apache Hadoop 2.7.2 > Apache Hive 1.2.1 > Apache HBase 1.1.3 > Apache Kylin 1.2 > > I ran the ${KYLIN_HOME}/bin/sample.sh and build it,but the build progress > is Error.The log print error caused by RuntimeException: native snappy > library not available: this version of libhadoop was built without snappy > support. > > I have already edit the conf/kylin.properties to set value to 'false' even > delete all items which include compress,but the error log also shows the > information below:(at the same time, I have ignore the related items in > $HADOOP_HOME/etc/hadoop/mapred-site.xml) > > SET mapreduce.job.split.metainfo.maxsize=-1; > SET mapred.compress.map.output=true; > SET mapred.map.output.compression.codec=org.apache.hadoop.io.compress.SnappyCodec; > SET mapred.output.compress=true; > SET mapred.output.compression.codec=org.apache.hadoop.io.compress.SnappyCodec; > SET mapred.output.compression.type=BLOCK; > SET mapreduce.job.max.split.locations=2000; > SET dfs.replication=2; > SET hive.merge.mapfiles=true; > SET hive.merge.mapredfiles=true; > SET hive.merge.size.per.task=268435456; > SET hive.support.concurrency=false; > SET hive.exec.compress.output=true; > SET hive.auto.convert.join.noconditionaltask = true; > SET hive.auto.convert.join.noconditionaltask.size = 300000000; > > Why it execute the set statement after I ignore the compress items like > mapred.map.output.compress etc. > > Is that some configurations I havn't done yet? > --001a11c349d6698caa052c7aa099 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
you should check some file like 'conf/kylin_job_conf.x= ml', those 'set key =3D value' read from this file.

2016-02-23 17:54 GMT+08= :00 Mars J <xujiao.mycafe@gmail.com>:
Hello,=C2= =A0
=C2=A0 =C2=A0 I ran into a problem = as the subject. Details as follow:

<= /div>
version of my envr
OS , CentOS 7
Apache Hadoo= p 2.7.2
Apache Hive 1.2.1
Apache HBase 1.1.3
= Apache Kylin 1.2

I ran the=C2=A0${KYLIN_HOME}/b= in/sample.sh and build it,but the build progress is Error.The log print err= or caused by=C2=A0RuntimeException: native snappy li= brary not available: this version of libhadoop was built without snappy sup= port.

=
I have al= ready edit the conf/kylin.properties to set value to 'false' even = =C2=A0delete all items which include compress,but the error log also shows = the information below:(at the same time, I have ignore the related items in= $HADOOP_HOME/etc/hadoop/mapred-site.xml)

SET mapreduce.job.split.metainfo.maxsize=3D-1;
SET mapred.compress.map.output=3Dtrue;
SET mapred.map.output.compression.codec=3Dorg.apache.hadoop.io.compress.Sna=
ppyCodec;
SET mapred.output.compress=3Dtrue;
SET mapred.output.compression.codec=3Dorg.apache.hadoop.io.compress.SnappyC=
odec;
SET mapred.output.compression.type=3DBLOCK;
SET mapreduce.job.max.split.locations=3D2000;
SET dfs.replication=3D2;
SET hive.merge.mapfiles=3Dtrue;
SET hive.merge.mapredfiles=3Dtrue;
SET hive.merge.size.per.task=3D268435456;
SET hive.support.concurrency=3Dfalse;
SET hive.exec.compress.output=3Dtrue;
SET hive.auto.convert.join.noconditionaltask =3D true;
SET hive.auto.convert.join.noconditionaltask.size =3D 300000000;
Why it execute the set= statement after I ignore the compress items like mapred.map.output.compres= s etc.
Is that = some configurations I havn't done yet?

--001a11c349d6698caa052c7aa099--