Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E20EB200B5F for ; Fri, 29 Jul 2016 19:02:46 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E0A9E160A79; Fri, 29 Jul 2016 17:02:46 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 46330160A6E for ; Fri, 29 Jul 2016 19:02:44 +0200 (CEST) Received: (qmail 72863 invoked by uid 500); 29 Jul 2016 17:02:43 -0000 Mailing-List: contact commits-help@bigtop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: bigtop-dev@bigtop.apache.org Delivered-To: mailing list commits@bigtop.apache.org Received: (qmail 72850 invoked by uid 99); 29 Jul 2016 17:02:43 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Jul 2016 17:02:43 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id DECAEE78B5; Fri, 29 Jul 2016 17:02:42 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: oflebbe@apache.org To: commits@bigtop.apache.org Date: Fri, 29 Jul 2016 17:02:42 -0000 Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: [01/51] [abbrv] bigtop git commit: BIGTOP-2414. Rename Tachyon to Alluxio [Forced Update!] archived-at: Fri, 29 Jul 2016 17:02:47 -0000 Repository: bigtop Updated Branches: refs/heads/BIGTOP-2253 c5a36fa77 -> 2fe4c1f5d (forced update) http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-packages/src/deb/tachyon/source/format ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/deb/tachyon/source/format b/bigtop-packages/src/deb/tachyon/source/format deleted file mode 100644 index 163aaf8..0000000 --- a/bigtop-packages/src/deb/tachyon/source/format +++ /dev/null @@ -1 +0,0 @@ -3.0 (quilt) http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-packages/src/deb/tachyon/tachyon-tfs.dirs ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/deb/tachyon/tachyon-tfs.dirs b/bigtop-packages/src/deb/tachyon/tachyon-tfs.dirs deleted file mode 100644 index b05374b..0000000 --- a/bigtop-packages/src/deb/tachyon/tachyon-tfs.dirs +++ /dev/null @@ -1,4 +0,0 @@ -/etc/tachyon/conf -/usr/lib/tachyon -/usr/bin -/var/run/tachyon http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-packages/src/deb/tachyon/tachyon-tfs.install ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/deb/tachyon/tachyon-tfs.install b/bigtop-packages/src/deb/tachyon/tachyon-tfs.install deleted file mode 100644 index 473e2f5..0000000 --- a/bigtop-packages/src/deb/tachyon/tachyon-tfs.install +++ /dev/null @@ -1,3 +0,0 @@ -/etc/tachyon/conf -/usr/lib/tachyon -/usr/bin/tachyon http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-packages/src/deb/tachyon/tachyon-tfs.postinst ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/deb/tachyon/tachyon-tfs.postinst b/bigtop-packages/src/deb/tachyon/tachyon-tfs.postinst deleted file mode 100644 index a2078db..0000000 --- a/bigtop-packages/src/deb/tachyon/tachyon-tfs.postinst +++ /dev/null @@ -1,32 +0,0 @@ -#!/bin/bash -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -set -e - -case "$1" in - configure) - ;; - - abort-upgrade|abort-remove|abort-deconfigure) - ;; - - *) - echo "postinst called with unknown argument \`$1'" >&2 - exit 1 - ;; -esac - -#DEBHELPER# http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-packages/src/deb/tachyon/tachyon-tfs.prerm ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/deb/tachyon/tachyon-tfs.prerm b/bigtop-packages/src/deb/tachyon/tachyon-tfs.prerm deleted file mode 100644 index 4ba1a94..0000000 --- a/bigtop-packages/src/deb/tachyon/tachyon-tfs.prerm +++ /dev/null @@ -1,37 +0,0 @@ -#!/bin/bash -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -set -e - -case "$1" in - remove|upgrade|deconfigure) - ;; - - failed-upgrade) - ;; - - *) - echo "prerm called with unknown argument \`$1'" >&2 - exit 1 - ;; -esac - -# dh_installdeb will replace this with shell code automatically -# generated by other debhelper scripts. - -#DEBHELPER# - -exit 0 http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-packages/src/rpm/alluxio/SPECS/alluxio.spec ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/rpm/alluxio/SPECS/alluxio.spec b/bigtop-packages/src/rpm/alluxio/SPECS/alluxio.spec new file mode 100644 index 0000000..6fb4b69 --- /dev/null +++ b/bigtop-packages/src/rpm/alluxio/SPECS/alluxio.spec @@ -0,0 +1,145 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +Name: alluxio +Version: %{alluxio_version} +Release: %{alluxio_release} +Summary: Reliable file sharing at memory speed across cluster frameworks +License: ASL 2.0 +URL: http://www.alluxio.org/ +Group: Development/Libraries +BuildArch: noarch + +Source0: %{alluxio_name}-%{alluxio_base_version}.tar.gz +Source1: do-component-build +Source2: install_alluxio.sh +Source3: init.d.tmpl +Source4: alluxio-master.svc +Source5: alluxio-worker.svc +%define alluxio_name alluxio +%define alluxio_home /usr/lib/%{alluxio_name} +%define alluxio_services master worker +%define var_lib /var/lib/%{alluxio_name} +%define var_run /var/run/%{alluxio_name} +%define var_log /var/log/%{alluxio_name} +%define etc_alluxio /etc/%{alluxio_name} +%define config_alluxio %{etc_alluxio}/conf + +%global initd_dir %{_sysconfdir}/init.d + +%if %{?suse_version:1}0 +%define alternatives_cmd update-alternatives +%else +%define alternatives_cmd alternatives +%endif + +%if %{?suse_version:1}0 +# Required for init scripts +Requires: insserv +%global initd_dir %{_sysconfdir}/rc.d + +%else +# Required for init scripts +Requires: /lib/lsb/init-functions + +%global initd_dir %{_sysconfdir}/rc.d/init.d + +%endif + +Requires: bigtop-utils + +# disable repacking jars +%define __arch_install_post %{nil} + +%description +Alluxio is a fault tolerant distributed file system +enabling reliable file sharing at memory-speed +across cluster frameworks, such as Spark and MapReduce. +It achieves high performance by leveraging lineage +information and using memory aggressively. +Alluxio caches working set files in memory, and +enables different jobs/queries and frameworks to +access cached files at memory speed. Thus, Alluxio +avoids going to disk to load data-sets that +are frequently read. + +%prep +%setup -n %{alluxio_name}-%{alluxio_base_version} + +%build +bash $RPM_SOURCE_DIR/do-component-build + +%install +rm -rf $RPM_BUILD_ROOT + +# See /usr/lib/rpm/macros for info on how vars are defined. +# Here we run the alluxio installation script. +bash %{SOURCE2} \ + --build-dir=%{buildroot} \ + --bin-dir=%{_bindir} \ + --data-dir=%{_datadir} \ + --libexec-dir=%{_libexecdir} \ + --var-dir=%{_var} \ + --prefix="${RPM_BUILD_ROOT}" + +for service in %{alluxio_services} +do + # Install init script + init_file=$RPM_BUILD_ROOT/%{initd_dir}/%{alluxio_name}-${service} + bash $RPM_SOURCE_DIR/init.d.tmpl $RPM_SOURCE_DIR/alluxio-${service}.svc rpm $init_file +done + +%pre +getent group alluxio >/dev/null || groupadd -r alluxio +getent passwd alluxio >/dev/null || useradd -c "Alluxio" -s /sbin/nologin -g alluxio -r -d %{var_lib_alluxio} alluxio 2> /dev/null || : + +%post +%{alternatives_cmd} --install %{config_alluxio} %{alluxio_name}-conf %{config_alluxio}.dist 30 + +%preun +if [ "$1" = 0 ]; then + %{alternatives_cmd} --remove %{alluxio_name}-conf %{config_alluxio}.dist || : +fi + +for service in %{alluxio_services}; do + /sbin/service %{alluxio_name}-${service} status > /dev/null 2>&1 + if [ $? -eq 0 ]; then + /sbin/service %{alluxio_name}-${service} stop > /dev/null 2>&1 + fi +done + + +%files +%defattr(-,root,root,-) +%doc LICENSE README.md +%dir %{_sysconfdir}/%{alluxio_name} +%config(noreplace) %{_sysconfdir}/%{alluxio_name}/conf/log4j.properties +%config(noreplace) %{initd_dir}/%{alluxio_name}-master +%config(noreplace) %{initd_dir}/%{alluxio_name}-worker +%config(noreplace) %{_sysconfdir}/%{alluxio_name}/conf/alluxio-env.sh +%config(noreplace) %{alluxio_home}/libexec/alluxio-layout.sh +%attr(0755,alluxio,alluxio) %{var_lib} +%attr(0755,alluxio,alluxio) %{var_run} +%attr(0755,alluxio,alluxio) %{var_log} +%{alluxio_home}/alluxio* +%{alluxio_home}/bin/alluxio* +%{alluxio_home}/libexec/alluxio* +%{_datadir}/%{alluxio_name} +/usr/bin/alluxio +%{alluxio_home}/share + + +%clean +rm -rf $RPM_BUILD_ROOT http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-packages/src/rpm/tachyon/SPECS/tachyon.spec ---------------------------------------------------------------------- diff --git a/bigtop-packages/src/rpm/tachyon/SPECS/tachyon.spec b/bigtop-packages/src/rpm/tachyon/SPECS/tachyon.spec deleted file mode 100644 index f5ee435..0000000 --- a/bigtop-packages/src/rpm/tachyon/SPECS/tachyon.spec +++ /dev/null @@ -1,131 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -Name: tachyon-tfs -Version: %{tachyon_version} -Release: %{tachyon_release} -Summary: Reliable file sharing at memory speed across cluster frameworks -License: ASL 2.0 -URL: http://tachyon-project.org/ -Group: Development/Libraries -BuildArch: noarch - -Source0: %{tachyon_name}-%{tachyon_base_version}.tar.gz -Source1: do-component-build -Source2: install_tachyon.sh -Source3: init.d.tmpl -Source4: tachyon-master.svc -Source5: tachyon-worker.svc -%define tachyon_name tachyon -%define tachyon_home /usr/lib/%{tachyon_name} -%define tachyon_services master worker -%define var_lib /var/lib/%{tachyon_name} -%define var_run /var/run/%{tachyon_name} -%define var_log /var/log/%{tachyon_name} - -%global initd_dir %{_sysconfdir}/init.d - -%if %{?suse_version:1}0 -# Required for init scripts -Requires: insserv -%global initd_dir %{_sysconfdir}/rc.d - -%else -# Required for init scripts -Requires: /lib/lsb/init-functions - -%global initd_dir %{_sysconfdir}/rc.d/init.d - -%endif - -Requires: bigtop-utils - -# disable repacking jars -%define __arch_install_post %{nil} - -%description -Tachyon is a fault tolerant distributed file system -enabling reliable file sharing at memory-speed -across cluster frameworks, such as Spark and MapReduce. -It achieves high performance by leveraging lineage -information and using memory aggressively. -Tachyon caches working set files in memory, and -enables different jobs/queries and frameworks to -access cached files at memory speed. Thus, Tachyon -avoids going to disk to load data-sets that -are frequently read. - -%prep -%setup -n %{tachyon_name}-%{tachyon_base_version} - -%build -bash $RPM_SOURCE_DIR/do-component-build - -%install -rm -rf $RPM_BUILD_ROOT - -# See /usr/lib/rpm/macros for info on how vars are defined. -# Here we run the tachyon installation script. -bash %{SOURCE2} \ - --build-dir=%{buildroot} \ - --bin-dir=%{_bindir} \ - --data-dir=%{_datadir} \ - --libexec-dir=%{_libexecdir} \ - --var-dir=%{_var} \ - --prefix="${RPM_BUILD_ROOT}" - -for service in %{tachyon_services} -do - # Install init script - init_file=$RPM_BUILD_ROOT/%{initd_dir}/%{tachyon_name}-${service} - bash $RPM_SOURCE_DIR/init.d.tmpl $RPM_SOURCE_DIR/tachyon-${service}.svc rpm $init_file -done - -%preun -for service in %{tachyon_services}; do - /sbin/service %{tachyon_name}-${service} status > /dev/null 2>&1 - if [ $? -eq 0 ]; then - /sbin/service %{tachyon_name}-${service} stop > /dev/null 2>&1 - fi -done - - -%files -%defattr(-,root,root,-) -%doc LICENSE README.md -%dir %{_sysconfdir}/%{tachyon_name} -%config(noreplace) %{_sysconfdir}/%{tachyon_name}/conf/log4j.properties -%config(noreplace) %{_sysconfdir}/%{tachyon_name}/conf/workers -%config(noreplace) %{initd_dir}/%{tachyon_name}-master -%config(noreplace) %{initd_dir}/%{tachyon_name}-worker -%config(noreplace) %{_sysconfdir}/%{tachyon_name}/conf/tachyon-env.sh -%config(noreplace) %{tachyon_home}/libexec/tachyon-layout.sh -%attr(0755,root,root) %{var_lib} -%attr(0755,root,root) %{var_run} -%attr(0755,root,root) %{var_log} -%{tachyon_home}/tachyon* -%{tachyon_home}/bin/tachyon* -%{tachyon_home}/libexec/tachyon* -%{_datadir}/%{tachyon_name} -/usr/bin/tachyon -%{tachyon_home}/share - - -%clean -rm -rf $RPM_BUILD_ROOT - -%changelog -* Thu Mar 05 2015 Huamin Chen -- upgrade to Tachyon 0.6.0 http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-tests/smoke-tests/alluxio/TestAlluxioSmoke.groovy ---------------------------------------------------------------------- diff --git a/bigtop-tests/smoke-tests/alluxio/TestAlluxioSmoke.groovy b/bigtop-tests/smoke-tests/alluxio/TestAlluxioSmoke.groovy new file mode 100644 index 0000000..ad26cf6 --- /dev/null +++ b/bigtop-tests/smoke-tests/alluxio/TestAlluxioSmoke.groovy @@ -0,0 +1,80 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + *

+ * http://www.apache.org/licenses/LICENSE-2.0 + *

+ * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.bigtop.itest.alluxio + +import org.junit.Before +import org.apache.bigtop.itest.shell.Shell +import static org.junit.Assert.assertTrue +import static org.junit.Assert.assertNotNull +import org.junit.Test +import org.apache.bigtop.itest.JarContent + +class TestAlluxioSmoke { + + def alluxioHome = prop('ALLUXIO_HOME'); + def alluxioMasterAddress = prop('ALLUXIO_MASTER_ADDRESS'); + def alluxioTestDir = prop('ALLUXIO_TEST_DIR', '/bigtop'); + def hadoopHome = prop('HADOOP_HOME'); + + def hadoop = "${hadoopHome}/bin/hadoop" + def alluxio = "${alluxioHome}/bin/alluxio" + + Shell sh = new Shell("/bin/bash -s"); + + String prop(String key) { + def value = System.getenv(key) + assertNotNull(value) + return value + } + + String prop(String key, String defaultValue) { + def value = System.getenv(key) + if (value == null) { + return defaultValue + } + return value + } + + /** + * Runs the Alluxio runTests command that runs the Alluxio examples against the cluster. + * This command takes care of cleanup at the start of each test, so this framework + * does not have to worry about cleanup. + */ + @Test + void runTests() { + sh.exec("$alluxio runTests"); + assertTrue("runTests failed. " + sh.getOut() + " " + sh.getErr(), sh.getRet() == 0); + } + + @Test + void hadoopCat() { + sh.exec(""" + set -x + set -e + + export LIB_JARS=\$(find ${alluxioHome} -name "alluxio-client-*-jar-with-dependencies.jar" | sort | head -n1) + export HADOOP_CLASSPATH=\${LIB_JARS} + + $alluxio tfs rm $alluxioTestDir/hadoopLs/datafile + $alluxio tfs copyFromLocal datafile $alluxioTestDir/hadoopLs/datafile + $hadoop fs -cat $alluxioMasterAddress/$alluxioTestDir/hadoopLs/datafile + """) + assertTrue("Unable to list from hadoop. " + sh.getOut().join('\n') + " " + sh.getErr().join('\n'), sh.getRet() == 0); + } +} http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-tests/smoke-tests/alluxio/build.gradle ---------------------------------------------------------------------- diff --git a/bigtop-tests/smoke-tests/alluxio/build.gradle b/bigtop-tests/smoke-tests/alluxio/build.gradle new file mode 100644 index 0000000..8b392f9 --- /dev/null +++ b/bigtop-tests/smoke-tests/alluxio/build.gradle @@ -0,0 +1,32 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + *

+ * http://www.apache.org/licenses/LICENSE-2.0 + *

+ * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +sourceSets { + test { + groovy { + srcDirs = ["./"] + } + } +} + +test.doFirst { + checkEnv([ + "ALLUXIO_HOME", + "ALLUXIO_MASTER_ADDRESS", + "HADOOP_HOME" + ]) +} http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-tests/smoke-tests/alluxio/datafile ---------------------------------------------------------------------- diff --git a/bigtop-tests/smoke-tests/alluxio/datafile b/bigtop-tests/smoke-tests/alluxio/datafile new file mode 100644 index 0000000..87f936f --- /dev/null +++ b/bigtop-tests/smoke-tests/alluxio/datafile @@ -0,0 +1 @@ +Test file to see if Hadoop is able to read this file from Alluxio. http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-tests/smoke-tests/alluxio/log4j.properties ---------------------------------------------------------------------- diff --git a/bigtop-tests/smoke-tests/alluxio/log4j.properties b/bigtop-tests/smoke-tests/alluxio/log4j.properties new file mode 100644 index 0000000..db5aacf --- /dev/null +++ b/bigtop-tests/smoke-tests/alluxio/log4j.properties @@ -0,0 +1,24 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Set root logger level to DEBUG and its only appender to A1. +log4j.rootLogger=TRACE, A1 + +# A1 is set to be a ConsoleAppender. +log4j.appender.A1=org.apache.log4j.ConsoleAppender + +# A1 uses PatternLayout. +log4j.appender.A1.layout=org.apache.log4j.PatternLayout +log4j.appender.A1.layout.ConversionPattern=%-4r [BIGTOP-TEST-LOG %t] %-5p %c %x --- %m%n http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-tests/smoke-tests/tachyon/TestTachyonSmoke.groovy ---------------------------------------------------------------------- diff --git a/bigtop-tests/smoke-tests/tachyon/TestTachyonSmoke.groovy b/bigtop-tests/smoke-tests/tachyon/TestTachyonSmoke.groovy deleted file mode 100644 index cde700b..0000000 --- a/bigtop-tests/smoke-tests/tachyon/TestTachyonSmoke.groovy +++ /dev/null @@ -1,80 +0,0 @@ -/** - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - *

- * http://www.apache.org/licenses/LICENSE-2.0 - *

- * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.apache.bigtop.itest.tachyon - -import org.junit.Before -import org.apache.bigtop.itest.shell.Shell -import static org.junit.Assert.assertTrue -import static org.junit.Assert.assertNotNull -import org.junit.Test -import org.apache.bigtop.itest.JarContent - -class TestTachyonSmoke { - - def tachyonHome = prop('TACHYON_HOME'); - def tachyonMasterAddress = prop('TACHYON_MASTER_ADDRESS'); - def tachyonTestDir = prop('TACHYON_TEST_DIR', '/bigtop'); - def hadoopHome = prop('HADOOP_HOME'); - - def hadoop = "${hadoopHome}/bin/hadoop" - def tachyon = "${tachyonHome}/bin/tachyon" - - Shell sh = new Shell("/bin/bash -s"); - - String prop(String key) { - def value = System.getenv(key) - assertNotNull(value) - return value - } - - String prop(String key, String defaultValue) { - def value = System.getenv(key) - if (value == null) { - return defaultValue - } - return value - } - - /** - * Runs the Tachyon runTests command that runs the Tachyon examples against the cluster. - * This command takes care of cleanup at the start of each test, so this framework - * does not have to worry about cleanup. - */ - @Test - void runTests() { - sh.exec("$tachyon runTests"); - assertTrue("runTests failed. " + sh.getOut() + " " + sh.getErr(), sh.getRet() == 0); - } - - @Test - void hadoopCat() { - sh.exec(""" - set -x - set -e - - export LIB_JARS=\$(find ${tachyonHome} -name "tachyon-client-*-jar-with-dependencies.jar" | sort | head -n1) - export HADOOP_CLASSPATH=\${LIB_JARS} - - $tachyon tfs rm $tachyonTestDir/hadoopLs/datafile - $tachyon tfs copyFromLocal datafile $tachyonTestDir/hadoopLs/datafile - $hadoop fs -cat $tachyonMasterAddress/$tachyonTestDir/hadoopLs/datafile - """) - assertTrue("Unable to list from hadoop. " + sh.getOut().join('\n') + " " + sh.getErr().join('\n'), sh.getRet() == 0); - } -} http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-tests/smoke-tests/tachyon/build.gradle ---------------------------------------------------------------------- diff --git a/bigtop-tests/smoke-tests/tachyon/build.gradle b/bigtop-tests/smoke-tests/tachyon/build.gradle deleted file mode 100644 index ecef527..0000000 --- a/bigtop-tests/smoke-tests/tachyon/build.gradle +++ /dev/null @@ -1,32 +0,0 @@ -/** - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - *

- * http://www.apache.org/licenses/LICENSE-2.0 - *

- * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -sourceSets { - test { - groovy { - srcDirs = ["./"] - } - } -} - -test.doFirst { - checkEnv([ - "TACHYON_HOME", - "TACHYON_MASTER_ADDRESS", - "HADOOP_HOME" - ]) -} http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-tests/smoke-tests/tachyon/datafile ---------------------------------------------------------------------- diff --git a/bigtop-tests/smoke-tests/tachyon/datafile b/bigtop-tests/smoke-tests/tachyon/datafile deleted file mode 100644 index a57d895..0000000 --- a/bigtop-tests/smoke-tests/tachyon/datafile +++ /dev/null @@ -1 +0,0 @@ -Test file to see if Hadoop is able to read this file from Tachyon. http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop-tests/smoke-tests/tachyon/log4j.properties ---------------------------------------------------------------------- diff --git a/bigtop-tests/smoke-tests/tachyon/log4j.properties b/bigtop-tests/smoke-tests/tachyon/log4j.properties deleted file mode 100644 index db5aacf..0000000 --- a/bigtop-tests/smoke-tests/tachyon/log4j.properties +++ /dev/null @@ -1,24 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# Set root logger level to DEBUG and its only appender to A1. -log4j.rootLogger=TRACE, A1 - -# A1 is set to be a ConsoleAppender. -log4j.appender.A1=org.apache.log4j.ConsoleAppender - -# A1 uses PatternLayout. -log4j.appender.A1.layout=org.apache.log4j.PatternLayout -log4j.appender.A1.layout.ConversionPattern=%-4r [BIGTOP-TEST-LOG %t] %-5p %c %x --- %m%n http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/bigtop.bom ---------------------------------------------------------------------- diff --git a/bigtop.bom b/bigtop.bom index b12d6ae..4268677 100644 --- a/bigtop.bom +++ b/bigtop.bom @@ -90,7 +90,7 @@ bigtop { zookeeper:['hadoop', 'hbase'], hadoop:['ignite-hadoop', 'hbase', 'crunch', 'pig', 'hive', 'tez', 'sqoop', 'sqoop2', 'oozie', 'mahout', 'flume', 'giraph', 'solr', 'crunch', 'spark', - 'phoenix', 'tachyon', 'kafka', 'ycsb', 'kite', 'hama', 'zeppelin', + 'phoenix', 'alluxio', 'kafka', 'ycsb', 'kite', 'hama', 'zeppelin', 'tajo', 'apex' ], hbase:['phoenix','giraph','ycsb','hive'], @@ -357,14 +357,14 @@ bigtop { site = "${apache.APACHE_MIRROR}/${download_path}" archive = "${apache.APACHE_ARCHIVE}/${download_path}" } } - 'tachyon' { - name = "tachyon" - pkg = "tachyon-tfs" - relNotes = "Tachyon: a memory-centric distributed file system" - version { base = '0.6.0'; pkg = base; release = 1 } + 'alluxio' { + name = "alluxio" + pkg = "alluxio" + relNotes = "Alluxio: a memory-centric distributed file system" + version { base = '1.0.1'; pkg = base; release = 1 } tarball { destination = "$name-${version.base}.tar.gz" source = "v${version.base}.tar.gz" } - url { site = "https://github.com/amplab/tachyon/archive" + url { site = "https://github.com/Alluxio/alluxio/archive" archive = site } } 'kafka' { http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/build.gradle ---------------------------------------------------------------------- diff --git a/build.gradle b/build.gradle index c3d0457..6395be8 100644 --- a/build.gradle +++ b/build.gradle @@ -73,7 +73,7 @@ rat { "**/.gitignore", /* Test data with rigid structure and/or binary */ "bigtop-tests/test-artifacts/**/resources/**", - "bigtop-tests/smoke-tests/tachyon/datafile", + "bigtop-tests/smoke-tests/alluxio/datafile", "bigtop-tests/smoke-tests/hive/passwd.ql", "bigtop-tests/smoke-tests/kite/sandwiches.csv", "bigtop-tests/smoke-tests/phoenix/*.csv", http://git-wip-us.apache.org/repos/asf/bigtop/blob/f9923e2e/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 5d3781e..267b6c9 100644 --- a/pom.xml +++ b/pom.xml @@ -299,7 +299,7 @@ **/.gitignore bigtop-tests/test-artifacts/**/resources/** - bigtop-tests/smoke-tests/tachyon/datafile + bigtop-tests/smoke-tests/alluxio/datafile bigtop-tests/smoke-tests/hive/passwd.ql bigtop-tests/smoke-tests/kite/sandwiches.csv bigtop-tests/smoke-tests/phoenix/*.csv