incubator-bigtop-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r..@apache.org
Subject svn commit: r1196308 [1/6] - in /incubator/bigtop/branches/hadoop-0.23: ./ bigtop-deploy/ bigtop-deploy/puppet/ bigtop-deploy/puppet/manifests/ bigtop-deploy/puppet/modules/ bigtop-deploy/puppet/modules/hadoop-flume/ bigtop-deploy/puppet/modules/hadoop...
Date Tue, 01 Nov 2011 21:53:09 GMT
Author: rvs
Date: Tue Nov  1 21:53:04 2011
New Revision: 1196308

URL: http://svn.apache.org/viewvc?rev=1196308&view=rev
Log:
Merging Bigtop 0.2.0

Added:
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/cluster.pp
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/site.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/manifests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/sqoop/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/tests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/tests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/whirr/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/manifests/init.pp
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-env.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-site.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/tests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/tests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/manifests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/templates/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/templates/hive-site.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/tests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/tests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/manifests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/templates/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/templates/oozie-site.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/tests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/tests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/manifests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/templates/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/templates/pig.properties
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/tests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/tests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/manifests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/tests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/tests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/manifests/init.pp
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/templates/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/templates/zoo.cfg
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/tests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/tests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/lib/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/lib/facter/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/lib/facter/hadoop_storage_locations.rb
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/sqoop/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/templates/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/templates/core-site.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/templates/hadoop
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/templates/hadoop-env.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/templates/hdfs-site.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/templates/mapred-site.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/templates/taskcontroller.cfg
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/lib/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/lib/facter/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/lib/facter/kadm_keytab.rb
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/manifests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/manifests/init.pp
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/templates/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/templates/kadm5.acl
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/templates/kdc.conf
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/templates/krb5.conf
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/tests/
    incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/kerberos/tests/init.pp
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/whirr/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/bigtop-utils/
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/bigtop-utils/LICENSE
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/bigtop-utils/bigtop-detect-javahome
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/whirr/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hbase/hbase.nofiles.conf
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/zookeeper/zoo.cfg
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/bigtop-utils/
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/bigtop-utils/compat
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/bigtop-utils/control
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/bigtop-utils/copyright
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/bigtop-utils/rules   (contents, props changed)
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/zookeeper/rules
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/bigtop-utils/
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/bigtop-utils/RPMS/
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/bigtop-utils/RPMS/.gitignore
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/bigtop-utils/SPECS/
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/bigtop-utils/SPECS/.gitignore
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/bigtop-utils/SPECS/bigtop-utils.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-test-framework/src/main/groovy/org/apache/bigtop/itest/pmanager/UrpmiCmdLinePackageManager.groovy
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/smoke/
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/smoke/IncrementalPELoad.java
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/smoke/TestHBaseCompression.groovy
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/smoke/TestHBasePigSmoke.groovy
      - copied, changed from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/hbasesmoke/TestHbasePigSmoke.groovy
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/smoke/TestHBaseSmoke.java
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/smoke/TestHFileOutputFormat.java
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/smoke/TestLoadIncrementalHFiles.java
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/system/
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/system/TestLoadAndVerify.java
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/util/
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/util/HBaseTestUtil.java
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/package/src/main/resources/package_data_urpmi.xml
Removed:
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/patch
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/src/main/groovy/org/apache/bigtop/itest/hbasesmoke/TestHbasePigSmoke.groovy
Modified:
    incubator/bigtop/branches/hadoop-0.23/README
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/flume/install_flume.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hbase/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hbase/install_hbase.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/hadoop-hive.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/hadoop-hive.sh.suse
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/install_hive.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/mahout/install_mahout.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/oozie/create-package-layout
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/oozie/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/oozie/oozie.init
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/oozie/patch
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/install_pig.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/sqoop/install_sqoop.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/sqoop/sqoop-metastore.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/sqoop/sqoop-metastore.sh.suse
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/whirr/do-component-build
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/whirr/install_whirr.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/zookeeper/hadoop-zookeeper.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/zookeeper/hadoop-zookeeper.sh.suse
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/zookeeper/install_zookeeper.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/flume/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/flume/service-init.d.tpl
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/hbase/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/hbase/hadoop-hbase.install
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/hbase/rules
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/hbase/service-init.d.tpl
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/hive/changelog
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/hive/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/mahout/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/oozie/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/oozie/oozie.postinst
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/pig/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/pig/rules
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/sqoop/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/whirr/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/whirr/rules
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/zookeeper/control
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/zookeeper/hadoop-zookeeper-server.init
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/zookeeper/rules
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/flume/SOURCES/init.d
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/flume/SOURCES/init.d.suse
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/flume/SPECS/flume.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/hbase/SOURCES/hadoop-hbase.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/hbase/SOURCES/hadoop-hbase.sh.suse
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/hbase/SPECS/hbase.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/hive/SOURCES/install_hive.sh
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/hive/SPECS/hive.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/mahout/SPECS/mahout.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/oozie/SPECS/oozie.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/pig/SPECS/pig.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/sqoop/SPECS/sqoop.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/whirr/SPECS/whirr.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/rpm/zookeeper/SPECS/zookeeper.spec
    incubator/bigtop/branches/hadoop-0.23/bigtop-test-framework/src/main/groovy/org/apache/bigtop/itest/pmanager/PackageManager.groovy
    incubator/bigtop/branches/hadoop-0.23/bigtop-test-framework/src/main/groovy/org/apache/bigtop/itest/pmanager/RPMPackage.groovy
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/flume/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hadoop/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hadoop/src/main/groovy/org/apache/bigtop/itest/hadoopexamples/TestHadoopExamples.groovy
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hadoop/src/main/groovy/org/apache/bigtop/itest/hadoopsmoke/TestHadoopSmoke.groovy
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hadoop/src/main/groovy/org/apache/bigtop/itest/hadooptests/TestTestCLI.groovy
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hadoop/src/main/resources/testConfCluster.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/hbase/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/oozie/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-artifacts/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-execution/smokes/hadoop/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-execution/smokes/hbase/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-execution/smokes/hive/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-execution/smokes/pig/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-execution/smokes/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop-tests/test-execution/smokes/sqoop/pom.xml
    incubator/bigtop/branches/hadoop-0.23/bigtop.mk
    incubator/bigtop/branches/hadoop-0.23/package.mk

Modified: incubator/bigtop/branches/hadoop-0.23/README
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/README?rev=1196308&r1=1196307&r2=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/README (original)
+++ incubator/bigtop/branches/hadoop-0.23/README Tue Nov  1 21:53:04 2011
@@ -49,6 +49,21 @@ On Debian-based systems one also needs
 * build-essential dh-make debhelper devscripts
 * reprepro
 
+On openSUSE 11.4+, in addition, one also needs to ensure the following is installed:
+
+
+relaxngDatatype
+docbook-utils docbook-simple
+asciidoc
+fuse-devel
+docbook5
+docbook5-xsl-stylesheets
+libxml2-devel
+xmlformat
+xmlto
+libxslt
+libopenssl-devel
+
 ## Building packages
 
     $ make [component-name]-[rpm|deb]

Added: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/cluster.pp
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/cluster.pp?rev=1196308&view=auto
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/cluster.pp (added)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/cluster.pp Tue Nov  1 21:53:04 2011
@@ -0,0 +1,161 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class hadoop_cluster_node {
+  $hadoop_namenode_host="$hadoop_head_node"
+  $hadoop_namenode_port="17020"
+  $hadoop_namenode_thrift_port="10090"
+  # $hadoop_dfs_namenode_plugins="org.apache.hadoop.thriftfs.NamenodePlugin"
+  # $hadoop_dfs_datanode_plugins="org.apache.hadoop.thriftfs.DatanodePlugin"
+
+  $hadoop_jobtracker_host="$hadoop_head_node"
+  $hadoop_jobtracker_port="8021"
+  $hadoop_jobtracker_thrift_port="9290"
+  # $hadoop_mapred_jobtracker_plugins="org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin"
+  # $hadoop_mapred_tasktracker_plugins="org.apache.hadoop.mapred.TaskTrackerCmonInst"
+
+  $hadoop_core_proxyusers = { oozie => { groups => 'root,hadoop,jenkins,oozie,users', hosts => "${hadoop_head_node},localhost,127.0.0.1" } }
+
+  $hadoop_hbase_rootdir = "hdfs://$hadoop_namenode_host:$hadoop_namenode_port/hbase"
+  $hadoop_hbase_zookeeper_quorum = "$hadoop_head_node"
+
+  $hadoop_zookeeper_ensemble = ["$hadoop_head_node:2888:3888"]
+
+  $namenode_data_dirs = ["/mnt/namenode"]
+  $hdfs_data_dirs = ["/mnt/hdfs"]
+  $mapred_data_dirs = ["/mnt/scratch"]
+
+  if ($hadoop_security_authentication == "kerberos") {
+    $kerberos_domain = "compute-1.internal"
+    $kerberos_realm = "EXAMPLE.COM"
+    $kerberos_kdc_server = "$hadoop_head_node"
+
+    include kerberos::client
+    kerberos::client::host_keytab { ["hdfs", "mapred", "hbase", "oozie"]:
+      princs_map => { hdfs   => [ "host", "hdfs" ],
+                      mapred => [ "mapred" ],
+                      hbase  => [ "hbase"  ],
+                      oozie  => [ "oozie"  ], },
+    }
+  }
+}
+
+class hadoop_worker_node inherits hadoop_cluster_node {
+  hadoop::datanode { "datanode":
+        namenode_host => $hadoop_namenode_host,
+        namenode_port => $hadoop_namenode_port,
+        dirs => $hdfs_data_dirs,
+        auth => $hadoop_security_authentication,
+  }
+
+  hadoop::tasktracker { "tasktracker":
+        namenode_host => $hadoop_namenode_host,
+        namenode_port => $hadoop_namenode_port,
+        jobtracker_host => $hadoop_jobtracker_host,
+        jobtracker_port => $hadoop_jobtracker_port,
+        dirs => $mapred_data_dirs,
+        auth => $hadoop_security_authentication,
+  }
+
+  hadoop-hbase::server { "hbase region server":
+        rootdir => $hadoop_hbase_rootdir,
+        zookeeper_quorum => $hadoop_hbase_zookeeper_quorum,
+        kerberos_realm => $kerberos_realm, 
+  }
+}
+
+class hadoop_head_node inherits hadoop_cluster_node {
+
+  if ($hadoop_security_authentication == "kerberos") {
+    include kerberos::kdc, kerberos::kdc::admin_server
+  }
+
+  hadoop::namenode { "namenode":
+        port => $hadoop_namenode_port,
+        jobtracker_host => $hadoop_jobtracker_host,
+        jobtracker_port => $hadoop_jobtracker_port,
+        dirs => $namenode_data_dirs,
+        # thrift_port => $hadoop_namenode_thrift_port,
+        auth => $hadoop_security_authentication,
+  }
+
+  hadoop::secondarynamenode { "secondary namenode":
+        namenode_host => $hadoop_namenode_host,
+        namenode_port => $hadoop_namenode_port,
+        auth => $hadoop_security_authentication,
+  }
+
+  hadoop::jobtracker { "jobtracker":
+        namenode_host => $hadoop_namenode_host,
+        namenode_port => $hadoop_namenode_port,
+        host => $hadoop_jobtracker_host,
+        port => $hadoop_jobtracker_port,
+        dirs => $mapred_data_dirs,
+        # thrift_port => $hadoop_jobtracker_thrift_port,
+        auth => $hadoop_security_authentication,
+  }
+
+  hadoop-hbase::master { "hbase master":
+        rootdir => $hadoop_hbase_rootdir,
+        zookeeper_quorum => $hadoop_hbase_zookeeper_quorum,
+        kerberos_realm => $kerberos_realm, 
+  }
+
+  hadoop-oozie::server { "oozie server":
+        kerberos_realm => $kerberos_realm, 
+  }
+
+  hadoop-zookeeper::server { "zookeeper":
+        myid => "0",
+        ensemble => $hadoop_zookeeper_ensemble,
+  }
+
+  hadoop::create_hdfs_dirs { [ "/mapred", "/tmp", "/system", "/user", "/hbase", "/benchmarks", "/user/jenkins", "/user/hive" ]:
+    hdfs_dirs_meta => { "/tmp"          => { perm => "777", user => "hdfs"   },
+                        "/mapred"       => { perm => "755", user => "mapred" },
+                        "/system"       => { perm => "755", user => "hdfs"   },
+                        "/user"         => { perm => "755", user => "hdfs"   },
+                        "/hbase"        => { perm => "755", user => "hbase"  },
+                        "/benchmarks"   => { perm => "777", user => "hdfs"   },
+                        "/user/jenkins" => { perm => "777", user => "jenkins"},
+                        "/user/hive"    => { perm => "777", user => "hive"   } },
+  }
+}
+
+class hadoop_gateway_node inherits hadoop_head_node {
+  # hadoop::client { "gateway":
+  #   namenode_host => $hadoop_namenode_host,
+  #   namenode_port => $hadoop_namenode_port,
+  #   jobtracker_host => $hadoop_jobtracker_host,
+  #   jobtracker_port => $hadoop_jobtracker_port,
+  #   # auth => $hadoop_security_authentication,
+  # }
+
+  hadoop-pig::client { "pig client":
+  }
+  hadoop-hive::client { "hive client":
+     hbase_zookeeper_quorum => $hadoop_hbase_zookeeper_quorum,
+  }
+  hadoop-sqoop::client { "sqoop client":
+  }
+  hadoop-oozie::client { "oozie client":
+  }
+  hadoop-hbase::client { "hbase client":
+  }
+  hadoop-zookeeper::client { "zookeeper client":
+  }
+  hadoop-flume::client { "flume client":
+  }
+}

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/site.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/site.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/site.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/manifests/site.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,8 +13,28 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
+stage {"pre": before => Stage["main"]}
 
-ant -Djavac.version=1.6 -Djava5.home=${JAVA5_HOME} -Dforrest.home=${FORREST_HOME} -Ddist.dir=debian/tmp -Dversion=${PIG_BASE_VERSION} package "$@"
+yumrepo { "Bigtop":
+    baseurl => "http://bigtop01.cloudera.org:8080/job/Bigtop-trunk-centos5/lastSuccessfulBuild/artifact/output/",
+    descr => "Bigtop packages",
+    enabled => 1,
+    gpgcheck => 0,
+}
 
+package { "jdk":
+   ensure => "installed",
+}
 
+# $hadoop_head_node="beefy.my.org"
+# $hadoop_security_authentication="kerberos"
+
+import "cluster.pp"
+
+node default {
+  if ($hadoop_head_node == $fqdn) {
+    include hadoop_head_node
+  } else {
+    include hadoop_worker_node
+  }
+}

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/manifests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/sqoop/control)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/manifests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/manifests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/sqoop/control&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/sqoop/control (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/manifests/init.pp Tue Nov  1 21:53:04 2011
@@ -12,24 +12,40 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-Source: sqoop
-Section: misc
-Priority: extra
-Maintainer: Alex Newman <newalex@cloudera.com>
-Build-Depends: debhelper (>= 6), ant, ant-optional, git-core, sed, asciidoc, xmlto
-Standards-Version: 3.8.0
-Homepage: http://www.cloudera.com
 
-Package:  sqoop
-Architecture: all
-Depends: hadoop
-Description: Tool for easy imports and exports of data sets between databases and HDFS
- Sqoop is a tool that provides the ability to import and export data sets between
- the Hadoop Distributed File System (HDFS) and relational databases.
+class hadoop-flume {
+  define client {
+    package { "flume":
+      ensure => latest,
+    } 
+  }
 
-Package: sqoop-metastore
-Architecture: all
-Depends: sqoop (= ${source:Version}), adduser
-Description: Shared metadata repository for Sqoop.
- This optional package hosts a metadata server for Sqoop clients across a network to use.
+  # It really is flume node, but node is a reserved keyword for puppet
+  define agent {
+    package { "flume-node":
+      ensure => latest,
+    } 
 
+    service { "flume-node":
+      ensure => running,
+      require => Package["flume-node"],
+      # FIXME: this need to be fixed in upstream flume
+      hasstatus => false,
+      hasrestart => true,
+    }
+  }
+
+  define master {
+    package { "flume-master":
+      ensure => latest,
+    } 
+
+    service { "flume-master":
+      ensure => running,
+      require => Package["flume-node"],
+      # FIXME: this need to be fixed in upstream flume
+      hasstatus => true,
+      hasrestart => true,
+    }
+  }
+}

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/tests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/whirr/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/tests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/tests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/whirr/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/whirr/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-flume/tests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,7 +13,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
+hadoop-flume::client { "test-flume": } 
+hadoop-flume::agent  { "test-flume-node": } 
+hadoop-flume::master { "test-flume-master": }
 
-mvn clean source:jar install assembly:assembly -Pjavadoc site "$@"
 

Added: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/manifests/init.pp
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/manifests/init.pp?rev=1196308&view=auto
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/manifests/init.pp (added)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/manifests/init.pp Tue Nov  1 21:53:04 2011
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class hadoop-hbase {
+  class client-package  {
+    package { "hadoop-hbase":
+      ensure => latest,
+    } 
+  }
+
+  class common-server-config {
+    include client-package
+
+    file { "/etc/hbase/conf/hbase-site.xml":
+      content => template("hadoop-hbase/hbase-site.xml"),
+      require => Package["hadoop-hbase"],
+    }
+    file { "/etc/hbase/conf/hbase-env.sh":
+      content => template("hadoop-hbase/hbase-env.sh"),
+      require => Package["hadoop-hbase"],
+    }
+  }
+
+  define client {
+    include client-package
+  }
+
+  define server($rootdir, $zookeeper_quorum, $kerberos_realm = "") {
+    include common-server-config
+
+    package { "hadoop-hbase-regionserver":
+      ensure => latest,
+    }
+
+    service { "hadoop-hbase-regionserver":
+      ensure => running,
+      require => Package["hadoop-hbase-regionserver"],
+      subscribe => File["/etc/hbase/conf/hbase-site.xml", "/etc/hbase/conf/hbase-env.sh"],
+      hasrestart => true,
+      hasstatus => true,
+    } 
+  }
+
+  define master($rootdir, $zookeeper_quorum, $kerberos_realm = "") {
+    include common-server-config
+
+    package { "hadoop-hbase-master":
+      ensure => latest,
+    }
+
+    service { "hadoop-hbase-master":
+      ensure => running,
+      require => Package["hadoop-hbase-master"],
+      subscribe => File["/etc/hbase/conf/hbase-site.xml", "/etc/hbase/conf/hbase-env.sh"],
+      hasrestart => true,
+      hasstatus => true,
+    } 
+  }
+}

Added: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-env.sh
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-env.sh?rev=1196308&view=auto
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-env.sh (added)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-env.sh Tue Nov  1 21:53:04 2011
@@ -0,0 +1,76 @@
+#
+#/**
+# * Copyright 2007 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements.  See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership.  The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License.  You may obtain a copy of the License at
+# *
+# *     http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
+
+# Set environment variables here.
+
+# The java implementation to use.  Java 1.6 required.
+# export JAVA_HOME=/usr/java/jdk1.6.0/
+
+# Extra Java CLASSPATH elements.  Optional.
+export HBASE_CLASSPATH=/etc/hadoop/conf
+
+# The maximum amount of heap to use, in MB. Default is 1000.
+# export HBASE_HEAPSIZE=1000
+
+# Extra Java runtime options.
+# Below are what we set by default.  May only work with SUN JVM.
+# For more on why as well as other possible settings,
+# see http://wiki.apache.org/hadoop/PerformanceTuning
+export HBASE_OPTS="$HBASE_OPTS -ea -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode"
+
+# Uncomment below to enable java garbage collection logging.
+# export HBASE_OPTS="$HBASE_OPTS -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$HBASE_HOME/logs/gc-hbase.log" 
+
+# Uncomment and adjust to enable JMX exporting
+# See jmxremote.password and jmxremote.access in $JRE_HOME/lib/management to configure remote password access.
+# More details at: http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html
+#
+# export HBASE_JMX_BASE="-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
+# export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10101"
+# export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10102"
+# export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10103"
+# export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10104"
+
+# File naming hosts on which HRegionServers will run.  $HBASE_HOME/conf/regionservers by default.
+# export HBASE_REGIONSERVERS=${HBASE_HOME}/conf/regionservers
+
+# Extra ssh options.  Empty by default.
+# export HBASE_SSH_OPTS="-o ConnectTimeout=1 -o SendEnv=HBASE_CONF_DIR"
+
+# Where log files are stored.  $HBASE_HOME/logs by default.
+# export HBASE_LOG_DIR=${HBASE_HOME}/logs
+
+# A string representing this instance of hbase. $USER by default.
+# export HBASE_IDENT_STRING=$USER
+
+# The scheduling priority for daemon processes.  See 'man nice'.
+# export HBASE_NICENESS=10
+
+# The directory where pid files are stored. /tmp by default.
+# export HBASE_PID_DIR=/var/hadoop/pids
+
+# Seconds to sleep between slave commands.  Unset by default.  This
+# can be useful in large clusters, where, e.g., slave rsyncs can
+# otherwise arrive faster than the master can service them.
+# export HBASE_SLAVE_SLEEP=0.1
+
+# Tell HBase whether it should manage it's own instance of Zookeeper or not.
+# export HBASE_MANAGES_ZK=true

Added: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-site.xml
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-site.xml?rev=1196308&view=auto
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-site.xml (added)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/templates/hbase-site.xml Tue Nov  1 21:53:04 2011
@@ -0,0 +1,58 @@
+<?xml version="1.0"?>
+<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+<!--
+/**
+ * Copyright 2010 The Apache Software Foundation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+<configuration>
+  <property>
+    <name>hbase.cluster.distributed</name>
+    <value>true</value>
+  </property>
+
+  <property>
+    <name>hbase.zookeeper.quorum</name>
+    <value><%= zookeeper_quorum %></value>
+  </property>
+
+  <property>
+    <name>hbase.rootdir</name>
+    <value><%= rootdir %></value>
+  </property>
+
+<% if kerberos_realm != "" %>
+  <property> 
+    <name>hbase.regionserver.kerberos.principal</name> 
+    <value>hbase/_HOST@<%= kerberos_realm %></value> 
+    </property> 
+  <property> 
+    <name>hbase.regionserver.keytab.file</name> 
+    <value>/etc/hbase.keytab</value> 
+  </property>
+  <property> 
+    <name>hbase.master.kerberos.principal</name> 
+    <value>hbase/_HOST@<%= kerberos_realm %></value> 
+  </property> 
+  <property> 
+    <name>hbase.master.keytab.file</name> 
+    <value>/etc/hbase.keytab</value> 
+  </property>
+<% end %>
+</configuration>

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/tests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/tests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/tests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hbase/tests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,6 +13,14 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
-
-ant -f src/build.xml package "$@"
+hadoop-hbase::client { "hbase shell": } 
+hadoop-hbase::master { "test-hbase-master": 
+  rootdir => "hdfs://localhost:17020/hbase",
+  zookeeper_quorum => "localhost",
+  kerberos_realm => "KRB.YOU.ORG",
+}
+hadoop-hbase::server { "test-hbase-server": 
+  rootdir => "hdfs://localhost:17020/hbase",
+  zookeeper_quorum => "localhost",
+  kerberos_realm => "KRB.YOU.ORG",
+}

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/manifests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/manifests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/manifests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/manifests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,6 +13,15 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
+class hadoop-hive {
+  define client($hbase_master = "", $hbase_zookeeper_quorum = "") {
+    package { "hadoop-hive":
+      ensure => latest,
+    } 
 
-ant -f src/build.xml package "$@"
+    file { "/etc/hive/conf/hive-site.xml":
+      content => template('hadoop-hive/hive-site.xml'),
+      require => Package["hadoop-hive"],
+    }
+  }
+}

Added: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/templates/hive-site.xml
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/templates/hive-site.xml?rev=1196308&view=auto
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/templates/hive-site.xml (added)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/templates/hive-site.xml Tue Nov  1 21:53:04 2011
@@ -0,0 +1,63 @@
+<?xml version="1.0"?>
+<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+
+<!-- Licensed to the Apache Software Foundation (ASF) under one or more       -->
+<!-- contributor license agreements.  See the NOTICE file distributed with    -->
+<!-- this work for additional information regarding copyright ownership.      -->
+<!-- The ASF licenses this file to You under the Apache License, Version 2.0  -->
+<!-- (the "License"); you may not use this file except in compliance with     -->
+<!-- the License.  You may obtain a copy of the License at                    -->
+<!--                                                                          -->
+<!--     http://www.apache.org/licenses/LICENSE-2.0                           -->
+<!--                                                                          -->
+<!-- Unless required by applicable law or agreed to in writing, software      -->
+<!-- distributed under the License is distributed on an "AS IS" BASIS,        -->
+<!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -->
+<!-- See the License for the specific language governing permissions and      -->
+<!-- limitations under the License.                                           -->
+
+<configuration>
+
+<!-- Hive Configuration can either be stored in this file or in the hadoop configuration files  -->
+<!-- that are implied by Hadoop setup variables.                                                -->
+<!-- Aside from Hadoop setup variables - this file is provided as a convenience so that Hive    -->
+<!-- users do not have to edit hadoop configuration files (that may be managed as a centralized -->
+<!-- resource).                                                                                 -->
+
+<!-- Hive Execution Parameters -->
+
+<% if hbase_master != "" %>
+<property>
+  <name>hbase.master</name>
+  <value><%= hbase_master %></value>
+  <description>http://wiki.apache.org/hadoop/Hive/HBaseIntegration</description>
+</property>
+<% end %>
+
+<% if hbase_zookeeper_quorum != "" %>
+<property>
+  <name>hbase.zookeeper.quorum</name>
+  <value><%= hbase_zookeeper_quorum %></value>
+  <description>http://wiki.apache.org/hadoop/Hive/HBaseIntegration</description>
+</property>
+<% end %>
+
+<property>
+  <name>javax.jdo.option.ConnectionURL</name>
+  <value>jdbc:derby:;databaseName=/var/lib/hive/metastore/metastore_db;create=true</value>
+  <description>JDBC connect string for a JDBC metastore</description>
+</property>
+
+<property>
+  <name>javax.jdo.option.ConnectionDriverName</name>
+  <value>org.apache.derby.jdbc.EmbeddedDriver</value>
+  <description>Driver class name for a JDBC metastore</description>
+</property>
+
+<property>
+  <name>hive.hwi.war.file</name>
+  <value>/usr/lib/hive/lib/hive-hwi-0.7.0-cdh3u1-SNAPSHOT.war</value>
+  <description>This is the WAR file with the jsp content for Hive Web Interface</description>
+</property>
+
+</configuration>

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/tests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/tests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/tests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-hive/tests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,6 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
-
-ant -f src/build.xml package "$@"
+hadoop-hive::client { "test-hive": } 

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/manifests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/manifests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/manifests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/manifests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,8 +13,29 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
+class hadoop-oozie {
+  define client($kerberos_realm = "") {
+    package { "oozie-client":
+      ensure => latest,
+    } 
+  }
 
-ant -Djavac.version=1.6 -Djava5.home=${JAVA5_HOME} -Dforrest.home=${FORREST_HOME} -Ddist.dir=debian/tmp -Dversion=${PIG_BASE_VERSION} package "$@"
+  define server($kerberos_realm = "") {
+    package { "oozie":
+      ensure => latest,
+    }
 
+    file { "/etc/oozie/oozie-site.xml":
+      content => template("hadoop-oozie/oozie-site.xml"),
+      require => Package["oozie"],
+    }
 
+    service { "oozie":
+      ensure => running,
+      require => Package["oozie"],
+      hasrestart => true,
+      hasstatus => true,
+    } 
+
+  }
+}

Added: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/templates/oozie-site.xml
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/templates/oozie-site.xml?rev=1196308&view=auto
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/templates/oozie-site.xml (added)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/templates/oozie-site.xml Tue Nov  1 21:53:04 2011
@@ -0,0 +1,298 @@
+<?xml version="1.0"?>
+<!--
+  Copyright (c) 2010 Yahoo! Inc. All rights reserved.
+  Licensed under the Apache License, Version 2.0 (the "License");
+  you may not use this file except in compliance with the License.
+  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License. See accompanying LICENSE file.
+-->
+
+<configuration>
+
+<!--
+    Refer to the oozie-default.xml file for the complete list of
+    Oozie configuration properties and their default values.
+-->
+
+    <property>
+        <name>oozie.system.id</name>
+        <value>oozie-${user.name}</value>
+        <description>
+            The Oozie system ID.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.systemmode</name>
+        <value>NORMAL</value>
+        <description>
+            System mode for  Oozie at startup.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.AuthorizationService.security.enabled</name>
+        <value>false</value>
+        <description>
+            Specifies whether security (user name/admin role) is enabled or not.
+            If disabled any user can manage Oozie system and manage any job.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.PurgeService.older.than</name>
+        <value>30</value>
+        <description>
+            Jobs older than this value, in days, will be purged by the PurgeService.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.PurgeService.purge.interval</name>
+        <value>3600</value>
+        <description>
+            Interval at which the purge service will run, in seconds.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.CallableQueueService.queue.size</name>
+        <value>10000</value>
+        <description>Max callable queue size</description>
+    </property>
+
+    <property>
+        <name>oozie.service.CallableQueueService.threads</name>
+        <value>10</value>
+        <description>Number of threads used for executing callables</description>
+    </property>
+
+    <property>
+        <name>oozie.service.CallableQueueService.callable.concurrency</name>
+        <value>3</value>
+        <description>
+            Maximum concurrency for a given callable type.
+            Each command is a callable type (submit, start, run, signal, job, jobs, suspend,resume, etc).
+            Each action type is a callable type (Map-Reduce, Pig, SSH, FS, sub-workflow, etc).
+            All commands that use action executors (action-start, action-end, action-kill and action-check) use
+            the action type as the callable type.
+        </description>
+    </property>
+
+    <property>
+		<name>oozie.service.coord.normal.default.timeout
+		</name>
+		<value>120</value>
+		<description>Default timeout for a coordinator action input check (in minutes) for normal job.
+            -1 means infinite timeout</description>
+	</property>
+
+    <property>
+        <name>oozie.db.schema.name</name>
+        <value>oozie</value>
+        <description>
+            Oozie DataBase Name
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.StoreService.create.db.schema</name>
+        <value>true</value>
+        <description>
+            Creates Oozie DB.
+
+            If set to true, it creates the DB schema if it does not exist. If the DB schema exists is a NOP.
+            If set to false, it does not create the DB schema. If the DB schema does not exist it fails start up.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.StoreService.jdbc.driver</name>
+        <value>org.apache.derby.jdbc.EmbeddedDriver</value>
+        <description>
+            JDBC driver class.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.StoreService.jdbc.url</name>
+        <value>jdbc:derby:${oozie.data.dir}/${oozie.db.schema.name}-db;create=true</value>
+        <description>
+            JDBC URL.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.StoreService.jdbc.username</name>
+        <value>sa</value>
+        <description>
+            DB user name.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.StoreService.jdbc.password</name>
+        <value> </value>
+        <description>
+            DB user password.
+
+            IMPORTANT: if password is emtpy leave a 1 space string, the service trims the value,
+                       if empty Configuration assumes it is NULL.
+
+            IMPORTANT: if the StoreServicePasswordService is active, it will reset this value with the value given in
+                       the console.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.StoreService.pool.max.active.conn</name>
+        <value>10</value>
+        <description>
+             Max number of connections.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.HadoopAccessorService.kerberos.enabled</name>
+        <value><%= if (kerberos_realm != "") ; "true" else "false" end %></value>
+        <description>
+            Indicates if Oozie is configured to use Kerberos.
+        </description>
+    </property>
+
+    <property>
+        <name>local.realm</name>
+        <value><%= kerberos_realm %></value>
+        <description>
+            Kerberos Realm used by Oozie and Hadoop. Using 'local.realm' to be aligned with Hadoop configuration
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.HadoopAccessorService.keytab.file</name>
+        <value>/etc/oozie.keytab</value>
+        <description>
+            Location of the Oozie user keytab file.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.HadoopAccessorService.kerberos.principal</name>
+        <value>${user.name}/<%= fqdn %>@${local.realm}</value>
+        <description>
+            Kerberos principal for Oozie service.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.HadoopAccessorService.jobTracker.whitelist</name>
+        <value> </value>
+        <description>
+            Whitelisted job tracker for Oozie service.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.HadoopAccessorService.nameNode.whitelist</name>
+        <value> </value>
+        <description>
+            Whitelisted job tracker for Oozie service.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.service.WorkflowAppService.system.libpath</name>
+        <value>/user/${user.name}/share/lib</value>
+        <description>
+            System library path to use for workflow applications.
+            This path is added to workflow application if their job properties sets
+            the property 'oozie.use.system.libpath' to true.
+        </description>
+    </property>
+
+    <property>
+        <name>use.system.libpath.for.mapreduce.and.pig.jobs</name>
+        <value>false</value>
+        <description>
+            If set to true, submissions of MapReduce and Pig jobs will include
+            automatically the system library path, thus not requiring users to
+            specify where the Pig JAR files are. Instead, the ones from the system
+            library path are used.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.authentication.type</name>
+        <value>simple</value>
+        <description>
+            Defines authentication used for Oozie HTTP endpoint.
+            Supported values are: simple | kerberos | #AUTHENTICATION_HANDLER_CLASSNAME#
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.authentication.token.validity</name>
+        <value>36000</value>
+        <description>
+            Indicates how long (in seconds) an authentication token is valid before it has
+            to be renewed.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.authentication.signature.secret</name>
+        <value>oozie</value>
+        <description>
+            The signature secret for signing the authentication tokens.
+            If not set a random secret is generated at startup time.
+            In order to authentiation to work correctly across multiple hosts
+            the secret must be the same across al the hosts.
+        </description>
+    </property>
+
+    <property>
+      <name>oozie.authentication.cookie.domain</name>
+      <value></value>
+      <description>
+        The domain to use for the HTTP cookie that stores the authentication token.
+        In order to authentiation to work correctly across multiple hosts
+        the domain must be correctly set.
+      </description>
+    </property>
+
+    <property>
+        <name>oozie.authentication.simple.anonymous.allowed</name>
+        <value>true</value>
+        <description>
+            Indicates if anonymous requests are allowed.
+            This setting is meaningful only when using 'simple' authentication.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.authentication.kerberos.principal</name>
+        <value>HTTP/<%= fqdn %>@${local.realm}</value>
+        <description>
+            Indicates the Kerberos principal to be used for HTTP endpoint.
+            The principal MUST start with 'HTTP/' as per Kerberos HTTP SPNEGO specification.
+        </description>
+    </property>
+
+    <property>
+        <name>oozie.authentication.kerberos.keytab</name>
+        <value>${oozie.service.HadoopAccessorService.keytab.file}</value>
+        <description>
+            Location of the keytab file with the credentials for the principal.
+            Referring to the same keytab file Oozie uses for its Kerberos credentials for Hadoop.
+        </description>
+    </property>
+
+</configuration>
+

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/tests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/tests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/tests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-oozie/tests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,6 +13,5 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
-
-ant -f src/build.xml package "$@"
+hadoop-oozie::server { "test-oozie-server": }
+hadoop-oozie::client { "test-oozie": } 

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/manifests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/manifests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/manifests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/manifests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,8 +13,19 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
-
-ant -Djavac.version=1.6 -Djava5.home=${JAVA5_HOME} -Dforrest.home=${FORREST_HOME} -Ddist.dir=debian/tmp -Dversion=${PIG_BASE_VERSION} package "$@"
-
-
+class hadoop-pig {
+  define client {
+    package { "hadoop-pig":
+      ensure => latest,
+      require => Package["hadoop"],
+    } 
+ 
+    file { "/etc/pig/conf/pig.properties":
+      content => template('hadoop-pig/pig.properties'),
+      require => Package["hadoop-pig"],
+      owner => "root", /* FIXME: I'm really no sure about these -- we might end  */
+      mode => "755",   /*        up deploying/testing a different thing compared */
+                       /*        to a straight rpm/deb deployment                */
+    }
+  }
+}

Added: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/templates/pig.properties
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/templates/pig.properties?rev=1196308&view=auto
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/templates/pig.properties (added)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/templates/pig.properties Tue Nov  1 21:53:04 2011
@@ -0,0 +1,70 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+# Pig configuration file. All values can be overwritten by command line arguments.
+# see bin/pig -help
+
+# log4jconf log4j configuration file
+# log4jconf=./conf/log4j.properties
+
+# brief logging (no timestamps)
+brief=false
+
+# clustername, name of the hadoop jobtracker. If no port is defined port 50020 will be used. 
+#cluster
+
+#debug level, INFO is default
+debug=INFO
+
+# a file that contains pig script
+#file=
+
+# load jarfile, colon separated
+#jar=
+
+#verbose print all log messages to screen (default to print only INFO and above to screen)
+verbose=false
+
+#exectype local|mapreduce, mapreduce is default
+#exectype=mapreduce
+# hod realted properties
+#ssh.gateway
+#hod.expect.root
+#hod.expect.uselatest
+#hod.command
+#hod.config.dir
+#hod.param
+
+
+#Do not spill temp files smaller than this size (bytes)
+pig.spill.size.threshold=5000000
+#EXPERIMENT: Activate garbage collection when spilling a file bigger than this size (bytes)
+#This should help reduce the number of files being spilled.
+pig.spill.gc.activation.size=40000000
+
+
+######################
+# Everything below this line is Yahoo specific.  Note that I've made
+# (almost) no changes to the lines above to make merging in from Apache
+# easier.  Any values I don't want from above I override below.
+#
+# This file is configured for use with HOD on the production clusters.  If you
+# want to run pig with a static cluster you will need to remove everything
+# below this line and set the cluster value (above) to the
+# hostname and port of your job tracker.
+
+exectype=mapreduce
+log.file=

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/tests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/tests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/tests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-pig/tests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,6 +13,5 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
-
-ant -f src/build.xml package "$@"
+include pig
+pig::client { "test-pig": } 

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/manifests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/manifests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/manifests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/manifests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,8 +13,30 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
+class hadoop-sqoop {
+  define client {
+    package { "sqoop":
+      ensure => latest,
+    } 
 
-ant -Djavac.version=1.6 -Djava5.home=${JAVA5_HOME} -Dforrest.home=${FORREST_HOME} -Ddist.dir=debian/tmp -Dversion=${PIG_BASE_VERSION} package "$@"
+    # FIXME: this is NOT supposed to be needed
+    #        but for now that's the easiest way to add
+    #        extra jars to sqoop classpath
+    file { "/usr/lib/sqoop/lib/mysql-connector-java-5.1.12.jar":
+      ensure => "/usr/share/java/mysql-connector-java-5.1.12.jar",
+    }
+  }
 
+  define metastore {
+    package { "sqoop-metastore":
+      ensure => latest,
+    } 
 
+    service { "sqoop-metastore":
+      ensure => running,
+      require => Package["sqoop-metastore"],
+      hasstatus => true,
+      hasrestart => true,
+    }
+  }
+}

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/tests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/tests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/tests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-sqoop/tests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,6 +13,5 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
-
-ant -f src/build.xml package "$@"
+hadoop-sqoop::client { "test-sqoop": } 
+hadoop-sqoop::metastore  { "test-sqoop-metastore": } 

Added: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/manifests/init.pp
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/manifests/init.pp?rev=1196308&view=auto
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/manifests/init.pp (added)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/manifests/init.pp Tue Nov  1 21:53:04 2011
@@ -0,0 +1,46 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class hadoop-zookeeper {
+  define client {
+    package { "hadoop-zookeeper":
+      ensure => latest,
+    } 
+  }
+
+  define server($myid, $ensemble = ["localhost:2888:3888"]) {
+    package { "hadoop-zookeeper-server":
+      ensure => latest,
+    }
+
+    service { "hadoop-zookeeper-server":
+      ensure => running,
+      require => Package["hadoop-zookeeper-server"],
+      subscribe => File["/etc/zookeeper/conf/zoo.cfg"],
+      hasrestart => true,
+      hasstatus => true,
+    } 
+
+    file { "/etc/zookeeper/conf/zoo.cfg":
+      content => template("hadoop-zookeeper/zoo.cfg"),
+      require => Package["hadoop-zookeeper-server"],
+    }
+
+    file { "/var/lib/zookeeper/myid":
+      content => inline_template("<%= myid %>"),
+      require => Package["hadoop-zookeeper-server"],
+    }
+  }
+}

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/templates/zoo.cfg (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/templates/zoo.cfg?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/templates/zoo.cfg&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/pig/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/templates/zoo.cfg Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,8 +13,19 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
-
-ant -Djavac.version=1.6 -Djava5.home=${JAVA5_HOME} -Dforrest.home=${FORREST_HOME} -Ddist.dir=debian/tmp -Dversion=${PIG_BASE_VERSION} package "$@"
-
-
+maxClientCnxns=50
+# The number of milliseconds of each tick
+tickTime=2000
+# The number of ticks that the initial 
+# synchronization phase can take
+initLimit=10
+# The number of ticks that can pass between 
+# sending a request and getting an acknowledgement
+syncLimit=5
+# the directory where the snapshot is stored.
+dataDir=/var/lib/zookeeper
+# the port at which the clients will connect
+clientPort=2181
+<% ensemble.each_with_index do |server,idx| %>
+server.<%= idx %>=<%= server %>
+<% end %>

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/tests/init.pp (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/tests/init.pp?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/tests/init.pp&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/common/hive/do-component-build (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop-zookeeper/tests/init.pp Tue Nov  1 21:53:04 2011
@@ -1,4 +1,3 @@
-#!/bin/sh
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -14,6 +13,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-set -ex
-
-ant -f src/build.xml package "$@"
+hadoop-zookeeper::client { "zoo visitor": } 
+hadoop-zookeeper::server { "test-oozie-server": 
+  myid => "0",
+  ensemble => ["foo:2888:3888", "bar:2888:3888", "baz:2888:3888"],
+}

Copied: incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/lib/facter/hadoop_storage_locations.rb (from r1196246, incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/sqoop/control)
URL: http://svn.apache.org/viewvc/incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/lib/facter/hadoop_storage_locations.rb?p2=incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/lib/facter/hadoop_storage_locations.rb&p1=incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/sqoop/control&r1=1196246&r2=1196308&rev=1196308&view=diff
==============================================================================
--- incubator/bigtop/branches/hadoop-0.23/bigtop-packages/src/deb/sqoop/control (original)
+++ incubator/bigtop/branches/hadoop-0.23/bigtop-deploy/puppet/modules/hadoop/lib/facter/hadoop_storage_locations.rb Tue Nov  1 21:53:04 2011
@@ -12,24 +12,29 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-Source: sqoop
-Section: misc
-Priority: extra
-Maintainer: Alex Newman <newalex@cloudera.com>
-Build-Depends: debhelper (>= 6), ant, ant-optional, git-core, sed, asciidoc, xmlto
-Standards-Version: 3.8.0
-Homepage: http://www.cloudera.com
-
-Package:  sqoop
-Architecture: all
-Depends: hadoop
-Description: Tool for easy imports and exports of data sets between databases and HDFS
- Sqoop is a tool that provides the ability to import and export data sets between
- the Hadoop Distributed File System (HDFS) and relational databases.
-
-Package: sqoop-metastore
-Architecture: all
-Depends: sqoop (= ${source:Version}), adduser
-Description: Shared metadata repository for Sqoop.
- This optional package hosts a metadata server for Sqoop clients across a network to use.
+
+# We make the assumption hadoop's data files will be located in /data/
+# Puppet needs to know where they are
+Facter.add("hadoop_storage_locations") do
+        setcode do
+
+            data_dir_path = "/data/"
+            storage_locations = ""
+
+            # We need to check /data/ exist
+            if File.directory?(data_dir_path)
+
+              # We assume all data directory will be a number
+              Dir.foreach(data_dir_path) { |directory|
+                  storage_locations += (data_dir_path + directory + ';') if directory =~ /\d+/
+              }
+            end
+
+            # Return the list of storage locations for hadoop
+            if storage_locations == ""
+              storage_locations = "/mnt"
+            end
+            storage_locations
+        end
+end
 



Mime
View raw message