Return-Path: Delivered-To: apmail-pig-commits-archive@www.apache.org Received: (qmail 22949 invoked from network); 9 Dec 2010 22:43:17 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 9 Dec 2010 22:43:17 -0000 Received: (qmail 42862 invoked by uid 500); 9 Dec 2010 22:43:17 -0000 Delivered-To: apmail-pig-commits-archive@pig.apache.org Received: (qmail 42843 invoked by uid 500); 9 Dec 2010 22:43:16 -0000 Mailing-List: contact commits-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list commits@pig.apache.org Received: (qmail 42836 invoked by uid 99); 9 Dec 2010 22:43:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Dec 2010 22:43:16 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Dec 2010 22:43:15 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 5D03F238897A; Thu, 9 Dec 2010 22:42:55 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: svn commit: r1044156 - in /pig/trunk: CHANGES.txt KEYS build.xml src/docs/src/documentation/content/xdocs/tutorial.xml src/docs/src/documentation/content/xdocs/udf.xml src/docs/src/documentation/content/xdocs/zebra_overview.xml Date: Thu, 09 Dec 2010 22:42:55 -0000 To: commits@pig.apache.org From: olga@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20101209224255.5D03F238897A@eris.apache.org> Author: olga Date: Thu Dec 9 22:42:54 2010 New Revision: 1044156 URL: http://svn.apache.org/viewvc?rev=1044156&view=rev Log: PIG-1728: doc updates (chandec via olgan) Modified: pig/trunk/CHANGES.txt pig/trunk/KEYS pig/trunk/build.xml pig/trunk/src/docs/src/documentation/content/xdocs/tutorial.xml pig/trunk/src/docs/src/documentation/content/xdocs/udf.xml pig/trunk/src/docs/src/documentation/content/xdocs/zebra_overview.xml Modified: pig/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/pig/trunk/CHANGES.txt?rev=1044156&r1=1044155&r2=1044156&view=diff ============================================================================== --- pig/trunk/CHANGES.txt (original) +++ pig/trunk/CHANGES.txt Thu Dec 9 22:42:54 2010 @@ -24,6 +24,8 @@ INCOMPATIBLE CHANGES IMPROVEMENTS +PIG-1728: doc updates (chandec via olgan) + PIG-1752: Enable UDFs to indicate files to load into the Distributed Cache (gates) Modified: pig/trunk/KEYS URL: http://svn.apache.org/viewvc/pig/trunk/KEYS?rev=1044156&r1=1044155&r2=1044156&view=diff ============================================================================== --- pig/trunk/KEYS (original) +++ pig/trunk/KEYS Thu Dec 9 22:42:54 2010 @@ -6,27 +6,28 @@ sig 06687D96 2008-07-14 Olga N -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.2.6 (GNU/Linux) -mQGiBEh7rwMRBACRWYBxkUT3jICHIbgSxCpPkEzr8aXwWlRu0dAKXxj9nklyfqz8 -R5XGaD7W1HfyvmQzO1g0oQ15JFXPIhgNJ5ZbZptq/YaQf0qWAK2RRjFZNyQIg9KF -nfFX6zdU2e5YIrx/aTQezOw6ThIUHE5hOP/vo6xo4IhjDjRBZxqtmFOtDwCgvklj -b8issjgf0jwFH9mB3qTCDGMD/3cZTjU3gx+M3e+/7g52BK16zbLRCy3FETV2kxAI -U0ghF032kENuuHzU9l1sGAeTSg9i+927ohg0zx+WP+HPgGZKY8CJLyswWmPWmAge -ZJmXVlCad84/X7GD5lu1t3Ueibnw9ZWKH+oCG5mKJpBiklyyzk1OIwwIEsdsIeOF -MrBFA/sF/cRDNWJWFVgb0jx0pLTkTN8SSePvklElIPfuYPmNj0fG95cQlXmooDbF -5RHHG8WRuECrrzJTSpiqQ/7AFVnRWdY6Mb1IhVudDEyGSlUYCtXC63nbPWomTKOd -NII92gS2DCBCXU+CWhsmTDsPyxbpA8sPGxzASJgoXvgVP1b4EbQkT2xnYSBOYXRr -b3ZpY2ggPG9sZ2FuQHlhaG9vLWluYy5jb20+iF4EExECAB4FAkh7rwMCGwMGCwkI -BwMCAxUCAwMWAgECHgECF4AACgkQ02eP0wZofZZUfQCgp+nujZ+/7uzlgNYecjfp -nKyxZ6AAoJm9hjNpTYnzCKSmXRKh2Gdo8N8+uQENBEh7rwQQBACqUm8KbPN/a1Yt -aG6Hlcj1Zx5jnbCZ+PYzkKN4LAvnmBHCgJUsliEiJ5XGVOVIbMyVT9+Ivwei81Px -F95HeBarD7cISOlnbGHLME7RCpMFx36I5YR0ZFAYtGS8q1YM6nUN8hqmRVlBYC/7 -D7j9PQ/cM6Z/jS1jJ1q7XIFXzPIplwADBQQAgHMMryaVQvN/7zhrgN5E9tLLeCCF -b7Al3yfZteuqg3hTU2fuHPuw9SwdL8whbglmKsGylZJ7ULdq1EZQztuGbQ1CwVVb -QWeLnLktB1Q2F0x2YqdtVi3/+uj5rXYOxcKns0iOS3y6mKccpuEk6CUiFMtLh7NB -+bQz/IDHO3Ql+vGISQQYEQIACQUCSHuvBAIbDAAKCRDTZ4/TBmh9lqZHAJsHomGp -LMYTpbXVonowdEk4YL45jQCfY2/fXDhD2IwYB+PGmdcsWeEv10Y= -=WLZG +mQGiBEz/+9QRBACtgXbyja8lZlYinW6pGW+a7A8jhaEUS1FN7IGTIqJtqwcN7pS3 +jL9x/8DpuMzI2H7/uEElqa3vUqwGsHZy1mK8xJ2IWfvDJkx8KKDBRFc/5gG4KrPz +zhEtqiYy+UYAyYTIa31G0Yi0tUtl7w4NMrk4SbAXvidWOLlc7fPbnzNwewCg29NA +d+DhtjMsGISw+9UdXskuK5kD/A2rRkkUGSXMMG3F6t8UxoEFS2hp6LFpjhPpp7bp +MEx4Dd3SaWYMgv2kX2Lw4GKY/cWMCBuRvSkoGvbUrLbyFk3eUFDxBthc9VudFmga +JpCXT/hIR1xJ8cqBJmfCc6K/z9U5kjyFfxm4lX/Y+iUL5f3QTvIAmVlawnZARAFu +H2aPA/4/s+qQDulGGYXRNtNoHzppzfO24mPPvjgfDJ4jupZwOISm3/Jc4zY/dGxi +x3/61cQIrZXJ+OBlsfIfPnz5zh1e+I4y1XzozZlvBPUaVoMKUZHUXyQsi9UlgvIz +QdU7iK0MpYEpXC+DC1KQtkXwNSdx63+a40wyh+vW5QZ583Fmz7QkT2xnYSBOYXRr +b3ZpY2ggPG9sZ2FuQHlhaG9vLWluYy5jb20+iF4EExECAB4FAkz/+9QCGwMGCwkI +BwMCAxUCAwMWAgECHgECF4AACgkQd9QuHGc7InkMSQCeKR0Q2Qst2zc6t/JA7T6y +X/ukBVIAnR5de4YI3+LsToqZJFUvPDg4uG5TuQENBEz/+9YQBADcTndyZ0Vmh46R +UjqNRmv27rcsH62fvLIE7vUE2AQaDE1hEMwawjZXhtFa6Bzh6WWRXqkdNUWnx0E8 +HIHhI6jAOJ2Zgc5zickkFOC/zLuipogCt0n1saK0ZXyKOLWOEX/ggxb9QUcHjpT2 +Rih5Sf95XQ8q5jI6kPvv6YwX0rwMhwADBQQAqSbFpnQIb+V1xo6cwqBpjizVSadM +M5lXmsVngW+QMRp5hrw2bGfzkyDo2eowItUVyleCzTl7npFFFHKtNDS2g40QOB3z +YzKXLa+dsst2BUzbZryXpHUZLQ0IynsY14m3CTjuhoBe4awQl2bZrl6Yuvad/RNy +eJYZ4TCJB3/0BnCISQQYEQIACQUCTP/71gIbDAAKCRB31C4cZzsieTujAKCfvkXU +b6o3+hGaiQ91NNa06K3eOQCffz+zGQJQ4Ab18AVUfjq69TRHEHA= +=0PGJ -----END PGP PUBLIC KEY BLOCK----- + -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.2.6 (GNU/Linux) Modified: pig/trunk/build.xml URL: http://svn.apache.org/viewvc/pig/trunk/build.xml?rev=1044156&r1=1044155&r2=1044156&view=diff ============================================================================== --- pig/trunk/build.xml (original) +++ pig/trunk/build.xml Thu Dec 9 22:42:54 2010 @@ -766,8 +766,7 @@ - - + Modified: pig/trunk/src/docs/src/documentation/content/xdocs/tutorial.xml URL: http://svn.apache.org/viewvc/pig/trunk/src/docs/src/documentation/content/xdocs/tutorial.xml?rev=1044156&r1=1044155&r2=1044156&view=diff ============================================================================== --- pig/trunk/src/docs/src/documentation/content/xdocs/tutorial.xml (original) +++ pig/trunk/src/docs/src/documentation/content/xdocs/tutorial.xml Thu Dec 9 22:42:54 2010 @@ -37,76 +37,49 @@

The Pig tutorial file (tutorial/pigtutorial.tar.gz file in the pig distribution) includes the Pig JAR file (pig.jar) and the tutorial files (tutorial.jar, Pigs scripts, log files). These files work with Hadoop 0.20.2 and include everything you need to run the Pig scripts.

- -

To get started, follow these basic steps:

-
    -

  1. Install Java

    -
    2. -

  2. Install Pig

    -
    3. -

  3. Run the Pig scripts - in Local or Hadoop mode

    -
    4. -
- Java Installation +Check Your Setup -

Make sure your run-time environment includes the following:

-
    -

  • Java 1.6 or higher (preferably from Sun)

    -
    • -

  • The JAVA_HOME environment variable is set the root of your Java installation.

    -
    • -
+

Check your run-time environment and do the following preliminary tasks:

-
- -
- Pig Installation - -

To install Pig, do the following:

    -

  1. Download the Pig tutorial file to your local directory.

    +
  2. Make sure the JAVA_HOME environment variable is set the root of your Java installation.
    3. +
  3. Make sure that bin/pig is in your PATH (this enables you to run the tutorials using the "pig" command). + +$ export PATH=/<my-path-to-pig>/pig-n.n.n/bin:$PATH +
    4. -

  4. Unzip the Pig tutorial file (the files are stored in a newly created directory, pigtmp).

    +
  5. Set the PIG_HOME environment variable: + +$ export PIG_HOME=/<my-path-to-pig>/pig-n.n.n +
    6. +
  6. Copy the pigtutorial.tar.gz file from the tutorial directory of your Pig installation to your local directory.
    7. +
  7. Unzip the Pig tutorial file (the files are stored in a newly created directory, pigtmp). $ tar -xzf pigtutorial.tar.gz
    8. -

  8. Move to the pigtmp directory.

    -
    9. -

  9. Review the contents of the Pig tutorial file.

    -
    10. -

  10. Copy the pig.jar file to the appropriate directory on your system. For example: /home/me/pig.

    -
    11. -

  11. Create an environment variable, PIGDIR, and point it to your directory; for example, export PIGDIR=/home/me/pig (bash, sh) or setenv PIGDIR /home/me/pig (tcsh, csh).

    -
    12. +
  12. Review Pig Script 1 and Pig Script 2.
+
Running the Pig Scripts in Local Mode

To run the Pig scripts in local mode, do the following:

    -
  1. -

    Set the maximum memory for Java.

    - -java -Xmx256m -cp pig.jar org.apache.pig.Main -x local script1-local.pig -java -Xmx256m -cp pig.jar org.apache.pig.Main -x local script2-local.pig - -
    2. -

  2. Move to the pigtmp directory.

    3. -

  3. Review Pig Script 1 and Pig Script 2.

    4. -
  4. -

    Execute the following command (using either script1-local.pig or script2-local.pig).

    + +
  5. Move to the pigtmp directory.
    6. +
  6. Execute the following command (using either script1-local.pig or script2-local.pig). -$ java -cp $PIGDIR/pig.jar org.apache.pig.Main -x local script1-local.pig +$ pig -x local script1-local.pig
    7. -

  7. Review the result files, located in the part-r-00000 directory.

    +
  8. Review the result files, located in the part-r-00000 directory.

    The output may contain a few Hadoop warnings which can be ignored:

    2010-04-08 12:55:33,642 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics @@ -114,8 +87,6 @@ $ java -cp $PIGDIR/pig.jar org.apache.pi
- -
@@ -123,25 +94,29 @@ $ java -cp $PIGDIR/pig.jar org.apache.pi

To run the Pig scripts in mapreduce mode, do the following:

    -

  1. Move to the pigtmp directory.

    -
    2. -

  2. Review Pig Script 1 and Pig Script 2.

    -
    3. -

  3. Copy the excite.log.bz2 file from the pigtmp directory to the HDFS directory.

    +
  4. Move to the pigtmp directory.
    5. +
  5. Copy the excite.log.bz2 file from the pigtmp directory to the HDFS directory. $ hadoop fs –copyFromLocal excite.log.bz2 .
    6. -

  6. Set the HADOOP_CONF_DIR environment variable to the location of your core-site.xml, hdfs-site.xml and mapred-site.xml files.

    -
    7. -

  7. Execute the following command (using either script1-hadoop.pig or script2-hadoop.pig):

    +
  8. Set the PIG_CLASSPATH environment variable to the location of the cluster configuration directory (the directory that contains the core-site.xml, hdfs-site.xml and mapred-site.xml files): + +export PIG_CLASSPATH=/mycluster/conf +
    9. +
  9. Set the HADOOP_CONF_DIR environment variable to the location of the cluster configuration directory: + +export HADOOP_CONF_DIR=/mycluster/conf +
    10. + +
  10. Execute the following command (using either script1-hadoop.pig or script2-hadoop.pig): -$ java -cp $PIGDIR/pig.jar:$HADOOP_CONF_DIR org.apache.pig.Main script1-hadoop.pig +$ pig script1-hadoop.pig
    11. -

  11. Review the result files, located in the script1-hadoop-results or script2-hadoop-results HDFS directory:

    +
  12. Review the result files, located in the script1-hadoop-results or script2-hadoop-results HDFS directory: $ hadoop fs -ls script1-hadoop-results $ hadoop fs -cat 'script1-hadoop-results/*' | less Modified: pig/trunk/src/docs/src/documentation/content/xdocs/udf.xml URL: http://svn.apache.org/viewvc/pig/trunk/src/docs/src/documentation/content/xdocs/udf.xml?rev=1044156&r1=1044155&r2=1044156&view=diff ============================================================================== --- pig/trunk/src/docs/src/documentation/content/xdocs/udf.xml (original) +++ pig/trunk/src/docs/src/documentation/content/xdocs/udf.xml Thu Dec 9 22:42:54 2010 @@ -110,7 +110,7 @@ public class UPPER extends EvalFunc<S

    Now that we have the function implemented, it needs to be compiled and included in a jar. You will need to build pig.jar to compile your UDF. You can use the following set of commands to checkout the code from SVN repository and create pig.jar:

    -svn co http://svn.apache.org/repos/asf/hadoop/pig/trunk +svn co http://svn.apache.org/repos/asf/pig/trunk cd trunk ant Modified: pig/trunk/src/docs/src/documentation/content/xdocs/zebra_overview.xml URL: http://svn.apache.org/viewvc/pig/trunk/src/docs/src/documentation/content/xdocs/zebra_overview.xml?rev=1044156&r1=1044155&r2=1044156&view=diff ============================================================================== --- pig/trunk/src/docs/src/documentation/content/xdocs/zebra_overview.xml (original) +++ pig/trunk/src/docs/src/documentation/content/xdocs/zebra_overview.xml Thu Dec 9 22:42:54 2010 @@ -58,9 +58,9 @@
    Download Zebra

    Zebra is a Pig contrib project and is available at:

    - http://svn.apache.org/viewvc/hadoop/pig/trunk/contrib/zebra/

    + http://svn.apache.org/viewvc/pig/trunk/contrib/zebra/

    To work with Zebra you need to check out the Pig trunk:

    - http://svn.apache.org/repos/asf/hadoop/pig/trunk/

    + http://svn.apache.org/repos/asf/pig/trunk/