Return-Path: X-Original-To: apmail-pig-commits-archive@www.apache.org Delivered-To: apmail-pig-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 56C4918151 for ; Mon, 30 Nov 2015 19:05:00 +0000 (UTC) Received: (qmail 22862 invoked by uid 500); 30 Nov 2015 19:05:00 -0000 Delivered-To: apmail-pig-commits-archive@pig.apache.org Received: (qmail 22799 invoked by uid 500); 30 Nov 2015 19:05:00 -0000 Mailing-List: contact commits-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list commits@pig.apache.org Received: (qmail 22790 invoked by uid 99); 30 Nov 2015 19:04:59 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Nov 2015 19:04:59 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 8C0C2C2745 for ; Mon, 30 Nov 2015 19:04:59 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.79 X-Spam-Level: * X-Spam-Status: No, score=1.79 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id z1zpIycRO8nl for ; Mon, 30 Nov 2015 19:04:58 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTP id 6ED3820FE7 for ; Mon, 30 Nov 2015 19:04:57 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 78DDDE0280 for ; Mon, 30 Nov 2015 19:04:56 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id 23EC03A0234 for ; Mon, 30 Nov 2015 19:04:56 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1717304 - in /pig/trunk: CHANGES.txt src/docs/src/documentation/content/xdocs/basic.xml src/docs/src/documentation/content/xdocs/pig-index.xml src/docs/src/documentation/content/xdocs/tabs.xml Date: Mon, 30 Nov 2015 19:04:55 -0000 To: commits@pig.apache.org From: daijy@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20151130190456.23EC03A0234@svn01-us-west.apache.org> Author: daijy Date: Mon Nov 30 19:04:55 2015 New Revision: 1717304 URL: http://svn.apache.org/viewvc?rev=1717304&view=rev Log: PIG-4742: Document Pig's Register Artifact Command added in PIG-4417 Modified: pig/trunk/CHANGES.txt pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml pig/trunk/src/docs/src/documentation/content/xdocs/pig-index.xml pig/trunk/src/docs/src/documentation/content/xdocs/tabs.xml Modified: pig/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/pig/trunk/CHANGES.txt?rev=1717304&r1=1717303&r2=1717304&view=diff ============================================================================== --- pig/trunk/CHANGES.txt (original) +++ pig/trunk/CHANGES.txt Mon Nov 30 19:04:55 2015 @@ -24,6 +24,8 @@ INCOMPATIBLE CHANGES IMPROVEMENTS +PIG-4742: Document Pig's Register Artifact Command added in PIG-4417 (akshayrai09 via daijy) + PIG-4417: Pig's register command should support automatic fetching of jars from repo (akshayrai09 via daijy) PIG-4713: Document Bloom UDF (gliptak via daijy) Modified: pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml URL: http://svn.apache.org/viewvc/pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml?rev=1717304&r1=1717303&r2=1717304&view=diff ============================================================================== --- pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml (original) +++ pig/trunk/src/docs/src/documentation/content/xdocs/basic.xml Mon Nov 30 19:04:55 2015 @@ -8922,8 +8922,8 @@ B = FOREACH A GENERATE myFunc($0); -
- REGISTER +
+ REGISTER (a jar/script)

Registers a JAR file so that the UDFs in the file can be used.

@@ -8995,7 +8995,222 @@ register jars/*.jar
-
+ +
+ + REGISTER (an artifact) + +

+ Instead of figuring out the dependencies manually, downloading them and registering each jar using the above + register command, you can specify the artifact's coordinates and expect pig to automatically + fetch the required dependencies, download and register them. +

+ + +
+ Syntax +

+ To download an Artifact (and its dependencies), you need to specify the artifact's group, module and version following + the syntax shown below. This command will download the Jar specified and all its dependencies and load it into the + classpath. +

+ + + + +
+

REGISTER ivy://group:module:version?querystring

+
+
+ + +
+ Terms + + + + + + + + + + + + + + + + + +
+

group

+ 		+

Which module group the module comes from. Translates directly to a Maven groupId or an Ivy Organization.

+
+

module

+ 		+

The name of the module to load. Translated directly to a Maven artifactId or an Ivy artifact.

+
+

version

+ 		+

The version of the module to use. You can specify a specific version or use "+" or "*" to use the latest version.

+
+

querystring

+ 		+

This will contain "&" separated key-value pairs to help us exclude all or specific dependencies etc.

+
+
+ +
+ Usage + +

+ The Register artifact command is an extension to the above register command used to register a + jar. In addition to registering a jar from a local system or from hdfs, you can now specify the coordinates of the + artifact and pig will download the artifact (and its dependencies if needed) from the configured repository. +

+ +
+ Parameters Supported in the Query String + +
    +
  • + Transitive +

    + Transitive helps specifying if you need the dependencies along with the registering jar. By setting transitive to + false in the querystring we can tell pig to register only the artifact without its dependencies. This will + download only the artifact specified and will not download the dependencies of the jar. The default value of + transitive is true. +

    + Syntax + + + + +
    +

    REGISTER ivy://org:module:version?transitive=false

    +
    +
    • +
  • + Exclude +

    + While registering an artifact if you wish to exclude some dependencies you can specify them using the exclude + key. Suppose you want to use a specific version of a dependent jar which doesn't match the version of the jar + when automatically fetched, then you could exclude such dependencies by specifying a comma separated list of + dependencies and register the dependent jar separately. +

    + Syntax + + + + +
    +

    REGISTER ivy://org:module:version?exclude=org:mod,org:mod,...

    +
    +
    • +
  • + Classifier +

    + Some maven dependencies need classifiers in order to be able to resolve. You can specify them using a classifier + key. +

    + Syntax + + + + +
    +

    REGISTER ivy://org:module:version?classifier=value

    +
    +
    • +
+
+ +
+ Other properties + +
    +
  • +

    + An optional pig property, pig.artifacts.download.location, can be used to configure the location where the + artifacts should be downloaded. By default, they will be downloaded to ~/.groovy/grapes +

    +
    • + +
  • +

    + This command can be used or can replace the register jar command wherever used + including macros.

    +

    +
    • + +
  • +

    + Group/Organization and Version are optional fields. In such cases you can leave them blank.

    +

    +
    • + +
  • +

    + The repositories can be configured using an ivysettings file. Pig will search for an ivysettings.xml file + in the following locations in order. PIG_CONF_DIR > PIG_HOME > Classpath

    +

    +
    • +
+
+
+ + +
+ Examples + +
    +
  • +

    Registering an Artifact and all its dependencies.

    + + -- Both are the same

    + REGISTER ivy://org.apache.avro:avro:1.5.1

    + REGISTER ivy://org.apache.avro:avro:1.5.1?transitive=true +
    • + +
  • +

    Registering an artifact without getting its dependencies.

    + + REGISTER ivy://org.apache.avro:avro:1.5.1?transitive=false +
    • + +
  • +

    Registering the latest artifact.

    + + -- Both of the following syntaxes work.

    + REGISTER ivy://org.apache.avro:avro:+

    + REGISTER ivy://org.apache.avro:avro:* +
    • + +
  • +

    Registering an artifact by excluding specific dependencies.

    + + REGISTER ivy://org.apache.pig:pig:0.10.0?exclude=commons-cli:commons-cli,commons-codec:commons-codec +
    • + +
  • +

    Specifying a classifier

    + + REGISTER ivy://net.sf.json-lib:json-lib:2.4?classifier=jdk15 +
    • + +
  • +

    Registering an artifact without a group or organization. Just skip them.

    + + REGISTER ivy://:module: +
    • +
+
+
+ + + Modified: pig/trunk/src/docs/src/documentation/content/xdocs/pig-index.xml URL: http://svn.apache.org/viewvc/pig/trunk/src/docs/src/documentation/content/xdocs/pig-index.xml?rev=1717304&r1=1717303&r2=1717304&view=diff ============================================================================== --- pig/trunk/src/docs/src/documentation/content/xdocs/pig-index.xml (original) +++ pig/trunk/src/docs/src/documentation/content/xdocs/pig-index.xml Mon Nov 30 19:04:55 2015 @@ -819,7 +819,7 @@

REGEX_EXTRACT_ALL function

-

REGISTER statement

+

REGISTER statement

regular expressions. See pattern matching

Modified: pig/trunk/src/docs/src/documentation/content/xdocs/tabs.xml URL: http://svn.apache.org/viewvc/pig/trunk/src/docs/src/documentation/content/xdocs/tabs.xml?rev=1717304&r1=1717303&r2=1717304&view=diff ============================================================================== --- pig/trunk/src/docs/src/documentation/content/xdocs/tabs.xml (original) +++ pig/trunk/src/docs/src/documentation/content/xdocs/tabs.xml Mon Nov 30 19:04:55 2015 @@ -32,6 +32,6 @@ --> - +