Return-Path: X-Original-To: apmail-manifoldcf-commits-archive@www.apache.org Delivered-To: apmail-manifoldcf-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0432210996 for ; Mon, 5 Jan 2015 19:49:40 +0000 (UTC) Received: (qmail 20468 invoked by uid 500); 5 Jan 2015 19:49:41 -0000 Delivered-To: apmail-manifoldcf-commits-archive@manifoldcf.apache.org Received: (qmail 20420 invoked by uid 500); 5 Jan 2015 19:49:40 -0000 Mailing-List: contact commits-help@manifoldcf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@manifoldcf.apache.org Delivered-To: mailing list commits@manifoldcf.apache.org Received: (qmail 20410 invoked by uid 99); 5 Jan 2015 19:49:40 -0000 Received: from eris.apache.org (HELO hades.apache.org) (140.211.11.105) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2015 19:49:40 +0000 Received: from hades.apache.org (localhost [127.0.0.1]) by hades.apache.org (ASF Mail Server at hades.apache.org) with ESMTP id A48DBAC0163; Mon, 5 Jan 2015 19:49:40 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1649630 - in /manifoldcf/branches/dev_1x: ./ site/src/documentation/content/xdocs/en_US/ site/src/documentation/resources/images/en_US/ Date: Mon, 05 Jan 2015 19:49:38 -0000 To: commits@manifoldcf.apache.org From: kwright@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20150105194940.A48DBAC0163@hades.apache.org> Author: kwright Date: Mon Jan 5 19:49:37 2015 New Revision: 1649630 URL: http://svn.apache.org/r1649630 Log: Pull up CONNECTORS-1136 fix from trunk. Added: manifoldcf/branches/dev_1x/site/src/documentation/resources/images/en_US/metadataadjuster-job-expressions-metadata.PNG - copied unchanged from r1649628, manifoldcf/trunk/site/src/documentation/resources/images/en_US/metadataadjuster-job-expressions-metadata.PNG Removed: manifoldcf/branches/dev_1x/site/src/documentation/resources/images/en_US/metadataadjuster-job-add-metadata.PNG manifoldcf/branches/dev_1x/site/src/documentation/resources/images/en_US/metadataadjuster-job-move-metadata.PNG Modified: manifoldcf/branches/dev_1x/ (props changed) manifoldcf/branches/dev_1x/CHANGES.txt manifoldcf/branches/dev_1x/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml Propchange: manifoldcf/branches/dev_1x/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Jan 5 19:49:37 2015 @@ -121,4 +121,4 @@ /manifoldcf/branches/CONNECTORS-981:1605049-1605773 /manifoldcf/branches/CONNECTORS-989:1611600-1612101 /manifoldcf/branches/CONNECTORS-990:1610284-1610707 -/manifoldcf/trunk:1620703,1620748,1620812,1620862,1621449,1621613,1621855,1622213,1622740,1622850,1622853-1622854,1623249,1623251,1623314,1623599,1623951,1623953-1623954,1623956,1623972,1624058,1624085,1624174,1624236,1624377,1624384,1624399,1624449,1624464,1624504,1624729-1624731,1624906,1624909-1624910,1624982,1625023,1625095,1625103,1625108,1625264,1625270,1625394,1625400,1625910,1626090,1626097,1626102,1626638-1626639,1626973,1627687,1627690,1627959,1628046,1628066,1628106,1628168,1628188,1628699,1628798,1628808,1628845,1628905,1629122,1629374-1629375,1629379,1629541,1629994,1630188,1630535,1630623,1630671,1630812,1630885,1631039,1631162,1631164,1631252,1631750,1631953,1632013,1632225,1632289,1632562,1632844,1632847,1632854,1633062-1633063,1633108,1633193,1633202,1633282,1633284,1633295,1633336,1633339,1633345,1633348,1633364,1633378,1633383,1633432,1633546,1633590,1633634,1633668,1633727,1633760,1633764,1633786,1633910,1633923,1634021,1634028,1634067,1634132,1634145,1634148,163 4155,1634188,1634202,1634264,1634373,1634530,1634688,1634850,1634857,1635103,1635116,1635421,1635438,1635478,1635481,1635484,1635490,1635809,1635939,1636146,1636167,1636180,1636207,1636215,1636232,1636334,1636519,1636570,1636684,1636940,1637011,1637310,1637350,1637364,1637373,1637378,1639259,1639593,1639600,1640018,1640101,1640199,1640204,1640208,1640314,1640319,1640749,1640772,1640805,1640888,1640925,1640941-1640942,1641222,1641328,1641557,1641559,1641629,1641633,1641724,1641754,1641911,1642163,1642255,1642318,1642531,1642650,1642658,1642673,1642716,1644197,1644399,1644538,1644920,1644931,1646317,1646397,1646403,1646408,1646640,1646947,1647574,1647585,1647608,1648686,1648976,1649201,1649203,1649529,1649605 +/manifoldcf/trunk:1620703,1620748,1620812,1620862,1621449,1621613,1621855,1622213,1622740,1622850,1622853-1622854,1623249,1623251,1623314,1623599,1623951,1623953-1623954,1623956,1623972,1624058,1624085,1624174,1624236,1624377,1624384,1624399,1624449,1624464,1624504,1624729-1624731,1624906,1624909-1624910,1624982,1625023,1625095,1625103,1625108,1625264,1625270,1625394,1625400,1625910,1626090,1626097,1626102,1626638-1626639,1626973,1627687,1627690,1627959,1628046,1628066,1628106,1628168,1628188,1628699,1628798,1628808,1628845,1628905,1629122,1629374-1629375,1629379,1629541,1629994,1630188,1630535,1630623,1630671,1630812,1630885,1631039,1631162,1631164,1631252,1631750,1631953,1632013,1632225,1632289,1632562,1632844,1632847,1632854,1633062-1633063,1633108,1633193,1633202,1633282,1633284,1633295,1633336,1633339,1633345,1633348,1633364,1633378,1633383,1633432,1633546,1633590,1633634,1633668,1633727,1633760,1633764,1633786,1633910,1633923,1634021,1634028,1634067,1634132,1634145,1634148,163 4155,1634188,1634202,1634264,1634373,1634530,1634688,1634850,1634857,1635103,1635116,1635421,1635438,1635478,1635481,1635484,1635490,1635809,1635939,1636146,1636167,1636180,1636207,1636215,1636232,1636334,1636519,1636570,1636684,1636940,1637011,1637310,1637350,1637364,1637373,1637378,1639259,1639593,1639600,1640018,1640101,1640199,1640204,1640208,1640314,1640319,1640749,1640772,1640805,1640888,1640925,1640941-1640942,1641222,1641328,1641557,1641559,1641629,1641633,1641724,1641754,1641911,1642163,1642255,1642318,1642531,1642650,1642658,1642673,1642716,1644197,1644399,1644538,1644920,1644931,1646317,1646397,1646403,1646408,1646640,1646947,1647574,1647585,1647608,1648686,1648976,1649201,1649203,1649529,1649605,1649628 Modified: manifoldcf/branches/dev_1x/CHANGES.txt URL: http://svn.apache.org/viewvc/manifoldcf/branches/dev_1x/CHANGES.txt?rev=1649630&r1=1649629&r2=1649630&view=diff ============================================================================== --- manifoldcf/branches/dev_1x/CHANGES.txt (original) +++ manifoldcf/branches/dev_1x/CHANGES.txt Mon Jan 5 19:49:37 2015 @@ -3,6 +3,10 @@ $Id$ ======================= 1.9-dev ===================== +CONNECTORS-1136: Update the user documentation to account for +changes in the metadata adjuster transformer. +(Karl Wright) + CONNECTORS-1134: Revamp metadata adjuster transformer to handle arbitrary combination expressions across multiple fields. This replaces the "move" and "add" paradigm with a more general Modified: manifoldcf/branches/dev_1x/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml URL: http://svn.apache.org/viewvc/manifoldcf/branches/dev_1x/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml?rev=1649630&r1=1649629&r2=1649630&view=diff ============================================================================== --- manifoldcf/branches/dev_1x/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml (original) +++ manifoldcf/branches/dev_1x/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml Mon Jan 5 19:49:37 2015 @@ -1014,27 +1014,31 @@ curl -XGET http://localhost:9200/index/_
Metadata Adjuster -

The Metadata Adjuster transformation filter optionally changes the name of incoming metadata, and then optionally adds additional metadata values. +

The Metadata Adjuster transformation filter reorganizes and concatenates metadata based on rules that you provide. This can be very helpful in many contexts. For example, you might use the Metadata Adjuster to label all documents from a particular job with a particular tag in an index. Or, you might need to map metadata from (say) SharePoint's schema to your final output connection type's schema. The Metadata Adjuster permits you to handle both of the scenarios.

As with all document transformers, more than one Metadata Adjuster transformation filter can be used in a single pipeline. This may be useful if other document transformers (such as the Tika Content Extractor, below) change the metadata of the document being processed.

The Metadata Adjuster transformation connection type does not require anything other than standard configuration information.

-

The Metadata Adjuster transformation connection type contributes two tabs to a job definition. These are "Move metadata" and "Add metadata" - tabs. The "Move metadata" tab looks like this:

+

The Metadata Adjuster transformation connection type contributes one tab to a job definition. This is the "Metadata expressions" tab. + The "Metadata expressions" tab looks like this:



-
+


-

Enter a input metadata name, and a target metadata name, and click the "Add" button to add the mapping to the list. Uncheck the "Keep all metadata" +

On the left, you must supply the target metadata name. You may then choose among two possibilities: either you choose to request that this metadata + field be removed from the target document, or you can choose to specify a rule that would provide a value for that target metadata item. You can + provide more than one rule for the same metadata item, by simply adding additional lines to the table with the same target metadata name. But if you specify + a rule to remove the metadata field from the document, that selection overrides all others.

+

Metadata rules are just strings, which may have field specifications within them. For example, the rule value "hello" will generate a single-valued metadata + field with the value "hello". The rule "hello ${there}", on the other hand, will generate a value for each value in the incoming field named "there". If the incoming + field had the values "a", "b", and "c", then the rule would generate a three-valued field with values "hello a", "hello b", and "hello c". If more than one incoming + field is specified, then a combinatoric combination of the field values will be produced.

+

Enter a parameter name, and either select to remove the value or provide an expression. If you chose to supply an expression, enter the expression in the box. + Then, click the "Add" button.

+

When your expressions have been edited, you can uncheck the "Keep all metadata" checkbox in order to prevent unspecified metadata fields from being passed through. Uncheck the "Remove empty metadata values" checkbox if you want to index empty metadata values.

-

The "Add metadata" tab looks like this:

-

-
-

-

Enter a parameter name and a value, and then click the "Add" button to add the new metadata field and value to the list. You may add more than - one value with the same field name.