apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2153) Add user documentation for Enricher on apex docs
Date Mon, 01 Aug 2016 07:10:20 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401623#comment-15401623
] 

ASF GitHub Bot commented on APEXMALHAR-2153:
--------------------------------------------

Github user chinmaykolhatkar commented on a diff in the pull request:

    https://github.com/apache/apex-malhar/pull/353#discussion_r72931863
  
    --- Diff: docs/operators/enricher.md ---
    @@ -0,0 +1,169 @@
    +POJO Enricher
    +=============
    +
    +## Operator Objective
    +This operator receives an POJO ([Plain Old Java Object](https://en.wikipedia.org/wiki/Plain_Old_Java_Object))
as an incoming tuple and uses an external source to enrich the data in 
    +the incoming tuple and finally emits the enriched data as a new enriched POJO.
    +
    +POJOEnricher supports enrichment from following external sources:
    +
    +1. **JSON File Based** - Reads the file in memory having content stored in JSON format
and use that to enrich the data. This can be done using FSLoader implementation.
    +2. **JDBC Based** - Any JDBC store can act as an external entity to which enricher can
request data for enriching incoming tuples. This can be done using JDBCLoader implementation.
    +
    +POJO Enricher does not hold any state and is **idempotent**, **fault-tolerance** and
**statically/dynamically partitionable**.
    +
    +## Operator Usecase
    +1. Bank ***transaction records*** usually contains customerId. For further analysis of
transaction one wants the customer name and other customer related information. 
    +Such information is present in another database. One could enrich the transaction's record
with customer information using POJOEnricher.
    +2. ***Call Data Record (CDR)*** contains only mobile/telephone numbers of the customer.
Customer information is missing in CDR. POJO Enricher can be used to enricher 
    +CDR with customer data for further analysis.
    +
    +## Operator Information
    +1. Operator location: ***malhar-contrib***
    +2. Available since: ***3.4.0***
    +3. Operator state: ***Evolving***
    +3. Java Packages:
    +    * Operator: ***[com.datatorrent.contrib.enrich.POJOEnricher](https://www.datatorrent.com/docs/apidocs/com/datatorrent/contrib/enrich/POJOEnricher.html)***
    +    * FSLoader: ***[com.datatorrent.contrib.enrich.FSLoader](https://www.datatorrent.com/docs/apidocs/com/datatorrent/contrib/enrich/FSLoader.html)***
    +    * JDBCLoader: ***[com.datatorrent.contrib.enrich.JDBCLoader](https://www.datatorrent.com/docs/apidocs/com/datatorrent/contrib/enrich/JDBCLoader.html)***
    +
    +## Properties, Attributes and Ports
    +### <a name="props"></a>Properties of POJOEnricher
    +| **Property** | **Description** | **Type** | **Mandatory** | **Default Value** |
    +| -------- | ----------- | ---- | ------------------ | ------------- |
    +| *includeFields* | List of fields from database that needs to be added to output POJO.
| List<String\> | Yes | N/A |
    +| *lookupFields* | List of fields from input POJO which will form a *unique composite*
key for querying to database | List<String\> | Yes | N/A |
    --- End diff --
    
    yes composite keys work with JSON.


> Add user documentation for Enricher on apex docs
> ------------------------------------------------
>
>                 Key: APEXMALHAR-2153
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2153
>             Project: Apache Apex Malhar
>          Issue Type: Documentation
>            Reporter: Chinmay Kolhatkar
>            Assignee: Chinmay Kolhatkar
>
> Add user documentation for Enricher on apex docs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message