streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (STREAMS-170) ES Utility Processor
Date Fri, 17 Oct 2014 18:48:34 GMT

    [ https://issues.apache.org/jira/browse/STREAMS-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14175371#comment-14175371
] 

ASF GitHub Bot commented on STREAMS-170:
----------------------------------------

GitHub user steveblackmon opened a pull request:

    https://github.com/apache/incubator-streams/pull/105

    Streams 170

    These processors support new use cases for streams, such as 
    1) generate datums in a stream based solely on elasticsearch metadata, specifically an
index,type,id tuple (assuming a cluster is configured).  
    2) branch the stream with full documents from indexes other than those specified in the
ESReader, by building metadata for those documents performing a 'join'.
    3) to tackle more advanced use cases, such as deleting or a specific set of documents,
or copying a specific set of documents between persisters if at any point they aren't present
in either.
    
    I have several examples of simple usage up in w2ogroup/streams-examples, STREAMS-170 branch,
and some more advanced use cases not yet pushed because they depend on changes in STREAMS-168.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/apache/incubator-streams STREAMS-170

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-streams/pull/105.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #105
    
----
commit bc367b3096d685235de2f44ba47999e126d9a141
Author: Steve Blackmon <sblackmon@w2odigital.com>
Date:   2014-09-11T17:51:22Z

    Utility Processors to
     - populate datum from metadata
     - populate datum from metadata in document field

commit 933060e42add5b3a640c78a34457e86014a591a5
Author: Steve Blackmon <sblackmon@w2odigital.com>
Date:   2014-09-16T16:22:36Z

    license / javadoc headers

commit aef8860f95002045322265e4834331b57643a50f
Author: Steve Blackmon <sblackmon@w2odigital.com>
Date:   2014-09-11T17:51:22Z

    Utility Processors to
     - populate datum from metadata
     - populate datum from metadata in document field

commit bfd1bc1d1f7c58f001c6aeb7fa430e7a70f299ed
Author: Steve Blackmon <sblackmon@w2odigital.com>
Date:   2014-09-16T16:22:36Z

    license / javadoc headers

commit 576b84970959bbdaa283b541d699214ecf86d2b5
Author: Steve Blackmon <sblackmon@w2odigital.com>
Date:   2014-09-24T17:31:23Z

    Merge branch 'STREAMS-170' of https://git-wip-us.apache.org/repos/asf/incubator-streams
into STREAMS-170

commit 6cd97b8e8a77c4a29c3f3079223a94a4ef41b877
Author: Steve Blackmon <sblackmon@w2odigital.com>
Date:   2014-10-01T20:42:55Z

    DocumentToMetadataProcessor utility processor

commit ddf359c508666faf89a2ff411c6c4eb76582e82a
Author: Steve Blackmon <sblackmon@w2odigital.com>
Date:   2014-10-16T17:19:29Z

    Minor change to javadoc header

----


> ES Utility Processor
> --------------------
>
>                 Key: STREAMS-170
>                 URL: https://issues.apache.org/jira/browse/STREAMS-170
>             Project: Streams
>          Issue Type: Improvement
>            Reporter: Steve Blackmon
>
> The ES module utilizes streams metadata to uniquely identify documents on a cluster by
their index, type, and ID.  It would be useful to have a processor that can join in the current
version of documents from their metadata.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message