camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CAMEL-8149) Support application-generated document identifiers in bulk index requests
Date Fri, 12 Dec 2014 02:16:13 GMT

    [ https://issues.apache.org/jira/browse/CAMEL-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243601#comment-14243601
] 

ASF GitHub Bot commented on CAMEL-8149:
---------------------------------------

GitHub user dabdine-r7 opened a pull request:

    https://github.com/apache/camel/pull/356

    CAMEL-8149: Better support for application-generated identifiers in bulk requests

    https://issues.apache.org/jira/browse/CAMEL-8149
    
    This design change allows upstream message processors to provide elasticsearch-java ActionRequest
subclasses in the message body for more fine-grain control over the data that is sent to the
es endpoint.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dabdine-r7/camel camel-2.15.0-elasticsearch-bulk

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/camel/pull/356.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #356
    
----
commit c6eb87c30a875d96330330d8894b2a686128bf73
Author: Derek Abdine <dabdine@rapid7.com>
Date:   2014-12-12T01:12:03Z

    Enhance elasticsearch producer to support elasticsearch-java ActionRequest object type
bodies

commit 46a9fa4ee22cfe6e4d7d922ee69b98e8ca4c7b64
Author: Derek Abdine <dabdine@rapid7.com>
Date:   2014-12-12T01:38:50Z

    Add unit tests

----


> Support application-generated document identifiers in bulk index requests
> -------------------------------------------------------------------------
>
>                 Key: CAMEL-8149
>                 URL: https://issues.apache.org/jira/browse/CAMEL-8149
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-elasticsearch
>    Affects Versions: 2.14.0
>            Reporter: Derek Abdine
>             Fix For: 2.15.0
>
>
> Elasticsearch (via the elasticsearch-java transport client) provides two categories of
APIs to write and read data: Individual requests (index, get, delete) and bulk requests.
> When performing bulk updates one creates individual index requests and adds them to the
bulk request. When creating an index request one can set the source document, id, etc. 
> The current design of the camel-elasticsearch component controls the transformation and
assembly of an input body (json string, byte[], xcontentfactory, map) to an index request.
Thus, it is impossible to set the id on the index request that goes into a bulk action. The
end result is that the id is set by the default behavior of the underlying elasticsearch-java
client which generates a random identifier.  This is problematic in situations where control
is needed over the id, e.g. for de-duplication purposes.
> My proposal is to improve the design of the producer to allow for elasticsearch-java
ActionRequest sub-classes in the message body so that upstream message processors can control
the creation of those requests.
> I've attached a patch and sent a pull request on github.
> Thank you!
> Derek Abdine



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message