camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (CAMEL-8149) Support application-generated document identifiers in bulk index requests
Date Fri, 12 Dec 2014 02:16:13 GMT


ASF GitHub Bot commented on CAMEL-8149:

GitHub user dabdine-r7 opened a pull request:

    CAMEL-8149: Better support for application-generated identifiers in bulk requests
    This design change allows upstream message processors to provide elasticsearch-java ActionRequest
subclasses in the message body for more fine-grain control over the data that is sent to the
es endpoint.

You can merge this pull request into a Git repository by running:

    $ git pull camel-2.15.0-elasticsearch-bulk

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #356
commit c6eb87c30a875d96330330d8894b2a686128bf73
Author: Derek Abdine <>
Date:   2014-12-12T01:12:03Z

    Enhance elasticsearch producer to support elasticsearch-java ActionRequest object type

commit 46a9fa4ee22cfe6e4d7d922ee69b98e8ca4c7b64
Author: Derek Abdine <>
Date:   2014-12-12T01:38:50Z

    Add unit tests


> Support application-generated document identifiers in bulk index requests
> -------------------------------------------------------------------------
>                 Key: CAMEL-8149
>                 URL:
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-elasticsearch
>    Affects Versions: 2.14.0
>            Reporter: Derek Abdine
>             Fix For: 2.15.0
> Elasticsearch (via the elasticsearch-java transport client) provides two categories of
APIs to write and read data: Individual requests (index, get, delete) and bulk requests.
> When performing bulk updates one creates individual index requests and adds them to the
bulk request. When creating an index request one can set the source document, id, etc. 
> The current design of the camel-elasticsearch component controls the transformation and
assembly of an input body (json string, byte[], xcontentfactory, map) to an index request.
Thus, it is impossible to set the id on the index request that goes into a bulk action. The
end result is that the id is set by the default behavior of the underlying elasticsearch-java
client which generates a random identifier.  This is problematic in situations where control
is needed over the id, e.g. for de-duplication purposes.
> My proposal is to improve the design of the producer to allow for elasticsearch-java
ActionRequest sub-classes in the message body so that upstream message processors can control
the creation of those requests.
> I've attached a patch and sent a pull request on github.
> Thank you!
> Derek Abdine

This message was sent by Atlassian JIRA

View raw message