camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomohisa Igarashi <>
Subject Re: Adding type awareness in Camel route
Date Tue, 18 Oct 2016 14:07:55 GMT

On 10/18/2016 10:31 PM, Siano, Stephan wrote:
> Hi,
> Well, you aim quite high.
> So the idea is to annotate each endpoint with an input and an output data format and
then do the mapping between these types automatically via the type converter registry. Did
I get this right?

Yes that direction, but not only TypeConverter as it just performs a Java->Java transformation.
With leveraging transformation endpoints and POJO as well, it enables Any->Any transformation.
That's why I needed to introduce <transformer> into CamelContext. These 2 testcases
demonstrate it.
[Java DSL]
[Spring DSL]

> In this case it might be a useful feature, but I would guess it is probably very hard
to implement.

Yes I went back and forth again and again in camel-core in depth. It eventually worked :)


> Best regards
> Stephan
> -----Original Message-----
> From: Tomohisa Igarashi []
> Sent: Montag, 17. Oktober 2016 16:51
> To:
> Subject: Re: Adding type awareness in Camel route
> Hi,
> On 10/17/2016 03:32 PM, Siano, Stephan wrote:
>> Hi,
>> I am not sure if I really understand the concept behind this. Is there any text about
the general ideas behind these changes?
> Good question, here is the original proposal my colleague made. "1) Extend the DSL" is
what I implemented.
>> If I got this right, you propose that endpoints declare a kind of data type that
can be verified as input and output types (like a specific XML schema) and the camel route
will automatically validate this constraint. Did I get this right?
> Validation is also one of the things type declaration enables, I don't yet implement
though. I implemented declarative transformation first. The type declaration specifies "how
content should look like at this point", and if a transformer which transforms from current
data type to that declared data type exists, it is triggered. The current data type is transfered
as an Exchange property, if it's not set then Java type of IN message body is used.
> The type awareness benefits all of 3 endpoint types you categorized:
>> Concerning this payload classification we have actually three kinds of endpoints
in camel:
>> 1. Endpoints that do not care about data content at all (e.g. a file)
> Let's say if there's from("file:dir").inputType("xml:{}aaaInput").to("cxf://")
and a transformer "byte[] -> xml:{}aaa" is declared, that transformer will
be picked up and triggered. Not yet implemented though, cxf endpoint would be able to declare
its input/output automatically from WSDL so that the "xml:{}aaaInput -> xml:{}"
transformer is triggered. Maybe we can even introduce a kind of "type parser" which user can
plugin on file consumer so that the file content type is recognized at first place?
>> 2. Endpoints that do care about data content, however the specific content type definition
may be part of the endpoint configuration. An example for this might be a CXF web service
endpoint with a WSDL. The endpoint as such only demands that the payload is some kind of XML
but the WSDL may (or may not e.g. if the message definition is xsd:any) put some additional
restriction on the payload data.
> This is a good example actually, CXF consumer would be able to set current input type
and expected output type automatically from WSDL. Including producer side I mentioned in #1,
that would be the 2nd step to improve this type awareness stuff.
>> 3. Endpoints that enforce some kind of (usually technical) format on the endpoint
(data types are usually of that kind. If you are trying a zipfile unmarshal, you'd better
have a zip file as a payload...).
> Just an idea yet though, type name "zip:xml:{}aaaInput" may imply the content
is zip compressed single text file, which contains "xml:{}aaaInput" type of
>> Are you focusing any of these kinds of endpoint types or are you just focusing some
special processors that operate on very specific data?
> The answer is all of them, plus not only for endpoint, but for arbitrary processors!
> Does it make sense?
> Thanks,
> Tomo
>> Best regards
>> Stephan
>> -----Original Message-----
>> From: Tomohisa Igarashi []
>> Sent: Montag, 17. Oktober 2016 04:36
>> To:
>> Subject: Re: Adding type awareness in Camel route
>> Hi,
>> I'd like to resume the discussion about this. I still think 3.0 would be the best
target to get this feature fully supported, but to achieve it in better shape, I'd like to
have it in 2.19 as well as an experimental, ask feedback and then reflect those for 3.0 full
support. Fortunately this is purely an addition to existing features, i.e. is not breaking
any existing API. What do you think?
>> Thanks,
>> Tomo
>> On 09/17/2016 10:20 PM, Tomohisa Igarashi wrote:
>>> Hi Claus,
>>> Thanks for the reply. Sure that's fine, I agree 3.0 would be better to be targeted
than 2.x as this introduces some schema updates.
>>> Including this one, I'm always looking for the chance to make any contribution
to Camel. If there's anything I can help please let me know.
>>> Thanks,
>>> Tomo
>>> On 09/17/2016 06:33 PM, Claus Ibsen wrote:
>>>> Hey
>>>> Can we take this discussion post Camel 2.18 release.
>>>> We are working on the last details to get it ready, and its our main
>>>> focus to get this new release out.
>>>> After this release we will pickup talks about the next releases
>>>> whether that is 2.19 or 3.0, and for the latter what the broad goals
>>>> of that is. What you talk about seems more of a 3.0 candidate to me,
>>>> than on 2.x.
>>>> On Fri, Sep 16, 2016 at 6:50 AM, Tomohisa Igarashi
>>>> <> wrote:
>>>>> Hi Camel developers,
>>>>> I'd like to propose an enhancement on handling data types of Camel message
>>>>> contents. To start a smooth discussion I implemented the idea first:
>>>>> And these testcases demonstrates the declarative transformer usage according
>>>>> to the declared data types:
>>>>> [Java DSL]
>>>>> [Spring DSL]
>>>>> This adds input/output content type declaration on from and all other
>>>>> processors. It also introduces well-known Exchange properties, INPUT_TYPE
>>>>> and OUTPUT_TYPE which are used to specify the current message content
>>>>> The data type is URN like string starts with scheme, like
>>>>> java:org.example.ItemA or xml:{org.example.xml}ItemA.
>>>>> If the content type is declared via inputType/outputType/contract, the
>>>>> ContractProcessor wraps the actual processor and process
>>>>> transformation/validation according to the type, say if INPUT_TYPE exchange
>>>>> property has xml:{org.example.xml}ItemA and the declared inputType is
>>>>> xml:{org.example.xml}ItemB, then it transforms xml:{org.example.xml}ItemA
>>>>> content into xml:{org.example.xml}ItemB. The <transformers> element
which is
>>>>> introduced right under the <camelContext> is the one to declare
the mappings
>>>>> between transformer implementation and those from/to data type. I
>>>>> implemented only transformer first, but validator would be brought in
in a
>>>>> same way. This way allows users to make data types visible in the route
>>>>> definition and keep the transformation/validation apart from route
>>>>> definition itself.
>>>>> The most important thing is that the ContractProcessor is involved only
>>>>> content type is explicitly declared in a route definition, so that it
>>>>> breaks existing camel routes. Ofcourse programatic
>>>>> transformations/validations we're doing today are still fully available.
>>>>> It's purely an addition to the existing camel features.
>>>>> Any thoughts? Does it sound acceptable to get merged into camel? ANY
>>>>> feedback would be really appreciated!
>>>>> Thanks,
>>>>> Tomo

View raw message