openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Thoemmes" <markus.thoem...@de.ibm.com>
Subject Re: Possible enhancements around attachment handling
Date Thu, 29 Mar 2018 11:10:59 GMT
Heya,

>> So for #C is there a way to proxy the stream without changing the runtime http api
where binary is passed embedded in the json object on /init?

> They would need to be modified. A less intrusive approach would have
> been to pass signed url and use client like wget to fetch the binary.
> But as David mentioned some setups may lock down access to external
> services. For such cases we would need to implement the support in
> client runtimes as part of init protocol

> Chetan Mehrotra


I believe they don't need to be changed at all. TCP as the transport protocol is streamed
by default (stuff is broken down into smallish packets). Say for example our payload we need
to pass is: {"main": "foo", "code": "Some veeeeery long string here"}. Let's also assume our
attachement is available as Source[ByteString]. What we can do, is generate the first bit
until the part we *need* to stream, in this case: {"main": "foo", "code": ". Now we consume
the chunks of the attachement Source and put them on the downstream. In the end, we put an
element like "} on the stream, to close the JSON object.

Pseudocode: Source.single(ByteString("""{"main": "foo", "code":""")).concat(attachementSource).to(httpConnectionToContainer).run()

This should be absolutely doable without a change in the runtimes at all, as they "shouldn't"
(to be verified) need a change to support TCP streaming.

Does that make sense?

Cheers,
Markus


Mime
View raw message