cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michal Mocny <mmo...@chromium.org>
Subject Re: Binary data across exec bridge
Date Tue, 15 Jan 2013 15:37:24 GMT
I should also note: I was encoding strings using only the lower 8-bits
assuming that JSON will UTF-8 encode anyway, but there is some
experimentation to be done using 16bit representations etc, but I right now
we need an order of magnitude perf increase and this won't get us there.


On Tue, Jan 15, 2013 at 10:28 AM, Michal Mocny <mmocny@chromium.org> wrote:

> Don, I had previously tried a byte array and found it quite a bit less
> efficient.  I'll add that implementation and benchmark it officially.
>
> I did benchmarks on iphone5 today for two initial implementations:
> (1) Using base64 encoded javascript strings
> (2) Using non base64 encoded, unicode javascript strings
>
> (1) YES BASE64 ENCODE
>
> 2013-01-15 10:16:02.233 MobileSpecTest[1930:907] Multi-tasking -> Device:
> YES, App: YES
> 2013-01-15 10:16:02.749 MobileSpecTest[1930:907] [LOG] Device = iOS 6.0.2
> 2013-01-15 10:16:38.104 MobileSpecTest[1930:907] [LOG] Started exec
> benchmark with payload length: 1
> 2013-01-15 10:16:39.103 MobileSpecTest[1930:907] [LOG] Calls per second:
> 274.72527472527474
> 2013-01-15 10:16:55.995 MobileSpecTest[1930:907] [LOG] Started exec
> benchmark with payload length: 32000
> 2013-01-15 10:17:01.007 MobileSpecTest[1930:907] [LOG] Calls per second:
> 18.909235668789808
> 2013-01-15 10:17:02.795 MobileSpecTest[1930:907] [LOG] Started exec
> benchmark with payload length: 32000
> 2013-01-15 10:17:07.785 MobileSpecTest[1930:907] [LOG] Calls per second:
> 18.59628074385123
> 2013-01-15 10:17:14.402 MobileSpecTest[1930:907] [LOG] Started exec
> benchmark with payload length: 32
> 2013-01-15 10:17:19.402 MobileSpecTest[1930:907] [LOG] Calls per second:
> 265.89364254298283
> 2013-01-15 10:17:20.651 MobileSpecTest[1930:907] [LOG] Started exec
> benchmark with payload length: 32
> 2013-01-15 10:17:25.651 MobileSpecTest[1930:907] [LOG] Calls per second:
> 263.69452219112355
> 2013-01-15 10:17:36.187 MobileSpecTest[1930:907] [LOG] Started exec
> benchmark with payload length: 1024
> 2013-01-15 10:17:41.189 MobileSpecTest[1930:907] [LOG] Calls per second:
> 185.8141858141858
> 2013-01-15 10:17:44.701 MobileSpecTest[1930:907] [LOG] Started exec
> benchmark with payload length: 1024
> 2013-01-15 10:17:49.699 MobileSpecTest[1930:907] [LOG] Calls per second:
> 186.9626074785043
> 2013-01-15 10:17:51.084 MobileSpecTest[1930:907] [LOG] Started exec
> benchmark with payload length: 1024
> 2013-01-15 10:17:56.082 MobileSpecTest[1930:907] [LOG] Calls per second:
> 184.8
>
> (2) NO BASE64 ENCODE
>
> 2013-01-15 10:12:12.387 MobileSpecTest[1905:907] Multi-tasking -> Device:
> YES, App: YES
> 2013-01-15 10:12:12.892 MobileSpecTest[1905:907] [LOG] Device = iOS 6.0.2
> 2013-01-15 10:12:21.462 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 1
> 2013-01-15 10:12:22.458 MobileSpecTest[1905:907] [LOG] Calls per second:
> 267
> 2013-01-15 10:12:32.212 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 32000
> 2013-01-15 10:12:37.204 MobileSpecTest[1905:907] [LOG] Calls per second:
> 15.18784972022382
> 2013-01-15 10:12:38.898 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 32000
> 2013-01-15 10:12:43.897 MobileSpecTest[1905:907] [LOG] Calls per second:
> 15.163607342378292
> 2013-01-15 10:12:51.802 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 32
> 2013-01-15 10:12:56.801 MobileSpecTest[1905:907] [LOG] Calls per second:
> 261.8952419032387
> 2013-01-15 10:12:58.534 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 32
> 2013-01-15 10:13:03.533 MobileSpecTest[1905:907] [LOG] Calls per second:
> 262.89484206317474
> 2013-01-15 10:14:19.153 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 1
> 2013-01-15 10:14:24.154 MobileSpecTest[1905:907] [LOG] Calls per second:
> 269.43833699780134
> 2013-01-15 10:14:27.605 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 1
> 2013-01-15 10:14:32.604 MobileSpecTest[1905:907] [LOG] Calls per second:
> 272.34553089382126
> 2013-01-15 10:14:42.134 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 1024
> 2013-01-15 10:14:47.132 MobileSpecTest[1905:907] [LOG] Calls per second:
> 173.6
> 2013-01-15 10:14:49.217 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 1024
> 2013-01-15 10:14:54.216 MobileSpecTest[1905:907] [LOG] Calls per second:
> 173.8
> 2013-01-15 10:14:55.518 MobileSpecTest[1905:907] [LOG] Started exec
> benchmark with payload length: 1024
> 2013-01-15 10:15:00.520 MobileSpecTest[1905:907] [LOG] Calls per second:
> 173.22677322677322
>
>
> A note about the data: I was using bytes from 0..255 evenly distributed,
> and the JSON serialization of that looks
> like: "\u0000\u0001\u0002\u0003\u0004\u0005\u0006\u0007\b\t\n\u000b\f\r\u000e\u000f\u0010\u0011\u0012\u0013\u0014\u0015\u0016\u0017\u0018\u0019\u001a\u001b\u001c\u001d\u001e\u001f
> !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~
> € ‚ƒ„…†‡ˆ‰Š‹Œ Ž ‘’“”•–—˜™š›œ žŸ
> ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ"
>
> Which is a 400 character length string.  Base64 encoding 256 characters is
> a 342 character string.  However, if the data were not evenly distributed,
> and instead consisted mostly of readable characters (as is the case with
> many protocols) then non-base64 encoded would be faster.
>
> I've left the base64 encoded version, assuming binary data will be random,
> and also because we already support sending raw strings across the bridge,
> so a plugin developer can do that if its more efficient.
>
> Either way, this is terribly slow, and I'll have to start getting creative.
>
> Finally, there is a bug where it seems that data larger than ~64k wont
> transfer across the bridge at all.  Andrew, is there a known limit to how
> much payload exec can send across?
>
> -Michal
>
>
>
> On Wed, Jan 9, 2013 at 3:57 PM, Don Coleman <don.coleman@gmail.com> wrote:
>
>> For the phonegap-nfc plugin I send binary data between native and
>> javascript as a byte array in JSON.
>>
>>     static JSONArray byteArrayToJSON(byte[] bytes) {
>>         JSONArray json = new JSONArray();
>>         for (int i = 0; i < bytes.length; i++) {
>>             json.put(bytes[i]);
>>         }
>>         return json;
>>     }
>>
>>
>> On Wed, Jan 9, 2013 at 3:00 PM, Andrew Grieve <agrieve@chromium.org>
>> wrote:
>> >
>> > On Wed, Jan 9, 2013 at 10:15 AM, Michal Mocny <mmocny@chromium.org>
>> wrote:
>> >
>> > > On Tue, Jan 8, 2013 at 10:22 PM, Andrew Grieve <agrieve@chromium.org>
>> > > wrote:
>> > >
>> > > > Rather than "binary data", I think we should focus strictly on
>> > > > "ArrayBuffer". I think the only other binary data concept is Blob,
>> and
>> > > Blob
>> > > > is easy to create from an ArrayBuffer.
>> > > >
>> > > > As for how to have one of these show up in a JSON object, I think
>> it will
>> > > > be tough to make this work. Most platforms assume that you can
>> paste the
>> > > > JSON into an eval / JSON.parse, and it will come out correctly. That
>> > > can't
>> > > > be the case for ArrayBuffers. Maybe we should just not support
>> this. The
>> > > > use-case here is being able to pass multiple values to callbacks.
>> For
>> > > > ArrayBuffers, maybe we can passing the usual payload plus an
>> optional
>> > > > ArrayBuffer as an additional parameter. I don't think it would be
>> > > necessary
>> > > > to have multiple ArrayBuffers for a single callback...
>> > > >
>> > >
>> > > To understand correct, does this mean that you propose a single
>> ArrayBuffer
>> > > instead of an arguments list, or, to add a 6th parameter to exec
>> which is
>> > > the one and only ArrayBuffer?
>> > >
>> > > I propose yet another option, maybe you can any number of
>> ArrayBuffers in
>> > > the arguments list, but not embedded in any subobject.  Thus,
>> semantically
>> > > the arguments array stops being JSON serializable, and the individual
>> > > arguments are either ArrayBuffers or json-serializable (this also
>> leaves
>> > > door open to add more types in the future).
>> > >
>> >
>> > Hmm, I wasn't even thinking about the JS->native case actually. I was
>> only
>> > thinking about native->JS.
>> > What you propose sounds good for JS->Native.
>> >
>> >
>> > >
>> > >
>> > > >
>> > > >
>> > > > My ideas for transferring this over the bridge on iOS include:
>> > >
>> > > - an array of numbers
>> > > > - a string of shorts encoded as chars with unicode chars encoded as
>> > > \uXXXX
>> > > > - a URL that the client can fetch via XHR and we can return the
>> binary
>> > > data
>> > > > via CDVURLProtocol (advantage here is that XHR2 can request the
>> payload
>> > > as
>> > > > an ArrayBuffer instead of having to construct the ArrayBuffer
>> > > > after-the-fact
>> > > >
>> > > interesting idea!
>> > >
>> > >
>> > > >
>> > > > For Android, the string of short-encoded chars is probably the best
>> bet
>> > > > since the bridge natively supports transferring strings without
>> having to
>> > > > parse them as JS literals.
>> > > >
>> > > >
>> > > >
>> > > > On Tue, Jan 8, 2013 at 4:05 PM, Michal Mocny <mmocny@google.com>
>> wrote:
>> > > >
>> > > > > Created https://issues.apache.org/jira/browse/CB-2173
>> > > > >
>> > > > >
>> > > > > On Tue, Jan 8, 2013 at 3:52 PM, Michal Mocny <mmocny@google.com>
>> > > wrote:
>> > > > >
>> > > > > > Background: I've been working on implementing chrome.socket
[1]
>> for
>> > > > > mobile
>> > > > > > chrome apps [2] and porting circ (an irc chrome app) [3].
>>  Teaser
>> > > > video:
>> > > > > > https://docs.google.com/open?id=0B0UdPHoQPXheTzhTZXZHUlpGWHM
(this
>> > > is
>> > > > > > still in its infancy, and I certainly do plan to submit
a
>> version of
>> > > > the
>> > > > > > socket api that will work in cordova without any mobile
chrome
>> app
>> > > > > magic).
>> > > > > >
>> > > > > > As part of doing that, I had to implement sending binary
data
>> across
>> > > > the
>> > > > > > exec bridge (on ios for now), and I think it may be a good
idea
>> to
>> > > just
>> > > > > add
>> > > > > > that functionality to cordova core.  My current implementation
>> is
>> > > > > certainly
>> > > > > > not the best, so I will extend the exec bridge echo benchmarks
>> to
>> > > test
>> > > > > > binary transfer speeds and we can improve over time.
>> > > > > >
>> > > > > > My first step would be to implement helpers in js/ios/android
>> which
>> > > > > plugin
>> > > > > > devs can call manually to serialize/deserialize binary data
>> into/from
>> > > > > > whatever magical form is most efficient.
>> > > > > >
>> > > > > > Next, I would try to automate these helpers from within
the
>> exec call
>> > > > > > (e.g. iterate arguments looking for ArrayBuffer/Blob/etc
types
>> before
>> > > > > json
>> > > > > > serialization).  As part of doing this, I would need to
add
>> "hints"
>> > > > about
>> > > > > > the underlying argument types/some clever way to encode
semantic
>> > > > > > information about the arguments list, which we currently
do not
>> do.
>> > > > > >
>> > > > > > If anyone has any suggestions or objections please let me
know!
>> > > > > >
>> > > > > > -Michal
>> > > > > >
>> > > > > > [1] http://developer.chrome.com/apps/socket.html
>> > > > > > [2]
>> > > > > >
>> > > > >
>> > > >
>> > >
>> https://github.com/MobileChromeApps/chrome-cordova/blob/master/api/chrome/socket.jsand
>> > > > > >
>> > >
>> https://github.com/MobileChromeApps/chrome-cordova/tree/master/plugins
>> > > > > > [3] https://github.com/flackr/circ
>> > > > > >
>> > > > >
>> > > >
>> > >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message