qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jimmy Jones" <jimmyjon...@gmx.co.uk>
Subject UTF8 / binary strings in dynamic languages
Date Tue, 20 Aug 2013 20:46:00 GMT
Hi,

Darryl tells me Justin is going to post about handling utf8/binary strings in dynamic languages,
but I'm going to slightly jump the gun and post about what Darryl and I have been looking
at.

AFAIK in perl, if you include unicode characters in a string it'll set the utf8 flag. If you
don't include any unicode characters (eg. 7 bit ascii, or raw bytes) the flag won't be set.
So given a perl scalar that doesn't contain any utf8 characters, you don't know if its a textual
string (str16) or a binary string (vbin). There is a is_utf8_string function, but that'll
only tell you if the string would be valid utf8, but it could be a binary string that happens
to be valid utf8, so that's not really safe.

My observations of the current perl bindings: Despite what the perlguts page says, hv_store/fetch
support both utf8 and binary keys's by using a +ve/-ve length parameter (see perlapi page).
We currently don't handle that (and looks from perl bug #79074 that HeSVKEY_force that we
use doesn't handle utf8 either). We also dont support NULL within keys.

Jimmy

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message