qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gordon Sim <g...@redhat.com>
Subject Re: UTF8 / binary strings in dynamic languages
Date Wed, 21 Aug 2013 10:30:12 GMT
On 08/20/2013 09:46 PM, Jimmy Jones wrote:
> Hi,
>
> Darryl tells me Justin is going to post about handling utf8/binary
> strings in dynamic languages,

Justin, when you do so, can I request that you do so on the user list as 
there may be others there using these APIs with a view or just 
interested in the discussion.

> but I'm going to slightly jump the gun
> and post about what Darryl and I have been looking at.
>
> AFAIK in perl, if you include unicode characters in a string it'll
> set the utf8 flag. If you don't include any unicode characters (eg. 7
> bit ascii, or raw bytes) the flag won't be set. So given a perl
> scalar that doesn't contain any utf8 characters, you don't know if
> its a textual string (str16) or a binary string (vbin). There is a
> is_utf8_string function, but that'll only tell you if the string
> would be valid utf8, but it could be a binary string that happens to
> be valid utf8, so that's not really safe.

You can explicitly mark it as utf8 using utf8::upgrade() though, right? 
Certainly I tried that in a simple test and the property in question was 
then sent as str16.

> My observations of the current perl bindings: Despite what the
> perlguts page says, hv_store/fetch support both utf8 and binary
> keys's by using a +ve/-ve length parameter (see perlapi page). We
> currently don't handle that (and looks from perl bug #79074 that
> HeSVKEY_force that we use doesn't handle utf8 either). We also dont
> support NULL within keys.
>
> Jimmy
>
> ---------------------------------------------------------------------
>
>
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message