qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fraser Adams <fraser.ad...@blueyonder.co.uk>
Subject Re: pn_data_format
Date Sat, 14 Feb 2015 07:40:12 GMT
Hello Mikhail,
I'm not an expert on pn_data_format and TBH I've never used it in pure C 
code, only to write other language bindings. I don't know if it's 
possible to find out beforehand. My guess is probably not - the python 
binding (which is fairly canonical) looks like this:

   def format(self):
     sz = 16
     while True:
       err, result = pn_data_format(self._data, sz)
       if err == PN_OVERFLOW:
         sz *= 2
         return result

And I borrowed that pattern when I did the JavaScript binding, which 
looks like this:

  * Format the encoded AMQP Data into a string representation and return it.
  * @method format
  * @memberof! proton.Data#
  * @returns {string} a formatted string representation of the encoded Data.
_Data_['format'] = function() {
     var sp = Runtime.stackSave();
     var sizeptr = allocate(4, 'i32', ALLOC_STACK);

     var size = 1024; // Pass by reference variable - need to use 
setValue to initialise it.
     while (true) {
         setValue(sizeptr, size, 'i32'); // Set pass by reference variable.
         var bytes = _malloc(size);   // Allocate storage from 
emscripten heap.
         var err = _pn_data_format(this._data, bytes, sizeptr);
         var size = getValue(sizeptr, 'i32'); // Dereference the real 
size value;

         if (err === Module['Error']['OVERFLOW']) {
             size *= 2;
         } else {
             var string = Pointer_stringify(bytes, size);
             // Tidy up the memory that we allocated on emscripten's stack.
             return string;

So basically the "pattern" is to allocate some storage and call 
pn_data_format passing in the allocated storage and the size of the 
allocated storage - the latter is a pass by reference parameter.

You need to check the error condition and if it's PN_OVERFLOW you need 
to free your original storage and try again with a bigger block - usual 
pattern is to double the size.

I don't *think* there's any way to know a priori how much storage you 
are going to need because it's the call to pn_data_format that is 
actually "deserialising" the internal pn_data_t values into a pretty 
formatted form, so it doesn't actually know how big it's going to be 
until it has tried. In other words even if you could get the pointer 
data->str its value isn't useful until pn_data_format is actually being 
called - I think it's the private method pni_data_inspectify that starts 
that all off - if you look at the codec.c code you'll see:

int pn_data_format(pn_data_t *data, char *bytes, size_t *size)
   int err = pni_data_inspectify(data);
   if (err) return err;
   if (pn_string_size(data->str) >= *size) {
     return PN_OVERFLOW;
   } else {
     pn_string_put(data->str, bytes);
     *size = pn_string_size(data->str);
     return 0;


static int pni_data_inspectify(pn_data_t *data)
   int err = pn_string_set(data->str, "");
   if (err) return err;
   return pn_data_inspect(data, data->str);

so inspectify initialises the internal string then calls inspect (which 
in turn calls pni_data_traverse which, as its name suggests crawls the 
internal data graph deserialising into the internal string).

Having looked at the code I'm now pretty sure that it's not possible to 
do what you'd like and the only option is to check PN_OVERFLOW and 
realloc as necessary.

Hope that helps,

On 13/02/15 21:22, Michael Ivanov wrote:
> Hallo,
> I am trying to use proton routine for data formatting.
> pn_data_format checks suire of provided buffer and in case it is not
> enough just refuses to copy anything at all.
> Is it possible to find out how much does it need beforehand?
> Ideally would be if one had the possibility to get the pointer to this
> internal string (data->str) which already holds the string representation
> of data. Does this option exist?
> Best regards,

To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

View raw message