avro-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dcrea...@apache.org
Subject svn commit: r1146546 [1/5] - in /avro/trunk/lang/c: docs/ src/ src/avro/ tests/
Date Thu, 14 Jul 2011 02:35:09 GMT
Author: dcreager
Date: Thu Jul 14 02:35:04 2011
New Revision: 1146546

URL: http://svn.apache.org/viewvc?rev=1146546&view=rev
Log:
AVRO-837. C: Avro value interface

This patch adds a new interface for Avro values.  Eventually, each part
of the library that operates on Avro data will work with any C type that
implements this interface, rather than only working on avro_datum_t
instances.

As of right now, we have a definition of the interface itself (along
with a bunch of macros that make it easier to call the interface methods
on a particular instance).  We also have a “generic” implementation of
the interface, which will serve as a replacement for avro_datum_t.

There is also an avro_value_t implementation that wraps avro_datum_t
instances.  This will let us migrate code slowly to the new interface,
where necessary.

As part of all of this, I've broken out the monolithic avro.h header
file into separate modular headers.  You can still include <avro.h> on
its own to pick up all of the parts.

Lastly, there's now a test program that compares the performance of the
old and new APIs.

Added:
    avro/trunk/lang/c/src/array.c
    avro/trunk/lang/c/src/avro/allocation.h
    avro/trunk/lang/c/src/avro/basics.h
    avro/trunk/lang/c/src/avro/data.h
    avro/trunk/lang/c/src/avro/errors.h
    avro/trunk/lang/c/src/avro/generic.h
    avro/trunk/lang/c/src/avro/io.h
    avro/trunk/lang/c/src/avro/legacy.h
    avro/trunk/lang/c/src/avro/schema.h
    avro/trunk/lang/c/src/avro/value.h
    avro/trunk/lang/c/src/datum_value.c
    avro/trunk/lang/c/src/generic-array.c
    avro/trunk/lang/c/src/generic-enum.c
    avro/trunk/lang/c/src/generic-fixed.c
    avro/trunk/lang/c/src/generic-map.c
    avro/trunk/lang/c/src/generic-primitives.c
    avro/trunk/lang/c/src/generic-record.c
    avro/trunk/lang/c/src/generic-union.c
    avro/trunk/lang/c/src/generic.c
    avro/trunk/lang/c/src/map.c
    avro/trunk/lang/c/src/memoize.c
    avro/trunk/lang/c/src/string.c
    avro/trunk/lang/c/src/value-read.c
    avro/trunk/lang/c/src/value-sizeof.c
    avro/trunk/lang/c/src/value-write.c
    avro/trunk/lang/c/src/value.c
    avro/trunk/lang/c/src/wrapped-buffer.c
    avro/trunk/lang/c/tests/test_avro_values.c
    avro/trunk/lang/c/tests/test_data_structures.c
Removed:
    avro/trunk/lang/c/src/allocation.h
    avro/trunk/lang/c/src/avro_errors.h
Modified:
    avro/trunk/lang/c/docs/index.txt
    avro/trunk/lang/c/src/.gitignore
    avro/trunk/lang/c/src/CMakeLists.txt
    avro/trunk/lang/c/src/Makefile.am
    avro/trunk/lang/c/src/allocation.c
    avro/trunk/lang/c/src/avro.h
    avro/trunk/lang/c/src/avro/consumer.h
    avro/trunk/lang/c/src/avro_private.h
    avro/trunk/lang/c/src/datafile.c
    avro/trunk/lang/c/src/datum.c
    avro/trunk/lang/c/src/datum.h
    avro/trunk/lang/c/src/datum_json.c
    avro/trunk/lang/c/src/datum_read.c
    avro/trunk/lang/c/src/datum_size.c
    avro/trunk/lang/c/src/datum_skip.c
    avro/trunk/lang/c/src/datum_validate.c
    avro/trunk/lang/c/src/datum_write.c
    avro/trunk/lang/c/src/dump.c
    avro/trunk/lang/c/src/encoding.h
    avro/trunk/lang/c/src/encoding_binary.c
    avro/trunk/lang/c/src/errors.c
    avro/trunk/lang/c/src/io.c
    avro/trunk/lang/c/src/resolver.c
    avro/trunk/lang/c/src/schema.c
    avro/trunk/lang/c/src/schema.h
    avro/trunk/lang/c/src/schema_specific.c
    avro/trunk/lang/c/src/st.c
    avro/trunk/lang/c/tests/.gitignore
    avro/trunk/lang/c/tests/CMakeLists.txt
    avro/trunk/lang/c/tests/Makefile.am
    avro/trunk/lang/c/tests/generate_interop_data.c
    avro/trunk/lang/c/tests/performance.c
    avro/trunk/lang/c/tests/test_avro_data.c
    avro/trunk/lang/c/tests/test_avro_schema.c
    avro/trunk/lang/c/tests/test_avro_schema_names.c

Modified: avro/trunk/lang/c/docs/index.txt
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/docs/index.txt?rev=1146546&r1=1146545&r2=1146546&view=diff
==============================================================================
--- avro/trunk/lang/c/docs/index.txt (original)
+++ avro/trunk/lang/c/docs/index.txt Thu Jul 14 02:35:04 2011
@@ -58,6 +58,501 @@ We're always looking for contributions s
 feel free to http://hadoop.apache.org/avro/[submit patches to the
 project].
 
+
+== Error reporting
+
+Most functions in the Avro C library return a single +int+ status code.
+Following the POSIX _errno.h_ convention, a status code of 0 indicates
+success.  Non-zero codes indiciate an error condition.  Some functions
+return a pointer value instead of an +int+ status code; for these
+functions, a +NULL+ pointer indicates an error.
+
+You can retrieve
+a string description of the most recent error using the +avro_strerror+
+function:
+
+[source,c]
+----
+avro_schema_t  schema = avro_schema_string();
+if (schema == NULL) {
+    fprintf(stderr, "Error was %s\n", avro_strerror());
+}
+----
+
+
+== Avro values
+
+Starting with version 1.6.0, the Avro C library has a new API for
+handling Avro data.  To help distinguish between the two APIs, we refer
+to the old one as the _legacy_ or _datum_ API, and the new one as the
+_value_ API.  (These names come from the names of the C types used to
+represent Avro data in the corresponding API — +avro_datum_t+ and
++avro_value_t+.)  The legacy API is still present, but it's deprecated —
+you shouldn't use the +avro_datum_t+ type or the +avro_datum_*+
+functions in new code.
+
+One main benefit of the new value API is that you can treat any existing
+C type as an Avro value; you just have to provide a custom
+implementation of the value interface.  In addition, we provide a
+_generic_ value implementation; “generic”, in this sense, meaning that
+this single implementation works for instances of any Avro schema type.
+Finally, we also provide a wrapper implementation for the deprecated
++avro_datum_t+ type, which lets you gradually transition to the new
+value API.
+
+
+=== Avro value interface
+
+You interact with Avro values using the _value interface_, which defines
+methods for setting and retrieving the contents of an Avro value.  An
+individual value is represented by an instance of the +avro_value_t+
+type.
+
+This section provides an overview of the methods that you can call on an
++avro_value_t+ instance.  There are quite a few methods in the value
+interface, but not all of them make sense for all Avro schema types.
+For instance, you won't be able to call +avro_value_set_boolean+ on an
+Avro array value.  If you try to call an inappropriate method, we'll
+return an +EINVAL+ error code.
+
+Note that the functions in this section apply to _all_ Avro values,
+regardless of which value implementation is used under the covers.  This
+section doesn't describe how to _create_ value instances, since those
+constructors will be specific to a particular value implementation.
+
+
+==== Common methods
+
+There are a handful of methods that can be used with any value,
+regardless of which Avro schema it's an instance of:
+
+[source,c]
+----
+#include <stdbool.h>
+#include <avro.h>
+
+avro_type_t avro_value_get_type(const avro_value_t *value);
+avro_schema_t avro_value_get_schema(const avro_value_t *value);
+
+bool avro_value_equal(const avro_value_t *v1, const avro_value_t *v2);
+bool avro_value_equal_fast(const avro_value_t *v1, const avro_value_t *v2);
+
+int avro_value_copy(avro_value_t *dest, const avro_value_t *src);
+int avro_value_copy_fast(avro_value_t *dest, const avro_value_t *src);
+
+int avro_value_reset(avro_value_t *value);
+----
+
+The +get_type+ and +get_schema+ methods can be used to get information
+about what kind of Avro value a given +avro_value_t+ instance
+represents.  (For +get_schema+, you borrow the value's reference to the
+schema; if you need to save it and ensure that it outlives the value,
+you need to call +avro_schema_incref+ on it.)
+
+The +equal+ and +equal_fast+ methods compare two values for equality.
+The two values do _not_ have to have the same value implementations, but
+they _do_ have to be instances of the same schema.  (Not _equivalent_
+schemas; the _same_ schema.)  The +equal+ method checks that the schemas
+match; the +equal_fast+ method assumes that they do.
+
+The +copy+ and +copy_fast+ methods copy the contents of one Avro value
+into another.  (Where possible, this is done without copying the actual
+content of a +bytes+, +string+, or +fixed+ value, using the
++avro_wrapped_buffer_t+ functions described in the next section.)  Like
++equal+, the two values must have the same schema; +copy+ checks this,
+while +copy_fast+ assumes it.
+
+The +reset+ method “clears out” an +avro_value_t instance, making sure
+that it's ready to accept the contents of a new value.  For scalars,
+this is usually a no-op, since the new value will just overwrite the old
+one.  For arrays and maps, this removes any existing elements from the
+container, so that we can append the elements of the new value.  For
+records and unions, this just recursively resets the fields or current
+branch.
+
+
+==== Scalar values
+
+The simplest case is handling instances of the scalar Avro schema types.
+In Avro, the scalars are all of the primitive schema types, as well as
++enum+ and +fixed+ — i.e., anything that can't contain another Avro
+value.  Note that we use standard C99 types to represent the primitive
+contents of an Avro scalar.
+
+To retrieve the contents of an Avro scalar, you can use one of the
+_getter_ methods:
+
+[source,c]
+----
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <avro.h>
+
+int avro_value_get_boolean(const avro_value_t *value, bool *dest);
+int avro_value_get_bytes(const avro_value_t *value,
+                         const void **dest, size_t *size);
+int avro_value_get_double(const avro_value_t *value, double *dest);
+int avro_value_get_float(const avro_value_t *value, float *dest);
+int avro_value_get_int(const avro_value_t *value, int32_t *dest);
+int avro_value_get_long(const avro_value_t *value, int64_t *dest);
+int avro_value_get_null(const avro_value_t *value);
+int avro_value_get_string(const avro_value_t *value,
+                          const char **dest, size_t *size);
+int avro_value_get_enum(const avro_value_t *value, int *dest);
+int avro_value_get_fixed(const avro_value_t *value,
+                         const void **dest, size_t *size);
+----
+
+For the most part, these should be self-explanatory.  For +bytes+,
++string+, and +fixed+ values, the pointer to the underlying content is
++const+ — you aren't allowed to modify the contents directly.  We
+guarantee that the content of a +string+ will be NUL-terminated, so you
+can use it as a C string as you'd expect.  The +size+ returned for a
++string+ object will include the NUL terminator; it will be one more
+than you'd get from calling +strlen+ on the content.
+
+Also, for +bytes+, +string+, and +fixed+, the +dest+ and +size+
+parameters are optional; if you only want to determine the length of a
++bytes+ value, you can use:
+
+[source,c]
+----
+avro_value_t  *value = /* from somewhere */;
+size_t  size;
+avro_value_get_bytes(value, NULL, &size);
+----
+
+To set the contents of an Avro scalar, you can use one of the _setter_
+methods:
+
+[source,c]
+----
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <avro.h>
+
+int avro_value_set_boolean(avro_value_t *value, bool src);
+int avro_value_set_bytes(avro_value_t *value,
+                         void *buf, size_t size);
+int avro_value_set_double(avro_value_t *value, double src);
+int avro_value_set_float(avro_value_t *value, float src);
+int avro_value_set_int(avro_value_t *value, int32_t src);
+int avro_value_set_long(avro_value_t *value, int64_t src);
+int avro_value_set_null(avro_value_t *value);
+int avro_value_set_string(avro_value_t *value, const char *src);
+int avro_value_set_string_len(avro_value_t *value,
+                              const char *src, size_t size);
+int avro_value_set_enum(avro_value_t *value, int src);
+int avro_value_set_fixed(avro_value_t *value,
+                         void *buf, size_t size);
+----
+
+These are also straightforward.  For +bytes+, +string+, and +fixed+
+values, the +set+ methods will make a copy of the underlying data.  For
++string+ values, the content must be NUL-terminated.  You can use
++set_string_len+ if you already know the length of the string content;
+the length you pass in should include the NUL terminator.  If you call
++set_string+, then we'll use +strlen+ to calculate the length.
+
+For +fixed+ values, the +size+ must match what's expected by the value's
+underlying +fixed+ schema; if the sizes don't match, you'll get an error
+code.
+
+If you don't want to copy the contents of a +bytes+, +string+, or
++fixed+ value, you can use the _giver_ and _grabber_ functions:
+
+[source,c]
+----
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <avro.h>
+
+typedef void
+(*avro_buf_free_t)(void *ptr, size_t sz, void *user_data);
+
+int avro_value_give_bytes(avro_value_t *value, avro_wrapped_buffer_t *src);
+int avro_value_give_string_len(avro_value_t *value, avro_wrapped_buffer_t *src);
+int avro_value_give_fixed(avro_value_t *value, avro_wrapped_buffer_t *src);
+
+int avro_value_grab_bytes(const avro_value_t *value, avro_wrapped_buffer_t *dest);
+int avro_value_grab_string(const avro_value_t *value, avro_wrapped_buffer_t *dest);
+int avro_value_grab_fixed(const avro_value_t *value, avro_wrapped_buffer_t *dest);
+
+typedef struct avro_wrapped_buffer {
+    const void  *buf;
+    size_t  size;
+    void (*free)(avro_wrapped_buffer_t *self);
+    int (*copy)(avro_wrapped_buffer_t *dest,
+                const avro_wrapped_buffer_t *src,
+                size_t offset, size_t length);
+    int (*slice)(avro_wrapped_buffer_t *self,
+                 size_t offset, size_t length);
+} avro_wrapped_buffer_t;
+
+void
+avro_wrapped_buffer_free(avro_wrapped_buffer_t *buf);
+
+int
+avro_wrapped_buffer_copy(avro_wrapped_buffer_t *dest,
+                         const avro_wrapped_buffer_t *src,
+                         size_t offset, size_t length);
+
+int
+avro_wrapped_buffer_slice(avro_wrapped_buffer_t *self,
+                          size_t offset, size_t length);
+----
+
+The +give+ functions give control of an existing buffer to the value.
+(You should *not* try to free the +src+ wrapped buffer after calling
+this method.)  The +grab+ function fills in a wrapped buffer with a
+pointer to the contents of an Avro value.  (You *should* free the +dest+
+wrapped buffer when you're done with it.)
+
+The +avro_wrapped_buffer_t+ struct encapsulates the location and size of
+the existing buffer.  It also includes several methods.  The +free+
+method will be called when the content of the buffer is no longer
+needed.  The +slice+ method will be called when the wrapped buffer needs
+to be updated to point at a subset of what it pointed at before.  (This
+doesn't create a new wrapped buffer; it updates an existing one.)  The
++copy+ method will be called if the content needs to be copied.  Note
+that if you're wrapping a buffer with nice reference counting features,
+you don't need to perform an actual copy; you just need to ensure that
+the +free+ function can be called on both the original and the copy, and
+not have things blow up.
+
+The “generic” value implementation takes advantage of this feature; if
+you pass in a wrapped buffer with a +give+ method, and then retrieve it
+later with a +grab+ method, then we'll use the wrapped buffer's +copy+
+method to fill in the +dest+ parameter.  If your wrapped buffer
+implements a +slice+ method that updates reference counts instead of
+actually copying, then you've got nice zero-copy access to the contents
+of an Avro value.
+
+
+==== Compound values
+
+The following sections describe the getter and setter methods for
+handling compound Avro values.  All of the compound values are
+responsible for the storage of their children; this means that there
+isn't a method, for instance, that lets you add an existing
++avro_value_t+ to an array.  Instead, there's a method that creates a
+new, empty +avro_value_t+ of the appropriate type, adds it to the array,
+and returns it for you to fill in as needed.
+
+You also shouldn't try to free the child elements that are created this
+way; the container value is responsible for their life cycle.  The child
+element is guaranteed to be valid for as long as the container value
+is.  You'll usually define an +avro_value_t+ in the stack, and let it
+fall out of scope when you're done with it:
+
+[source,c]
+----
+avro_value_t  *array = /* from somewhere else */;
+
+{
+    avro_value_t  child;
+    avro_value_get_by_index(array, 0, &child, NULL);
+    /* do something interesting with the array element */
+}
+----
+
+
+==== Arrays
+
+There are three methods that can be used with array values:
+
+[source,c]
+----
+#include <stdlib.h>
+#include <avro.h>
+
+int avro_value_get_size(const avro_value_t *array, size_t *size);
+int avro_value_get_by_index(const avro_value_t *array, size_t index,
+                            avro_value_t *element, const char **unused);
+int avro_value_append(avro_value_t *array, avro_value_t *element,
+                      size_t *new_index);
+----
+
+The +get_size+ method returns the number of elements currently in the
+array.  The +get_by_index+ method fills in +element+ to point at the
+array element with the given index.  (You should use +NULL+ for the
++unused+ parameter; it's ignored for array values.)
+
+The +append+ method creates a new value, appends it to the array, and
+returns it in +element+.  If +new_index+ is given, then it will be
+filled in with the index of the new element.
+
+
+==== Maps
+
+There are four methods that can be used with map values:
+
+[source,c]
+----
+#include <stdlib.h>
+#include <avro.h>
+
+int avro_value_get_size(const avro_value_t *map, size_t *size);
+int avro_value_get_by_name(const avro_value_t *map, const char *key,
+                           avro_value_t *element, size_t *index);
+int avro_value_get_by_index(const avro_value_t *map, size_t index,
+                            avro_value_t *element, const char **key);
+int avro_value_add(avro_value_t *map,
+                   const char *key, avro_value_t *element,
+                   size_t *index, bool *is_new);
+----
+
+The +get_size+ method returns the number of elements currently in the
+map.  Map elements can be retrieved either by their key (+get_by_name+)
+or by their numeric index (+get_by_index+).  (Numeric indices in a map
+are based on the order that the elements were added to the map.)  In
+either case, the method takes in an optional output parameter that let
+you retrieve the index associated with a key, and vice versa.
+
+The +add+ method will add a new value to the map, if the given key isn't
+already present.  If the key is present, then the existing value with be
+returned.  The +index+ parameter, if given, will be filled in the
+element's index.  The +is_new+ parameter, if given, can be used to
+determine whether the mapped value is new or not.
+
+
+==== Records
+
+There are three methods that can be used with record values:
+
+[source,c]
+----
+#include <stdlib.h>
+#include <avro.h>
+
+int avro_value_get_size(const avro_value_t *record, size_t *size);
+int avro_value_get_by_index(const avro_value_t *record, size_t index,
+                            avro_value_t *element, const char **field_name);
+int avro_value_get_by_name(const avro_value_t *record, const char *field_name,
+                           avro_value_t *element, size_t *index);
+----
+
+The +get_size+ method returns the number of fields in the record.  (You
+can also get this by querying the value's schema, but for some
+implementations, this method can be faster.)
+
+The +get_by_index+ and +get_by_name+ functions can be used to retrieve
+one of the fields in the record, either by its ordinal position within
+the record, or by the name of the underlying field.  Like with maps, the
+methods take in an additional parameter that let you retrieve the index
+associated with a field name, and vice versa.
+
+When possible, it's recommended that you access record fields by their
+numeric index, rather than by their field name.  For most
+implementations, this will be more efficient.
+
+
+==== Unions
+
+There are three methods that can be used with union values:
+
+[source,c]
+----
+#include <avro.h>
+
+int avro_value_get_discriminant(const avro_value_t *union_val, int *disc);
+int avro_value_get_current_branch(const avro_value_t *union_val, avro_value_t *branch);
+int avro_value_set_branch(avro_value_t *union_val,
+                          int discriminant, avro_value_t *branch);
+----
+
+The +get_discriminant+ and +get_current_branch+ methods return the
+current state of the union value, without modifying which branch is
+currently selected.  The +set_branch+ method can be used to choose the
+active branch, filling in the +branch+ value to point at the branch's
+value instance.  (Most implementations will be smart enough to detect
+when the desired branch is already selected, so you should always call
+this method unless you can _guarantee_ that the right branch is already
+current.)
+
+
+=== Creating value instances
+
+Okay, so we've described how to interact with a value that you already
+have a pointer to, but how do you create one in the first place?  Since
+there can be many implementations of the value interface, this requires
+a couple of steps:
+
+1. Get an _implementation struct_ for the value implementation that you
+   want to use.  (This is represented by an +avro_value_iface_t+
+   pointer.)
+
+2. Use +avro_value_new+ to allocate instances of that value
+   implementation on the heap.
+
+3. Do whatever you need to the value (using the methods described in the
+   previous section).
+
+4. Free the value instance using +avro_value_free+.
+
+5. Free the implementation struct when you're done creating value
+   instances.
+
+These steps use the following functions:
+
+[source,c]
+----
+#include <avro.h>
+
+avro_value_iface_t *avro_value_iface_incref(avro_value_iface_t *iface);
+void avro_value_iface_decref(avro_value_iface_t *iface);
+
+int avro_value_new(const avro_value_iface_t *iface, avro_value_t *value);
+void avro_value_free(avro_value_t *val);
+----
+
+Note that for most value implementations, it's fine to reuse a single
++avro_value_t+ instance for multiple values, using the
++avro_value_reset+ function before filling in the instance for each
+value.  (This helps reduce the number of +malloc+ and +free+ calls that
+your application will make.)
+
+For most applications, you won't need to write your own value
+implementation; the Avro C library provides an efficient “generic”
+implementation, which supports the full range of Avro schema types.  You
+can use the following function to create an implementation struct for a
+particular schema:
+
+[source,c]
+----
+#include <avro.h>
+
+avro_value_iface_t *avro_generic_class_from_schema(avro_schema_t schema);
+----
+
+Combining all of this together, you might have the following snippet of
+code:
+
+[source,c]
+----
+avro_schema_t  schema = avro_schema_long();
+avro_value_iface_t  *iface = avro_generic_class_from_schema(schema);
+
+avro_value_t  val;
+avro_value_new(iface, &val);
+
+/* Generate Avro longs from 0-499 */
+int  i;
+for (i = 0; i < 500; i++) {
+    avro_value_reset(&val);
+    avro_value_set_long(&val, i);
+    /* do something with the value */
+}
+
+avro_value_free(&val);
+avro_value_iface_decref(iface);
+avro_schema_decref(schema);
+----
+
+
 == Reference Counting
 
 +Avro C+ does reference counting for all schema and data objects.

Modified: avro/trunk/lang/c/src/.gitignore
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/.gitignore?rev=1146546&r1=1146545&r2=1146546&view=diff
==============================================================================
--- avro/trunk/lang/c/src/.gitignore (original)
+++ avro/trunk/lang/c/src/.gitignore Thu Jul 14 02:35:04 2011
@@ -1 +1,2 @@
 avro-c.pc
+avropipe

Modified: avro/trunk/lang/c/src/CMakeLists.txt
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/CMakeLists.txt?rev=1146546&r1=1146545&r2=1146546&view=diff
==============================================================================
--- avro/trunk/lang/c/src/CMakeLists.txt (original)
+++ avro/trunk/lang/c/src/CMakeLists.txt Thu Jul 14 02:35:04 2011
@@ -18,12 +18,20 @@
 #
 
 set(AVRO_SRC
-    allocation.h
     allocation.c
+    array.c
     avro.h
+    avro/allocation.h
+    avro/basics.h
     avro/consumer.h
+    avro/data.h
+    avro/errors.h
+    avro/generic.h
+    avro/io.h
+    avro/legacy.h
     avro/refcount.h
-    avro_errors.h
+    avro/schema.h
+    avro/value.h
     avro_private.h
     consumer.c
     datafile.c
@@ -35,19 +43,36 @@ set(AVRO_SRC
     datum_size.c
     datum_skip.c
     datum_validate.c
+    datum_value.c
     datum_write.c
     dump.c
     dump.h
     encoding.h
     encoding_binary.c
     errors.c
+    generic.c
+    generic-array.c
+    generic-enum.c
+    generic-fixed.c
+    generic-map.c
+    generic-primitives.c
+    generic-record.c
+    generic-union.c
     io.c
+    map.c
+    memoize.c
     resolver.c
     schema.c
     schema.h
     schema_equal.c
     st.c
     st.h
+    string.c
+    value.c
+    value-read.c
+    value-sizeof.c
+    value-write.c
+    wrapped-buffer.c
 )
 
 set(JANSSON_SRC

Modified: avro/trunk/lang/c/src/Makefile.am
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/Makefile.am?rev=1146546&r1=1146545&r2=1146546&view=diff
==============================================================================
--- avro/trunk/lang/c/src/Makefile.am (original)
+++ avro/trunk/lang/c/src/Makefile.am Thu Jul 14 02:35:04 2011
@@ -7,18 +7,33 @@ pkgconfigdir = $(libdir)/pkgconfig
 pkgconfig_DATA = avro-c.pc
 
 nobase_include_HEADERS = avro.h \
+avro/allocation.h \
+avro/basics.h \
 avro/consumer.h \
-avro/refcount.h
+avro/data.h \
+avro/errors.h \
+avro/generic.h \
+avro/io.h \
+avro/legacy.h \
+avro/refcount.h \
+avro/schema.h \
+avro/value.h
 
 lib_LTLIBRARIES = libavro.la
 libavro_la_SOURCES = st.c st.h schema.c schema.h schema_equal.c \
+value.c generic.c value-read.c value-sizeof.c value-write.c \
+generic-array.c generic-enum.c generic-fixed.c generic-map.c \
+generic-primitives.c generic-record.c generic-union.c \
 datum.c datum_equal.c datum_validate.c datum_read.c datum_skip.c datum_write.c datum_size.c datum.h \
 datum_json.c \
+datum_value.c \
+array.c map.c memoize.c string.c \
 consumer.c resolver.c \
 io.c dump.c dump.h encoding_binary.c \
-allocation.h allocation.c \
+allocation.c \
+wrapped-buffer.c \
 avro_private.h encoding.h datafile.c \
-avro_errors.h errors.c
+errors.c
 libavro_la_LIBADD = $(top_builddir)/jansson/src/.libs/libjansson.a
 libavro_la_LDFLAGS = \
         -version-info $(LIBAVRO_VERSION) \

Modified: avro/trunk/lang/c/src/allocation.c
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/allocation.c?rev=1146546&r1=1146545&r2=1146546&view=diff
==============================================================================
--- avro/trunk/lang/c/src/allocation.c (original)
+++ avro/trunk/lang/c/src/allocation.c Thu Jul 14 02:35:04 2011
@@ -15,11 +15,12 @@
  * permissions and limitations under the License. 
  */
 
+#include <stdlib.h>
 #include <string.h>
 
-#include "avro.h"
 #include "avro_private.h"
-#include "allocation.h"
+#include "avro/allocation.h"
+#include "avro/data.h"
 
 static void *
 avro_default_allocator(void *ud, void *ptr, size_t osize, size_t nsize)
@@ -35,7 +36,7 @@ avro_default_allocator(void *ud, void *p
 	}
 }
 
-struct allocator_state  AVRO_CURRENT_ALLOCATOR = {
+struct avro_allocator_state  AVRO_CURRENT_ALLOCATOR = {
 	avro_default_allocator,
 	NULL
 };
@@ -88,7 +89,7 @@ void avro_str_free(char *str)
 
 
 void
-avro_alloc_free(void *ptr, size_t sz)
+avro_alloc_free_func(void *ptr, size_t sz)
 {
 	avro_free(ptr, sz);
 }

Added: avro/trunk/lang/c/src/array.c
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/array.c?rev=1146546&view=auto
==============================================================================
--- avro/trunk/lang/c/src/array.c (added)
+++ avro/trunk/lang/c/src/array.c Thu Jul 14 02:35:04 2011
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "avro/allocation.h"
+#include "avro/data.h"
+#include "avro/errors.h"
+
+
+void avro_raw_array_init(avro_raw_array_t *array, size_t element_size)
+{
+	memset(array, 0, sizeof(avro_raw_array_t));
+	array->element_size = element_size;
+}
+
+
+void avro_raw_array_done(avro_raw_array_t *array)
+{
+	if (array->data) {
+		avro_free(array->data, array->allocated_size);
+	}
+	memset(array, 0, sizeof(avro_raw_array_t));
+}
+
+
+void avro_raw_array_clear(avro_raw_array_t *array)
+{
+	array->element_count = 0;
+}
+
+
+int
+avro_raw_array_ensure_size(avro_raw_array_t *array, size_t desired_count)
+{
+	size_t  required_size = array->element_size * desired_count;
+	if (array->allocated_size >= required_size) {
+		return 0;
+	}
+
+	/*
+	 * Double the old size when reallocating.
+	 */
+
+	size_t  new_size;
+	if (array->allocated_size == 0) {
+		/*
+		 * Start with an arbitrary 10 items.
+		 */
+
+		new_size = 10 * array->element_size;
+	} else {
+		new_size = array->allocated_size * 2;
+	}
+
+	array->data = avro_realloc(array->data, array->allocated_size, new_size);
+	if (array->data == NULL) {
+		avro_set_error("Cannot allocate space in array for %zu elements",
+			       desired_count);
+		return ENOMEM;
+	}
+	array->allocated_size = new_size;
+
+	return 0;
+}
+
+
+void *avro_raw_array_append(avro_raw_array_t *array)
+{
+	int  rval;
+
+	rval = avro_raw_array_ensure_size(array, array->element_count + 1);
+	if (rval) {
+		return NULL;
+	}
+
+	size_t  offset = array->element_size * array->element_count;
+	array->element_count++;
+	return array->data + offset;
+}

Modified: avro/trunk/lang/c/src/avro.h
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/avro.h?rev=1146546&r1=1146545&r2=1146546&view=diff
==============================================================================
--- avro/trunk/lang/c/src/avro.h (original)
+++ avro/trunk/lang/c/src/avro.h Thu Jul 14 02:35:04 2011
@@ -23,426 +23,17 @@ extern "C" {
 #define CLOSE_EXTERN
 #endif
 
-#include <stdio.h>
-#include <stdint.h>
-
-/*
- * Allocation interface.  You can provide a custom allocator for the
- * library, should you wish.  The allocator is provided as a single
- * generic function, which can emulate the standard malloc, realloc, and
- * free functions.  The design of this allocation interface is inspired
- * by the implementation of the Lua interpreter.
- *
- * The ptr parameter will be the location of any existing memory
- * buffer.  The osize parameter will be the size of this existing
- * buffer.  If ptr is NULL, then osize will be 0.  The nsize parameter
- * will be the size of the new buffer, or 0 if the new buffer should be
- * freed.
- *
- * If nsize is 0, then the allocation function must return NULL.  If
- * nsize is not 0, then it should return NULL if the allocation fails.
- */
-
-typedef void *
-(*avro_allocator_t)(void *user_data, void *ptr, size_t osize, size_t nsize);
-
-void avro_set_allocator(avro_allocator_t alloc, void *user_data);
-
-/*
- * Returns a textual description of the last error condition returned by
- * an Avro function.
- */
-
-const char *avro_strerror(void);
-
-
-enum avro_type_t {
-	AVRO_STRING,
-	AVRO_BYTES,
-	AVRO_INT32,
-	AVRO_INT64,
-	AVRO_FLOAT,
-	AVRO_DOUBLE,
-	AVRO_BOOLEAN,
-	AVRO_NULL,
-	AVRO_RECORD,
-	AVRO_ENUM,
-	AVRO_FIXED,
-	AVRO_MAP,
-	AVRO_ARRAY,
-	AVRO_UNION,
-	AVRO_LINK
-};
-typedef enum avro_type_t avro_type_t;
-
-enum avro_class_t {
-	AVRO_SCHEMA,
-	AVRO_DATUM
-};
-typedef enum avro_class_t avro_class_t;
-
-struct avro_obj_t {
-	avro_type_t type;
-	avro_class_t class_type;
-	volatile int  refcount;
-};
-
-#define avro_classof(obj)     ((obj)->class_type)
-#define is_avro_schema(obj)   (obj && avro_classof(obj) == AVRO_SCHEMA)
-#define is_avro_datum(obj)    (obj && avro_classof(obj) == AVRO_DATUM)
-
-#define avro_typeof(obj)      ((obj)->type)
-#define is_avro_string(obj)   (obj && avro_typeof(obj) == AVRO_STRING)
-#define is_avro_bytes(obj)    (obj && avro_typeof(obj) == AVRO_BYTES)
-#define is_avro_int32(obj)    (obj && avro_typeof(obj) == AVRO_INT32)
-#define is_avro_int64(obj)    (obj && avro_typeof(obj) == AVRO_INT64)
-#define is_avro_float(obj)    (obj && avro_typeof(obj) == AVRO_FLOAT)
-#define is_avro_double(obj)   (obj && avro_typeof(obj) == AVRO_DOUBLE)
-#define is_avro_boolean(obj)  (obj && avro_typeof(obj) == AVRO_BOOLEAN)
-#define is_avro_null(obj)     (obj && avro_typeof(obj) == AVRO_NULL)
-#define is_avro_primitive(obj)(is_avro_string(obj) \
-                             ||is_avro_bytes(obj) \
-                             ||is_avro_int32(obj) \
-                             ||is_avro_int64(obj) \
-                             ||is_avro_float(obj) \
-                             ||is_avro_double(obj) \
-                             ||is_avro_boolean(obj) \
-                             ||is_avro_null(obj))
-#define is_avro_record(obj)   (obj && avro_typeof(obj) == AVRO_RECORD)
-#define is_avro_enum(obj)     (obj && avro_typeof(obj) == AVRO_ENUM)
-#define is_avro_fixed(obj)    (obj && avro_typeof(obj) == AVRO_FIXED)
-#define is_avro_named_type(obj)(is_avro_record(obj) \
-                              ||is_avro_enum(obj) \
-                              ||is_avro_fixed(obj))
-#define is_avro_map(obj)      (obj && avro_typeof(obj) == AVRO_MAP)
-#define is_avro_array(obj)    (obj && avro_typeof(obj) == AVRO_ARRAY)
-#define is_avro_union(obj)    (obj && avro_typeof(obj) == AVRO_UNION)
-#define is_avro_complex_type(obj) (!(is_avro_primitive(obj))
-#define is_avro_link(obj)     (obj && avro_typeof(obj) == AVRO_LINK)
-
-typedef struct avro_reader_t_ *avro_reader_t;
-typedef struct avro_writer_t_ *avro_writer_t;
-
-/*
- * schema 
- */
-typedef struct avro_obj_t *avro_schema_t;
-
-avro_schema_t avro_schema_string(void);
-avro_schema_t avro_schema_bytes(void);
-avro_schema_t avro_schema_int(void);
-avro_schema_t avro_schema_long(void);
-avro_schema_t avro_schema_float(void);
-avro_schema_t avro_schema_double(void);
-avro_schema_t avro_schema_boolean(void);
-avro_schema_t avro_schema_null(void);
-
-avro_schema_t avro_schema_record(const char *name, const char *space);
-avro_schema_t avro_schema_record_field_get(const avro_schema_t
-					   record, const char *field_name);
-const char *avro_schema_record_field_name(const avro_schema_t schema, int index);
-int avro_schema_record_field_get_index(const avro_schema_t schema,
-				       const char *field_name);
-avro_schema_t avro_schema_record_field_get_by_index
-(const avro_schema_t record, int index);
-int avro_schema_record_field_append(const avro_schema_t record,
-				    const char *field_name,
-				    const avro_schema_t type);
-size_t avro_schema_record_size(const avro_schema_t record);
-
-avro_schema_t avro_schema_enum(const char *name);
-const char *avro_schema_enum_get(const avro_schema_t enump,
-				 int index);
-int avro_schema_enum_get_by_name(const avro_schema_t enump,
-				 const char *symbol_name);
-int avro_schema_enum_symbol_append(const avro_schema_t
-				   enump, const char *symbol);
-
-avro_schema_t avro_schema_fixed(const char *name, const int64_t len);
-int64_t avro_schema_fixed_size(const avro_schema_t fixed);
-
-avro_schema_t avro_schema_map(const avro_schema_t values);
-avro_schema_t avro_schema_map_values(avro_schema_t map);
-
-avro_schema_t avro_schema_array(const avro_schema_t items);
-avro_schema_t avro_schema_array_items(avro_schema_t array);
-
-avro_schema_t avro_schema_union(void);
-size_t avro_schema_union_size(const avro_schema_t union_schema);
-int avro_schema_union_append(const avro_schema_t
-			     union_schema, const avro_schema_t schema);
-avro_schema_t avro_schema_union_branch(avro_schema_t union_schema,
-				       int branch_index);
-avro_schema_t avro_schema_union_branch_by_name
-(avro_schema_t union_schema, int *branch_index, const char *name);
-
-avro_schema_t avro_schema_link(avro_schema_t schema);
-avro_schema_t avro_schema_link_target(avro_schema_t schema);
-
-typedef struct avro_schema_error_t_ *avro_schema_error_t;
-int avro_schema_from_json(const char *jsontext,
-			  const int32_t len,
-			  avro_schema_t * schema, avro_schema_error_t * error);
-int avro_schema_to_json(const avro_schema_t schema, avro_writer_t out);
-
-int avro_schema_to_specific(avro_schema_t schema, const char *prefix);
-
-avro_schema_t avro_schema_get_subschema(const avro_schema_t schema,
-         const char *name);
-const char *avro_schema_name(const avro_schema_t schema);
-const char *avro_schema_type_name(const avro_schema_t schema);
-avro_schema_t avro_schema_copy(avro_schema_t schema);
-int avro_schema_equal(avro_schema_t a, avro_schema_t b);
-
-avro_schema_t avro_schema_incref(avro_schema_t schema);
-void avro_schema_decref(avro_schema_t schema);
-
-/*
- * io 
- */
-avro_reader_t avro_reader_file(FILE * fp);
-avro_writer_t avro_writer_file(FILE * fp);
-avro_reader_t avro_reader_memory(const char *buf, int64_t len);
-avro_writer_t avro_writer_memory(const char *buf, int64_t len);
-
-int avro_read(avro_reader_t reader, void *buf, int64_t len);
-int avro_skip(avro_reader_t reader, int64_t len);
-int avro_write(avro_writer_t writer, void *buf, int64_t len);
-
-void avro_writer_reset(avro_writer_t writer);
-int64_t avro_writer_tell(avro_writer_t writer);
-void avro_writer_flush(avro_writer_t writer);
-
-void avro_writer_dump(avro_writer_t writer, FILE * fp);
-void avro_reader_dump(avro_reader_t reader, FILE * fp);
-
-void avro_reader_free(avro_reader_t reader);
-void avro_writer_free(avro_writer_t writer);
-
-/*
- * datum 
- */
-
-/**
- * A function used to free a bytes, string, or fixed buffer once it is
- * no longer needed by the datum that wraps it.
- */
-
-typedef void
-(*avro_free_func_t)(void *ptr, size_t sz);
-
-/**
- * An avro_free_func_t that frees the buffer using the custom allocator
- * provided to avro_set_allocator.
- */
-
-void
-avro_alloc_free(void *ptr, size_t sz);
-
-/*
- * Datum constructors.  Each datum stores a reference to the schema that
- * the datum is an instance of.  The primitive datum constructors don't
- * need to take in an explicit avro_schema_t parameter, since there's
- * only one schema that they could be an instance of.  The complex
- * constructors do need an explicit schema parameter.
- */
-
-typedef struct avro_obj_t *avro_datum_t;
-avro_datum_t avro_string(const char *str);
-avro_datum_t avro_givestring(const char *str,
-			     avro_free_func_t free);
-avro_datum_t avro_bytes(const char *buf, int64_t len);
-avro_datum_t avro_givebytes(const char *buf, int64_t len,
-			    avro_free_func_t free);
-avro_datum_t avro_int32(int32_t i);
-avro_datum_t avro_int64(int64_t l);
-avro_datum_t avro_float(float f);
-avro_datum_t avro_double(double d);
-avro_datum_t avro_boolean(int8_t i);
-avro_datum_t avro_null(void);
-avro_datum_t avro_record(avro_schema_t schema);
-avro_datum_t avro_enum(avro_schema_t schema, int i);
-avro_datum_t avro_fixed(avro_schema_t schema,
-			const char *bytes, const int64_t size);
-avro_datum_t avro_givefixed(avro_schema_t schema,
-			    const char *bytes, const int64_t size,
-			    avro_free_func_t free);
-avro_datum_t avro_map(avro_schema_t schema);
-avro_datum_t avro_array(avro_schema_t schema);
-avro_datum_t avro_union(avro_schema_t schema,
-			int64_t discriminant, const avro_datum_t datum);
-
-/**
- * Returns the schema that the datum is an instance of.
- */
-
-avro_schema_t avro_datum_get_schema(const avro_datum_t datum);
-
-/*
- * Constructs a new avro_datum_t instance that's appropriate for holding
- * values of the given schema.
- */
-
-avro_datum_t avro_datum_from_schema(const avro_schema_t schema);
-
-/* getters */
-int avro_string_get(avro_datum_t datum, char **p);
-int avro_bytes_get(avro_datum_t datum, char **bytes, int64_t * size);
-int avro_int32_get(avro_datum_t datum, int32_t * i);
-int avro_int64_get(avro_datum_t datum, int64_t * l);
-int avro_float_get(avro_datum_t datum, float *f);
-int avro_double_get(avro_datum_t datum, double *d);
-int avro_boolean_get(avro_datum_t datum, int8_t * i);
-
-int avro_enum_get(const avro_datum_t datum);
-const char *avro_enum_get_name(const avro_datum_t datum);
-int avro_fixed_get(avro_datum_t datum, char **bytes, int64_t * size);
-int avro_record_get(const avro_datum_t record, const char *field_name,
-		    avro_datum_t * value);
-
-/*
- * A helper macro that extracts the value of the given field of a
- * record.
- */
-
-#define avro_record_get_field_value(rc, rec, typ, fname, ...)	\
-	do {							\
-		avro_datum_t  field = NULL;			\
-		(rc) = avro_record_get((rec), (fname), &field);	\
-		if (rc) break;					\
-		(rc) = avro_##typ##_get(field, __VA_ARGS__);	\
-	} while (0)
-
-
-int avro_map_get(const avro_datum_t datum, const char *key,
-		 avro_datum_t * value);
-/*
- * For maps, the "index" for each entry is based on the order that they
- * were added to the map.
- */
-int avro_map_get_key(const avro_datum_t datum, int index,
-		     const char **key);
-size_t avro_map_size(const avro_datum_t datum);
-int avro_array_get(const avro_datum_t datum, int64_t index, avro_datum_t * value);
-size_t avro_array_size(const avro_datum_t datum);
-
-/*
- * These accessors allow you to query the current branch of a union
- * value, returning either the branch's discriminant value or the
- * avro_datum_t of the branch.  A union value can be uninitialized, in
- * which case the discriminant will be -1 and the datum NULL.
- */
-
-int64_t avro_union_discriminant(const avro_datum_t datum);
-avro_datum_t avro_union_current_branch(avro_datum_t datum);
-
-/* setters */
-int avro_string_set(avro_datum_t datum, const char *p);
-int avro_givestring_set(avro_datum_t datum, const char *p,
-			avro_free_func_t free);
-
-int avro_bytes_set(avro_datum_t datum, const char *bytes, const int64_t size);
-int avro_givebytes_set(avro_datum_t datum, const char *bytes,
-		       const int64_t size,
-		       avro_free_func_t free);
-
-int avro_int32_set(avro_datum_t datum, const int32_t i);
-int avro_int64_set(avro_datum_t datum, const int64_t l);
-int avro_float_set(avro_datum_t datum, const float f);
-int avro_double_set(avro_datum_t datum, const double d);
-int avro_boolean_set(avro_datum_t datum, const int8_t i);
-
-int avro_enum_set(avro_datum_t datum, const int symbol_value);
-int avro_enum_set_name(avro_datum_t datum, const char *symbol_name);
-int avro_fixed_set(avro_datum_t datum, const char *bytes, const int64_t size);
-int avro_givefixed_set(avro_datum_t datum, const char *bytes,
-		       const int64_t size,
-		       avro_free_func_t free);
-
-int avro_record_set(avro_datum_t record, const char *field_name,
-		    avro_datum_t value);
-
-/*
- * A helper macro that sets the value of the given field of a record.
- */
-
-#define avro_record_set_field_value(rc, rec, typ, fname, ...)	\
-	do {							\
-		avro_datum_t  field = NULL;			\
-		(rc) = avro_record_get((rec), (fname), &field);	\
-		if (rc) break;					\
-		(rc) = avro_##typ##_set(field, __VA_ARGS__);	\
-	} while (0)
-
-int avro_map_set(avro_datum_t map, const char *key,
-		 avro_datum_t value);
-int avro_array_append_datum(avro_datum_t array_datum,
-			    avro_datum_t datum);
-
-/*
- * This function selects the active branch of a union value, and can be
- * safely called on an existing union to change the current branch.  If
- * the branch changes, we'll automatically construct a new avro_datum_t
- * for the new branch's schema type.  If the desired branch is already
- * the active branch of the union, we'll leave the existing datum
- * instance as-is.  The branch datum will be placed into the "branch"
- * parameter, regardless of whether we have to create a new datum
- * instance or not.
- */
-
-int avro_union_set_discriminant(avro_datum_t unionp,
-				int discriminant,
-				avro_datum_t *branch);
-
-/* reference counting */
-avro_datum_t avro_datum_incref(avro_datum_t value);
-void avro_datum_decref(avro_datum_t value);
-
-void avro_datum_print(avro_datum_t value, FILE * fp);
-
-int avro_datum_equal(avro_datum_t a, avro_datum_t b);
-
-/*
- * Returns a string containing the JSON encoding of an Avro value.  You
- * must free this string when you're done with it, using the standard
- * free() function.  (*Not* using the custom Avro allocator.)
- */
-
-int avro_datum_to_json(const avro_datum_t datum,
-		       int one_line, char **json_str);
-
-int avro_schema_match(avro_schema_t writers_schema,
-		      avro_schema_t readers_schema);
-
-int avro_schema_datum_validate(avro_schema_t
-			       expected_schema, avro_datum_t datum);
-
-int avro_read_data(avro_reader_t reader,
-		   avro_schema_t writer_schema,
-		   avro_schema_t reader_schema, avro_datum_t * datum);
-int avro_skip_data(avro_reader_t reader, avro_schema_t writer_schema);
-int avro_write_data(avro_writer_t writer,
-		    avro_schema_t writer_schema, avro_datum_t datum);
-int64_t avro_size_data(avro_writer_t writer,
-		       avro_schema_t writer_schema, avro_datum_t datum);
-
-/* File object container */
-typedef struct avro_file_reader_t_ *avro_file_reader_t;
-typedef struct avro_file_writer_t_ *avro_file_writer_t;
-
-int avro_file_writer_create(const char *path, avro_schema_t schema,
-			    avro_file_writer_t * writer);
-int avro_file_writer_open(const char *path, avro_file_writer_t * writer);
-int avro_file_reader(const char *path, avro_file_reader_t * reader);
-
-int avro_file_writer_append(avro_file_writer_t writer, avro_datum_t datum);
-int avro_file_writer_sync(avro_file_writer_t writer);
-int avro_file_writer_flush(avro_file_writer_t writer);
-int avro_file_writer_close(avro_file_writer_t writer);
-
-int avro_file_reader_read(avro_file_reader_t reader,
-			  avro_schema_t readers_schema, avro_datum_t * datum);
-int avro_file_reader_close(avro_file_reader_t reader);
+#include <avro/allocation.h>
+#include <avro/basics.h>
+#include <avro/consumer.h>
+#include <avro/data.h>
+#include <avro/errors.h>
+#include <avro/generic.h>
+#include <avro/io.h>
+#include <avro/legacy.h>
+#include <avro/refcount.h>
+#include <avro/schema.h>
+#include <avro/value.h>
 
 CLOSE_EXTERN
 #endif

Added: avro/trunk/lang/c/src/avro/allocation.h
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/avro/allocation.h?rev=1146546&view=auto
==============================================================================
--- avro/trunk/lang/c/src/avro/allocation.h (added)
+++ avro/trunk/lang/c/src/avro/allocation.h Thu Jul 14 02:35:04 2011
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0 
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License. 
+ */
+
+#ifndef AVRO_ALLOCATION_H
+#define AVRO_ALLOCATION_H
+#include <stdlib.h>
+
+/*
+ * Allocation interface.  You can provide a custom allocator for the
+ * library, should you wish.  The allocator is provided as a single
+ * generic function, which can emulate the standard malloc, realloc, and
+ * free functions.  The design of this allocation interface is inspired
+ * by the implementation of the Lua interpreter.
+ *
+ * The ptr parameter will be the location of any existing memory
+ * buffer.  The osize parameter will be the size of this existing
+ * buffer.  If ptr is NULL, then osize will be 0.  The nsize parameter
+ * will be the size of the new buffer, or 0 if the new buffer should be
+ * freed.
+ *
+ * If nsize is 0, then the allocation function must return NULL.  If
+ * nsize is not 0, then it should return NULL if the allocation fails.
+ */
+
+typedef void *
+(*avro_allocator_t)(void *user_data, void *ptr, size_t osize, size_t nsize);
+
+void avro_set_allocator(avro_allocator_t alloc, void *user_data);
+
+struct avro_allocator_state {
+	avro_allocator_t  alloc;
+	void  *user_data;
+};
+
+extern struct avro_allocator_state  AVRO_CURRENT_ALLOCATOR;
+
+#define avro_realloc(ptr, osz, nsz)          \
+	(AVRO_CURRENT_ALLOCATOR.alloc        \
+	 (AVRO_CURRENT_ALLOCATOR.user_data,  \
+	  (ptr), (osz), (nsz)))
+
+#define avro_malloc(sz) (avro_realloc(NULL, 0, (sz)))
+#define avro_free(ptr, osz) (avro_realloc((ptr), (osz), 0))
+
+#define avro_new(type) (avro_realloc(NULL, 0, sizeof(type)))
+#define avro_freet(type, ptr) (avro_realloc((ptr), sizeof(type), 0))
+
+void *avro_calloc(size_t count, size_t size);
+
+/*
+ * This is probably too clever for our own good, but when we duplicate a
+ * string, we actually store its size in the same allocated memory
+ * buffer.  That lets us free the string later, without having to call
+ * strlen to get its size, and without the containing struct having to
+ * manually store the strings length.
+ *
+ * This means that any string return by avro_strdup MUST be freed using
+ * avro_str_free, and the only thing that can be passed into
+ * avro_str_free is a string created via avro_strdup.
+ */
+
+char *avro_strdup(const char *str);
+void avro_str_free(char *str);
+
+#endif

Added: avro/trunk/lang/c/src/avro/basics.h
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/avro/basics.h?rev=1146546&view=auto
==============================================================================
--- avro/trunk/lang/c/src/avro/basics.h (added)
+++ avro/trunk/lang/c/src/avro/basics.h Thu Jul 14 02:35:04 2011
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+#ifndef AVRO_BASICS_H
+#define AVRO_BASICS_H
+#ifdef __cplusplus
+extern "C" {
+#define CLOSE_EXTERN }
+#else
+#define CLOSE_EXTERN
+#endif
+
+
+enum avro_type_t {
+	AVRO_STRING,
+	AVRO_BYTES,
+	AVRO_INT32,
+	AVRO_INT64,
+	AVRO_FLOAT,
+	AVRO_DOUBLE,
+	AVRO_BOOLEAN,
+	AVRO_NULL,
+	AVRO_RECORD,
+	AVRO_ENUM,
+	AVRO_FIXED,
+	AVRO_MAP,
+	AVRO_ARRAY,
+	AVRO_UNION,
+	AVRO_LINK
+};
+typedef enum avro_type_t avro_type_t;
+
+enum avro_class_t {
+	AVRO_SCHEMA,
+	AVRO_DATUM
+};
+typedef enum avro_class_t avro_class_t;
+
+struct avro_obj_t {
+	avro_type_t type;
+	avro_class_t class_type;
+	volatile int  refcount;
+};
+
+#define avro_classof(obj)     ((obj)->class_type)
+#define is_avro_schema(obj)   (obj && avro_classof(obj) == AVRO_SCHEMA)
+#define is_avro_datum(obj)    (obj && avro_classof(obj) == AVRO_DATUM)
+
+#define avro_typeof(obj)      ((obj)->type)
+#define is_avro_string(obj)   (obj && avro_typeof(obj) == AVRO_STRING)
+#define is_avro_bytes(obj)    (obj && avro_typeof(obj) == AVRO_BYTES)
+#define is_avro_int32(obj)    (obj && avro_typeof(obj) == AVRO_INT32)
+#define is_avro_int64(obj)    (obj && avro_typeof(obj) == AVRO_INT64)
+#define is_avro_float(obj)    (obj && avro_typeof(obj) == AVRO_FLOAT)
+#define is_avro_double(obj)   (obj && avro_typeof(obj) == AVRO_DOUBLE)
+#define is_avro_boolean(obj)  (obj && avro_typeof(obj) == AVRO_BOOLEAN)
+#define is_avro_null(obj)     (obj && avro_typeof(obj) == AVRO_NULL)
+#define is_avro_primitive(obj)(is_avro_string(obj) \
+                             ||is_avro_bytes(obj) \
+                             ||is_avro_int32(obj) \
+                             ||is_avro_int64(obj) \
+                             ||is_avro_float(obj) \
+                             ||is_avro_double(obj) \
+                             ||is_avro_boolean(obj) \
+                             ||is_avro_null(obj))
+#define is_avro_record(obj)   (obj && avro_typeof(obj) == AVRO_RECORD)
+#define is_avro_enum(obj)     (obj && avro_typeof(obj) == AVRO_ENUM)
+#define is_avro_fixed(obj)    (obj && avro_typeof(obj) == AVRO_FIXED)
+#define is_avro_named_type(obj)(is_avro_record(obj) \
+                              ||is_avro_enum(obj) \
+                              ||is_avro_fixed(obj))
+#define is_avro_map(obj)      (obj && avro_typeof(obj) == AVRO_MAP)
+#define is_avro_array(obj)    (obj && avro_typeof(obj) == AVRO_ARRAY)
+#define is_avro_union(obj)    (obj && avro_typeof(obj) == AVRO_UNION)
+#define is_avro_complex_type(obj) (!(is_avro_primitive(obj))
+#define is_avro_link(obj)     (obj && avro_typeof(obj) == AVRO_LINK)
+
+
+
+CLOSE_EXTERN
+#endif

Modified: avro/trunk/lang/c/src/avro/consumer.h
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/avro/consumer.h?rev=1146546&r1=1146545&r2=1146546&view=diff
==============================================================================
--- avro/trunk/lang/c/src/avro/consumer.h (original)
+++ avro/trunk/lang/c/src/avro/consumer.h Thu Jul 14 02:35:04 2011
@@ -24,7 +24,11 @@ extern "C" {
 #define CLOSE_EXTERN
 #endif
 
-#include <avro.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#include <avro/io.h>
+#include <avro/schema.h>
 
 
 /*---------------------------------------------------------------------

Added: avro/trunk/lang/c/src/avro/data.h
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/avro/data.h?rev=1146546&view=auto
==============================================================================
--- avro/trunk/lang/c/src/avro/data.h (added)
+++ avro/trunk/lang/c/src/avro/data.h Thu Jul 14 02:35:04 2011
@@ -0,0 +1,503 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+#ifndef AVRO_DATA_H
+#define AVRO_DATA_H
+#ifdef __cplusplus
+extern "C" {
+#define CLOSE_EXTERN }
+#else
+#define CLOSE_EXTERN
+#endif
+
+#include <stdbool.h>
+#include <stdlib.h>
+#include <string.h>
+
+/*
+ * This file defines some helper data structures that are used within
+ * Avro, and in the schema-specific types created by avrocc.
+ */
+
+
+/*---------------------------------------------------------------------
+ * Arrays
+ */
+
+/**
+ * A resizeable array of fixed-size elements.
+ */
+
+typedef struct avro_raw_array {
+	size_t  element_size;
+	size_t  element_count;
+	size_t  allocated_size;
+	void  *data;
+} avro_raw_array_t;
+
+/**
+ * Initializes a new avro_raw_array_t that you've allocated yourself.
+ */
+
+void avro_raw_array_init(avro_raw_array_t *array, size_t element_size);
+
+/**
+ * Finalizes an avro_raw_array_t.
+ */
+
+void avro_raw_array_done(avro_raw_array_t *array);
+
+/**
+ * Clears an avro_raw_array_t.  This does not deallocate any space; this
+ * allows us to reuse the underlying array buffer as we start to re-add
+ * elements to the array.
+ */
+
+void avro_raw_array_clear(avro_raw_array_t *array);
+
+/**
+ * Ensures that there is enough allocated space to store the given
+ * number of elements in an avro_raw_array_t.  If we can't allocate that
+ * much space, we return ENOMEM.
+ */
+
+int
+avro_raw_array_ensure_size(avro_raw_array_t *array, size_t desired_count);
+
+/**
+ * Returns the number of elements in an avro_raw_array_t.
+ */
+
+#define avro_raw_array_size(array) ((array)->element_count)
+
+/**
+ * Returns the given element of an avro_raw_array_t as a <code>void
+ * *</code>.
+ */
+
+#define avro_raw_array_get_raw(array, index) \
+	((array)->data + (array)->element_size * index)
+
+/**
+ * Returns the given element of an avro_raw_array_t, using element_type
+ * as the type of the elements.  The result is *not* a pointer to the
+ * element; you get the element itself.
+ */
+
+#define avro_raw_array_get(array, element_type, index) \
+	(((element_type *) (array)->data)[index])
+
+/**
+ * Appends a new element to an avro_raw_array_t, expanding it if
+ * necessary.  Returns a pointer to the new element, or NULL if we
+ * needed to reallocate the array and couldn't.
+ */
+
+void *avro_raw_array_append(avro_raw_array_t *array);
+
+
+/*---------------------------------------------------------------------
+ * Maps
+ */
+
+/**
+ * The type of the elements in a map's elements array.
+ */
+
+typedef struct avro_raw_map_entry {
+	const char  *key;
+} avro_raw_map_entry_t;
+
+/**
+ * A string-indexed map of fixed-size elements.
+ */
+
+typedef struct avro_raw_map {
+	avro_raw_array_t  elements;
+	void  *indices_by_key;
+} avro_raw_map_t;
+
+/**
+ * Initializes a new avro_raw_map_t that you've allocated yourself.
+ */
+
+void avro_raw_map_init(avro_raw_map_t *map, size_t element_size);
+
+/**
+ * Finalizes an avro_raw_map_t.
+ */
+
+void avro_raw_map_done(avro_raw_map_t *map);
+
+/**
+ * Clears an avro_raw_map_t.
+ */
+
+void avro_raw_map_clear(avro_raw_map_t *map);
+
+/**
+ * Ensures that there is enough allocated space to store the given
+ * number of elements in an avro_raw_map_t.  If we can't allocate that
+ * much space, we return ENOMEM.
+ */
+
+int
+avro_raw_map_ensure_size(avro_raw_map_t *map, size_t desired_count);
+
+/**
+ * Returns the number of elements in an avro_raw_map_t.
+ */
+
+#define avro_raw_map_size(map)  avro_raw_array_size(&((map)->elements))
+
+/**
+ * Returns the avro_raw_map_entry_t for a given index.
+ */
+
+#define avro_raw_get_entry(map, index) \
+	((avro_raw_map_entry_t *) \
+	 avro_raw_array_get_raw(&(map)->elements, index))
+
+/**
+ * Returns the given element of an avro_raw_array_t as a <code>void
+ * *</code>.  The indexes are assigned based on the order that the
+ * elements are added to the map.
+ */
+
+#define avro_raw_map_get_raw(map, index) \
+	(avro_raw_array_get_raw(&(map)->elements, index) + \
+	 sizeof(avro_raw_map_entry_t))
+
+/**
+ * Returns the element of an avro_raw_map_t with the given numeric
+ * index.  The indexes are assigned based on the order that the elements
+ * are added to the map.
+ */
+
+#define avro_raw_map_get_by_index(map, element_type, index) \
+	(*((element_type *) avro_raw_map_get_raw(map, index)))
+
+/**
+ * Returns the key of the element with the given numeric index.
+ */
+
+#define avro_raw_map_get_key(map, index) \
+	(avro_raw_get_entry(map, index)->key)
+
+/**
+ * Returns the element of an avro_raw_map_t with the given string key.
+ * If the given element doesn't exist, returns NULL.  If @ref index
+ * isn't NULL, it will be filled in with the index of the element.
+ */
+
+void *avro_raw_map_get(const avro_raw_map_t *map, const char *key,
+		       size_t *index);
+
+/**
+ * Retrieves the element of an avro_raw_map_t with the given string key,
+ * creating it if necessary.  A pointer to the element is placed into
+ * @ref element.  If @ref index isn't NULL, it will be filled in with
+ * the index of the element.  We return 1 if the element is new; 0 if
+ * it's not, and a negative error code if there was some problem.
+ */
+
+int avro_raw_map_get_or_create(avro_raw_map_t *map, const char *key,
+			       void **element, size_t *index);
+
+
+/*---------------------------------------------------------------------
+ * Wrapped buffers
+ */
+
+/**
+ * A pointer to an unmodifiable external memory region, along with
+ * functions for freeing that buffer when it's no longer needed, and
+ * copying it.
+ */
+
+typedef struct avro_wrapped_buffer  avro_wrapped_buffer_t;
+
+struct avro_wrapped_buffer {
+	/** A pointer to the memory region */
+	const void  *buf;
+
+	/** The size of the memory region */
+	size_t  size;
+
+	/** Additional data needed by the methods below */
+	void  *user_data;
+
+	/**
+	 * A function that will be called when the memory region is no
+	 * longer needed.  This pointer can be NULL if nothing special
+	 * needs to be done to free the buffer.
+	 */
+	void
+	(*free)(avro_wrapped_buffer_t *self);
+
+	/**
+	 * A function that makes a copy of a portion of a wrapped
+	 * buffer.  This doesn't have to involve duplicating the memory
+	 * region, but it should ensure that the free method can be
+	 * safely called on both copies without producing any errors or
+	 * memory corruption.  If this function is NULL, then we'll use
+	 * a default implementation that calls @ref
+	 * avro_wrapped_buffer_new_copy.
+	 */
+	int
+	(*copy)(avro_wrapped_buffer_t *dest, const avro_wrapped_buffer_t *src,
+		size_t offset, size_t length);
+
+	/**
+	 * A function that "slices" a wrapped buffer, causing it to
+	 * point at a subset of the existing buffer.  Usually, this just
+	 * requires * updating the @ref buf and @ref size fields.  If
+	 * you don't need to do anything other than this, this function
+	 * pointer can be left @c NULL.  The function can assume that
+	 * the @a offset and @a length parameters point to a valid
+	 * subset of the existing wrapped buffer.
+	 */
+	int
+	(*slice)(avro_wrapped_buffer_t *self, size_t offset, size_t length);
+};
+
+/**
+ * Free a wrapped buffer.
+ */
+
+#define avro_wrapped_buffer_free(self) \
+	do { \
+		if ((self)->free != NULL) { \
+			(self)->free((self)); \
+		} \
+	} while (0)
+
+/**
+ * A static initializer for an empty wrapped buffer.
+ */
+
+#define AVRO_WRAPPED_BUFFER_EMPTY  { NULL, 0, NULL, NULL, NULL, NULL }
+
+/**
+ * Moves a wrapped buffer.  After returning, @a dest will wrap the
+ * buffer that @a src used to point at, and @a src will be empty.
+ */
+
+void
+avro_wrapped_buffer_move(avro_wrapped_buffer_t *dest,
+			 avro_wrapped_buffer_t *src);
+
+/**
+ * Copies a buffer.
+ */
+
+int
+avro_wrapped_buffer_copy(avro_wrapped_buffer_t *dest,
+			 const avro_wrapped_buffer_t *src,
+			 size_t offset, size_t length);
+
+/**
+ * Slices a buffer.
+ */
+
+int
+avro_wrapped_buffer_slice(avro_wrapped_buffer_t *self,
+			  size_t offset, size_t length);
+
+/**
+ * Creates a new wrapped buffer wrapping the given memory region.  The
+ * wrapped buffer's copy method will create an actual copy.
+ */
+
+int
+avro_wrapped_buffer_new(avro_wrapped_buffer_t *dest,
+			const void *buf, size_t length);
+
+/**
+ * Creates a new wrapped buffer wrapping the given C string.
+ */
+
+#define avro_wrapped_buffer_new_string(dest, str) \
+    (avro_wrapped_buffer_new((dest), (str), strlen((str))+1))
+
+/**
+ * Creates a new wrapped buffer containing a copy of the given memory
+ * region.  The wrapped buffer's copy method will create further copies.
+ */
+
+int
+avro_wrapped_buffer_new_copy(avro_wrapped_buffer_t *dest,
+			     const void *buf, size_t length);
+
+/**
+ * Creates a new wrapped buffer containing a copy of the given C string.
+ */
+
+#define avro_wrapped_buffer_new_string_copy(dest, str) \
+    (avro_wrapped_buffer_new_copy((dest), (str), strlen((str))+1))
+
+
+/*---------------------------------------------------------------------
+ * Strings
+ */
+
+/**
+ * A resizable buffer for storing strings and bytes values.
+ */
+
+typedef struct avro_raw_string {
+	avro_wrapped_buffer_t  wrapped;
+} avro_raw_string_t;
+
+/**
+ * Initializes an avro_raw_string_t that you've allocated yourself.
+ */
+
+void avro_raw_string_init(avro_raw_string_t *str);
+
+/**
+ * Finalizes an avro_raw_string_t.
+ */
+
+void avro_raw_string_done(avro_raw_string_t *str);
+
+/**
+ * Returns the length of the data stored in an avro_raw_string_t.  If
+ * the buffer contains a C string, this length includes the NUL
+ * terminator.
+ */
+
+#define avro_raw_string_length(str)  ((str)->wrapped.size)
+
+/**
+ * Returns a pointer to the data stored in an avro_raw_string_t.
+ */
+
+#define avro_raw_string_get(str)  ((str)->wrapped.buf)
+
+/**
+ * Fills an avro_raw_string_t with a copy of the given buffer.
+ */
+
+void avro_raw_string_set_length(avro_raw_string_t *str,
+				const void *src,
+				size_t length);
+
+/**
+ * Fills an avro_raw_string_t with a copy of the given C string.
+ */
+
+void avro_raw_string_set(avro_raw_string_t *str, const char *src);
+
+/**
+ * Appends the given C string to an avro_raw_string_t.
+ */
+
+void avro_raw_string_append(avro_raw_string_t *str, const char *src);
+
+/**
+ * Gives control of a buffer to an avro_raw_string_t.
+ */
+
+void
+avro_raw_string_give(avro_raw_string_t *str,
+		     avro_wrapped_buffer_t *src);
+
+/**
+ * Returns an avro_wrapped_buffer_t for the content of the string,
+ * ideally without copying it.
+ */
+
+int
+avro_raw_string_grab(const avro_raw_string_t *str,
+		     avro_wrapped_buffer_t *dest);
+
+/**
+ * Clears an avro_raw_string_t.
+ */
+
+void avro_raw_string_clear(avro_raw_string_t *str);
+
+
+/**
+ * Tests two avro_raw_string_t instances for equality.
+ */
+
+int avro_raw_string_equals(const avro_raw_string_t *str1,
+			   const avro_raw_string_t *str2);
+
+
+/*---------------------------------------------------------------------
+ * Memoization
+ */
+
+/**
+ * A specialized map that can be used to memoize the results of a
+ * function.  The API allows you to use two keys as the memoization
+ * keys; if you only need one key, just use NULL for the second key.
+ * The result of the function should be a single pointer, or an integer
+ * that can be cast into a pointer (i.e., an intptr_t).
+ */
+
+typedef struct avro_memoize {
+	void  *cache;
+} avro_memoize_t;
+
+/**
+ * Initialize an avro_memoize_t that you've allocated for yourself.
+ */
+
+void
+avro_memoize_init(avro_memoize_t *mem);
+
+/**
+ * Finalizes an avro_memoize_t.
+ */
+
+void
+avro_memoize_done(avro_memoize_t *mem);
+
+/**
+ * Search for a cached value in an avro_memoize_t.  Returns a boolean
+ * indicating whether there's a value in the cache for the given keys.
+ * If there is, the cached result is placed into @ref result.
+ */
+
+int
+avro_memoize_get(avro_memoize_t *mem,
+		 void *key1, void *key2,
+		 void **result);
+
+/**
+ * Stores a new cached value into an avro_memoize_t, overwriting it if
+ * necessary.
+ */
+
+void
+avro_memoize_set(avro_memoize_t *mem,
+		 void *key1, void *key2,
+		 void *result);
+
+/**
+ * Removes any cached value for the given key from an avro_memoize_t.
+ */
+
+void
+avro_memoize_delete(avro_memoize_t *mem, void *key1, void *key2);
+
+CLOSE_EXTERN
+#endif

Added: avro/trunk/lang/c/src/avro/errors.h
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/avro/errors.h?rev=1146546&view=auto
==============================================================================
--- avro/trunk/lang/c/src/avro/errors.h (added)
+++ avro/trunk/lang/c/src/avro/errors.h Thu Jul 14 02:35:04 2011
@@ -0,0 +1,34 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+#ifndef AVRO_ERRORS_H
+#define AVRO_ERRORS_H
+
+/*
+ * Returns a textual description of the last error condition returned by
+ * an Avro function.
+ */
+
+const char *avro_strerror(void);
+
+void
+avro_set_error(const char *fmt, ...);
+
+void
+avro_prefix_error(const char *fmt, ...);
+
+#endif

Added: avro/trunk/lang/c/src/avro/generic.h
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/avro/generic.h?rev=1146546&view=auto
==============================================================================
--- avro/trunk/lang/c/src/avro/generic.h (added)
+++ avro/trunk/lang/c/src/avro/generic.h Thu Jul 14 02:35:04 2011
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+#ifndef AVRO_GENERIC_H
+#define AVRO_GENERIC_H
+#ifdef __cplusplus
+extern "C" {
+#define CLOSE_EXTERN }
+#else
+#define CLOSE_EXTERN
+#endif
+
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#include <avro/schema.h>
+#include <avro/value.h>
+
+/*
+ * This file contains an avro_value_t implementation that can store
+ * values of any Avro schema.  It replaces the old avro_datum_t class.
+ */
+
+
+/*
+ * Return a generic avro_value_iface_t implementation for the given
+ * schema, regardless of what type it is.
+ */
+
+avro_value_iface_t *avro_generic_class_from_schema(avro_schema_t schema);
+
+
+/*
+ * These functions return an avro_value_iface_t implementation for each
+ * scalar schema type.
+ */
+
+avro_value_iface_t *avro_generic_boolean_class(void);
+avro_value_iface_t *avro_generic_bytes_class(void);
+avro_value_iface_t *avro_generic_double_class(void);
+avro_value_iface_t *avro_generic_float_class(void);
+avro_value_iface_t *avro_generic_int_class(void);
+avro_value_iface_t *avro_generic_long_class(void);
+avro_value_iface_t *avro_generic_null_class(void);
+avro_value_iface_t *avro_generic_string_class(void);
+
+avro_value_iface_t *avro_generic_enum_class(avro_schema_t schema);
+avro_value_iface_t *avro_generic_fixed_class(avro_schema_t schema);
+
+/*
+ * These functions instantiate a new generic scalar value.
+ */
+
+int avro_generic_boolean_new(avro_value_t *value, bool val);
+int avro_generic_bytes_new(avro_value_t *value, void *buf, size_t size);
+int avro_generic_double_new(avro_value_t *value, double val);
+int avro_generic_float_new(avro_value_t *value, float val);
+int avro_generic_int_new(avro_value_t *value, int32_t val);
+int avro_generic_long_new(avro_value_t *value, int64_t val);
+int avro_generic_null_new(avro_value_t *value);
+int avro_generic_string_new(avro_value_t *value, char *val);
+int avro_generic_string_new_length(avro_value_t *value, char *val, size_t size);
+
+
+/*
+ * These functions return an avro_value_iface_t implementation for each
+ * compound schema type.
+ */
+
+/*
+ * The generic classes for compound schemas work with any avro_value_t
+ * implementation for their subschemas.  They'll use the given function
+ * pointer to instantiate a value implementation for each child schema.
+ */
+
+typedef avro_value_iface_t *
+(*avro_value_iface_creator_t)(avro_schema_t schema, void *user_data);
+
+avro_value_iface_t *
+avro_generic_array_class(avro_schema_t schema,
+			 avro_value_iface_creator_t creator,
+			 void *user_data);
+
+avro_value_iface_t *
+avro_generic_map_class(avro_schema_t schema,
+		       avro_value_iface_creator_t creator,
+		       void *user_data);
+
+avro_value_iface_t *
+avro_generic_record_class(avro_schema_t schema,
+			  avro_value_iface_creator_t creator,
+			  void *user_data);
+
+avro_value_iface_t *
+avro_generic_union_class(avro_schema_t schema,
+			 avro_value_iface_creator_t creator,
+			 void *user_data);
+
+
+CLOSE_EXTERN
+#endif

Added: avro/trunk/lang/c/src/avro/io.h
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/avro/io.h?rev=1146546&view=auto
==============================================================================
--- avro/trunk/lang/c/src/avro/io.h (added)
+++ avro/trunk/lang/c/src/avro/io.h Thu Jul 14 02:35:04 2011
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+#ifndef AVRO_IO_H
+#define AVRO_IO_H
+#ifdef __cplusplus
+extern "C" {
+#define CLOSE_EXTERN }
+#else
+#define CLOSE_EXTERN
+#endif
+
+#include <stdint.h>
+#include <stdio.h>
+
+#include <avro/basics.h>
+#include <avro/legacy.h>
+#include <avro/schema.h>
+#include <avro/value.h>
+
+typedef struct avro_reader_t_ *avro_reader_t;
+typedef struct avro_writer_t_ *avro_writer_t;
+
+/*
+ * io
+ */
+
+avro_reader_t avro_reader_file(FILE * fp);
+avro_writer_t avro_writer_file(FILE * fp);
+avro_reader_t avro_reader_memory(const char *buf, int64_t len);
+avro_writer_t avro_writer_memory(const char *buf, int64_t len);
+
+int avro_read(avro_reader_t reader, void *buf, int64_t len);
+int avro_skip(avro_reader_t reader, int64_t len);
+int avro_write(avro_writer_t writer, void *buf, int64_t len);
+
+void avro_writer_reset(avro_writer_t writer);
+int64_t avro_writer_tell(avro_writer_t writer);
+void avro_writer_flush(avro_writer_t writer);
+
+void avro_writer_dump(avro_writer_t writer, FILE * fp);
+void avro_reader_dump(avro_reader_t reader, FILE * fp);
+
+void avro_reader_free(avro_reader_t reader);
+void avro_writer_free(avro_writer_t writer);
+
+int avro_schema_to_json(const avro_schema_t schema, avro_writer_t out);
+
+/*
+ * Reads a binary-encoded Avro value from the given reader object,
+ * storing the result into dest.
+ */
+
+int
+avro_value_read(avro_reader_t reader, avro_value_t *dest);
+
+/*
+ * Writes a binary-encoded Avro value to the given writer object.
+ */
+
+int
+avro_value_write(avro_writer_t writer, avro_value_t *src);
+
+/*
+ * Returns the size of the binary encoding of the given Avro value.
+ */
+
+int
+avro_value_sizeof(avro_value_t *src, size_t *size);
+
+
+/* File object container */
+typedef struct avro_file_reader_t_ *avro_file_reader_t;
+typedef struct avro_file_writer_t_ *avro_file_writer_t;
+
+int avro_file_writer_create(const char *path, avro_schema_t schema,
+			    avro_file_writer_t * writer);
+int avro_file_writer_open(const char *path, avro_file_writer_t * writer);
+int avro_file_reader(const char *path, avro_file_reader_t * reader);
+
+int avro_file_writer_sync(avro_file_writer_t writer);
+int avro_file_writer_flush(avro_file_writer_t writer);
+int avro_file_writer_close(avro_file_writer_t writer);
+
+int avro_file_reader_close(avro_file_reader_t reader);
+
+/*
+ * Legacy avro_datum_t API
+ */
+
+int avro_read_data(avro_reader_t reader,
+		   avro_schema_t writer_schema,
+		   avro_schema_t reader_schema, avro_datum_t * datum);
+int avro_skip_data(avro_reader_t reader, avro_schema_t writer_schema);
+int avro_write_data(avro_writer_t writer,
+		    avro_schema_t writer_schema, avro_datum_t datum);
+int64_t avro_size_data(avro_writer_t writer,
+		       avro_schema_t writer_schema, avro_datum_t datum);
+
+int avro_file_writer_append(avro_file_writer_t writer, avro_datum_t datum);
+
+int avro_file_reader_read(avro_file_reader_t reader,
+			  avro_schema_t readers_schema, avro_datum_t * datum);
+
+CLOSE_EXTERN
+#endif

Added: avro/trunk/lang/c/src/avro/legacy.h
URL: http://svn.apache.org/viewvc/avro/trunk/lang/c/src/avro/legacy.h?rev=1146546&view=auto
==============================================================================
--- avro/trunk/lang/c/src/avro/legacy.h (added)
+++ avro/trunk/lang/c/src/avro/legacy.h Thu Jul 14 02:35:04 2011
@@ -0,0 +1,264 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+#ifndef AVRO_LEGACY_H
+#define AVRO_LEGACY_H
+#ifdef __cplusplus
+extern "C" {
+#define CLOSE_EXTERN }
+#else
+#define CLOSE_EXTERN
+#endif
+
+#include <stdint.h>
+#include <stdio.h>
+
+#include <avro/basics.h>
+#include <avro/data.h>
+#include <avro/schema.h>
+#include <avro/value.h>
+
+/*
+ * This file defines the deprecated interface for handling Avro values.
+ * It's here solely for backwards compatibility.  New code should use
+ * the avro_value_t interface (defined in avro/value.h).  The
+ * avro_datum_t type has been replaced by the “generic” implementation
+ * of the value interface, which is defined in avro/generic.h.  You can
+ * also use your own application-specific types as Avro values by
+ * defining your own avro_value_t implementation for them.
+ */
+
+/**
+ * A function used to free a bytes, string, or fixed buffer once it is
+ * no longer needed by the datum that wraps it.
+ */
+
+typedef void
+(*avro_free_func_t)(void *ptr, size_t sz);
+
+/**
+ * An avro_free_func_t that frees the buffer using the custom allocator
+ * provided to avro_set_allocator.
+ */
+
+void
+avro_alloc_free_func(void *ptr, size_t sz);
+
+/*
+ * Datum constructors.  Each datum stores a reference to the schema that
+ * the datum is an instance of.  The primitive datum constructors don't
+ * need to take in an explicit avro_schema_t parameter, since there's
+ * only one schema that they could be an instance of.  The complex
+ * constructors do need an explicit schema parameter.
+ */
+
+typedef struct avro_obj_t *avro_datum_t;
+avro_datum_t avro_string(const char *str);
+avro_datum_t avro_givestring(const char *str,
+			     avro_free_func_t free);
+avro_datum_t avro_bytes(const char *buf, int64_t len);
+avro_datum_t avro_givebytes(const char *buf, int64_t len,
+			    avro_free_func_t free);
+avro_datum_t avro_int32(int32_t i);
+avro_datum_t avro_int64(int64_t l);
+avro_datum_t avro_float(float f);
+avro_datum_t avro_double(double d);
+avro_datum_t avro_boolean(int8_t i);
+avro_datum_t avro_null(void);
+avro_datum_t avro_record(avro_schema_t schema);
+avro_datum_t avro_enum(avro_schema_t schema, int i);
+avro_datum_t avro_fixed(avro_schema_t schema,
+			const char *bytes, const int64_t size);
+avro_datum_t avro_givefixed(avro_schema_t schema,
+			    const char *bytes, const int64_t size,
+			    avro_free_func_t free);
+avro_datum_t avro_map(avro_schema_t schema);
+avro_datum_t avro_array(avro_schema_t schema);
+avro_datum_t avro_union(avro_schema_t schema,
+			int64_t discriminant, const avro_datum_t datum);
+
+/**
+ * Returns the schema that the datum is an instance of.
+ */
+
+avro_schema_t avro_datum_get_schema(const avro_datum_t datum);
+
+/*
+ * Constructs a new avro_datum_t instance that's appropriate for holding
+ * values of the given schema.
+ */
+
+avro_datum_t avro_datum_from_schema(const avro_schema_t schema);
+
+/* getters */
+int avro_string_get(avro_datum_t datum, char **p);
+int avro_bytes_get(avro_datum_t datum, char **bytes, int64_t * size);
+int avro_int32_get(avro_datum_t datum, int32_t * i);
+int avro_int64_get(avro_datum_t datum, int64_t * l);
+int avro_float_get(avro_datum_t datum, float *f);
+int avro_double_get(avro_datum_t datum, double *d);
+int avro_boolean_get(avro_datum_t datum, int8_t * i);
+
+int avro_enum_get(const avro_datum_t datum);
+const char *avro_enum_get_name(const avro_datum_t datum);
+int avro_fixed_get(avro_datum_t datum, char **bytes, int64_t * size);
+int avro_record_get(const avro_datum_t record, const char *field_name,
+		    avro_datum_t * value);
+
+/*
+ * A helper macro that extracts the value of the given field of a
+ * record.
+ */
+
+#define avro_record_get_field_value(rc, rec, typ, fname, ...)	\
+	do {							\
+		avro_datum_t  field = NULL;			\
+		(rc) = avro_record_get((rec), (fname), &field);	\
+		if (rc) break;					\
+		(rc) = avro_##typ##_get(field, __VA_ARGS__);	\
+	} while (0)
+
+
+int avro_map_get(const avro_datum_t datum, const char *key,
+		 avro_datum_t * value);
+/*
+ * For maps, the "index" for each entry is based on the order that they
+ * were added to the map.
+ */
+int avro_map_get_key(const avro_datum_t datum, int index,
+		     const char **key);
+int avro_map_get_index(const avro_datum_t datum, const char *key,
+		       int *index);
+size_t avro_map_size(const avro_datum_t datum);
+int avro_array_get(const avro_datum_t datum, int64_t index, avro_datum_t * value);
+size_t avro_array_size(const avro_datum_t datum);
+
+/*
+ * These accessors allow you to query the current branch of a union
+ * value, returning either the branch's discriminant value or the
+ * avro_datum_t of the branch.  A union value can be uninitialized, in
+ * which case the discriminant will be -1 and the datum NULL.
+ */
+
+int64_t avro_union_discriminant(const avro_datum_t datum);
+avro_datum_t avro_union_current_branch(avro_datum_t datum);
+
+/* setters */
+int avro_string_set(avro_datum_t datum, const char *p);
+int avro_givestring_set(avro_datum_t datum, const char *p,
+			avro_free_func_t free);
+
+int avro_bytes_set(avro_datum_t datum, const char *bytes, const int64_t size);
+int avro_givebytes_set(avro_datum_t datum, const char *bytes,
+		       const int64_t size,
+		       avro_free_func_t free);
+
+int avro_int32_set(avro_datum_t datum, const int32_t i);
+int avro_int64_set(avro_datum_t datum, const int64_t l);
+int avro_float_set(avro_datum_t datum, const float f);
+int avro_double_set(avro_datum_t datum, const double d);
+int avro_boolean_set(avro_datum_t datum, const int8_t i);
+
+int avro_enum_set(avro_datum_t datum, const int symbol_value);
+int avro_enum_set_name(avro_datum_t datum, const char *symbol_name);
+int avro_fixed_set(avro_datum_t datum, const char *bytes, const int64_t size);
+int avro_givefixed_set(avro_datum_t datum, const char *bytes,
+		       const int64_t size,
+		       avro_free_func_t free);
+
+int avro_record_set(avro_datum_t record, const char *field_name,
+		    avro_datum_t value);
+
+/*
+ * A helper macro that sets the value of the given field of a record.
+ */
+
+#define avro_record_set_field_value(rc, rec, typ, fname, ...)	\
+	do {							\
+		avro_datum_t  field = NULL;			\
+		(rc) = avro_record_get((rec), (fname), &field);	\
+		if (rc) break;					\
+		(rc) = avro_##typ##_set(field, __VA_ARGS__);	\
+	} while (0)
+
+int avro_map_set(avro_datum_t map, const char *key,
+		 avro_datum_t value);
+int avro_array_append_datum(avro_datum_t array_datum,
+			    avro_datum_t datum);
+
+/*
+ * This function selects the active branch of a union value, and can be
+ * safely called on an existing union to change the current branch.  If
+ * the branch changes, we'll automatically construct a new avro_datum_t
+ * for the new branch's schema type.  If the desired branch is already
+ * the active branch of the union, we'll leave the existing datum
+ * instance as-is.  The branch datum will be placed into the "branch"
+ * parameter, regardless of whether we have to create a new datum
+ * instance or not.
+ */
+
+int avro_union_set_discriminant(avro_datum_t unionp,
+				int discriminant,
+				avro_datum_t *branch);
+
+/**
+ * Resets a datum instance.  For arrays and maps, this frees all
+ * elements and clears the container.  For records and unions, this
+ * recursively resets any child datum instances.
+ */
+
+int
+avro_datum_reset(avro_datum_t value);
+
+/* reference counting */
+avro_datum_t avro_datum_incref(avro_datum_t value);
+void avro_datum_decref(avro_datum_t value);
+
+void avro_datum_print(avro_datum_t value, FILE * fp);
+
+int avro_datum_equal(avro_datum_t a, avro_datum_t b);
+
+/*
+ * Returns a string containing the JSON encoding of an Avro value.  You
+ * must free this string when you're done with it, using the standard
+ * free() function.  (*Not* using the custom Avro allocator.)
+ */
+
+int avro_datum_to_json(const avro_datum_t datum,
+		       int one_line, char **json_str);
+
+
+int avro_schema_datum_validate(avro_schema_t
+			       expected_schema, avro_datum_t datum);
+
+/*
+ * An avro_value_t implementation for avro_datum_t objects.
+ */
+
+avro_value_iface_t *
+avro_datum_class(void);
+
+/*
+ * Creates a new avro_value_t instance for the given datum.
+ */
+
+int
+avro_datum_as_value(avro_value_t *value, avro_datum_t src);
+
+
+CLOSE_EXTERN
+#endif



Mime
View raw message