arrow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject arrow git commit: ARROW-844: [Format] Update README documents in format/
Date Tue, 18 Apr 2017 15:43:18 GMT
Repository: arrow
Updated Branches:
  refs/heads/master 7f20f6e73 -> 38efabea9

ARROW-844: [Format] Update README documents in format/

Added a section reflecting specification maturity and stability.

Author: Wes McKinney <>

Closes #556 from wesm/ARROW-844 and squashes the following commits:

03dbb71 [Wes McKinney] Update README documents in format/


Branch: refs/heads/master
Commit: 38efabea9bbc8d6386f96a635a95c53ba70e6149
Parents: 7f20f6e
Author: Wes McKinney <>
Authored: Tue Apr 18 11:43:13 2017 -0400
Committer: Wes McKinney <>
Committed: Tue Apr 18 11:43:13 2017 -0400

 format/ | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/format/ b/format/
index 048badb..3aa8fdd 100644
--- a/format/
+++ b/format/
@@ -14,16 +14,14 @@
 ## Arrow specification documents
-> **Work-in-progress specification documents**. These are discussion documents
-> created by the Arrow developers during late 2015 and in no way represents a
-> finalized specification.
 Currently, the Arrow specification consists of these pieces:
 - Metadata specification (see
 - Physical memory layout specification (see
-- Metadata serialized representation (see Message.fbs)
+- Logical Types, Schemas, and Record Batch Metadata (see Schema.fbs)
+- Encapsulated Messages (see Message.fbs)
 - Mechanics of messaging between Arrow systems (IPC, RPC, etc.) (see
+- Tensor (Multi-dimensional array) Metadata (see Tensor.fbs)
 The metadata currently uses Google's [flatbuffers library][1] for serializing a
 couple related pieces of information:
@@ -35,4 +33,16 @@ couple related pieces of information:
    schema, and enable a system to send and receive Arrow row batches in a form
    that can be precisely disassembled or reconstructed.
+## Arrow Format Maturity and Stability
+We have made significant progress hardening the Arrow in-memory format and
+Flatbuffer metadata since the project started in February 2016. We have
+integration tests which verify binary compatibility between the Java and C++
+implementations, for example.
+Major versions may still include breaking changes to the memory format or
+metadata, so it is recommended to use the same released version of all
+libraries in your applications for maximum compatibility. Data stored in the
+Arrow IPC formats should not be used for long term storage.

View raw message