Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 338F3200C7E for ; Tue, 23 May 2017 23:22:28 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 322CA160BA4; Tue, 23 May 2017 21:22:28 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 080E6160BC3 for ; Tue, 23 May 2017 23:22:25 +0200 (CEST) Received: (qmail 56405 invoked by uid 500); 23 May 2017 21:22:25 -0000 Mailing-List: contact commits-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@arrow.apache.org Delivered-To: mailing list commits@arrow.apache.org Received: (qmail 56335 invoked by uid 99); 23 May 2017 21:22:25 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 May 2017 21:22:25 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 11840E00AF; Tue, 23 May 2017 21:22:25 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: wesm@apache.org To: commits@arrow.apache.org Date: Tue, 23 May 2017 21:22:27 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [3/4] arrow-site git commit: Update Python documentation archived-at: Tue, 23 May 2017 21:22:28 -0000 http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/data.html ---------------------------------------------------------------------- diff --git a/docs/python/data.html b/docs/python/data.html index e16f145..99ed571 100644 --- a/docs/python/data.html +++ b/docs/python/data.html @@ -241,7 +241,7 @@ the numpy.array f In [24]: arr Out[24]: -<pyarrow._array.Int64Array object at 0x2b0bcdd11318> +<pyarrow.lib.Int64Array object at 0x7fce90483278> [ 1, 2, @@ -306,7 +306,7 @@ consider an example:

In [36]: dict_array Out[36]: -<pyarrow._array.DictionaryArray object at 0x2b0bcdd19588> +<pyarrow.lib.DictionaryArray object at 0x7fce90480768> [ 'foo', 'bar', @@ -325,7 +325,7 @@ consider an example:

In [38]: dict_array.indices Out[38]: -<pyarrow._array.Int64Array object at 0x2b0bcdd29908> +<pyarrow.lib.Int64Array object at 0x7fce904914f8> [ 0, 1, @@ -338,8 +338,8 @@ consider an example:

] In [39]: dict_array.dictionary -Out[39]: -<pyarrow._array.StringArray object at 0x2b0bcdd29818> +Out[39]: +<pyarrow.lib.StringArray object at 0x7fce90491408> [ 'foo', 'bar', @@ -384,11 +384,10 @@ instances. Let’s consider a collection of arrays:

f0: int64 f1: string f2: bool --- metadata -- In [46]: batch[1] -Out[46]: -<pyarrow._array.StringArray object at 0x2b0bcdd36f98> +Out[46]: +<pyarrow.lib.StringArray object at 0x7fce904a6638> [ 'foo', 'bar', @@ -402,7 +401,7 @@ instances. Let’s consider a collection of arrays:

In [48]: batch2[1] Out[48]: -<pyarrow._array.StringArray object at 0x2b0bcdd36ea8> +<pyarrow.lib.StringArray object at 0x7fce904b9138> [ 'bar', 'baz', @@ -431,10 +430,9 @@ one or more copies of the batch using f0: int64 f1: string f2: bool --- metadata -- In [52]: table.num_rows -Out[52]: 20 +Out[52]: 20

The table’s columns are instances of Column, which is a container @@ -442,17 +440,17 @@ for one or more arrays of the same type.

In [53]: c = table[0]
 
 In [54]: c
-Out[54]: <pyarrow._table.Column at 0x2b0bcda54f00>
+Out[54]: <pyarrow.lib.Column at 0x7fce8f9e3e10>
 
 In [55]: c.data
-Out[55]: <pyarrow._table.ChunkedArray at 0x2b0bcda54ed0>
+Out[55]: <pyarrow.lib.ChunkedArray at 0x7fce8f9e3c00>
 
 In [56]: c.data.num_chunks
-Out[56]: 5
+Out[56]: 5
 
 In [57]: c.data.chunk(0)
-Out[57]: 
-<pyarrow._array.Int64Array object at 0x2b0bcdd4b3b8>
+Out[57]: 
+<pyarrow.lib.Int64Array object at 0x7fce904b9818>
 [
   1,
   2,

http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/development.html
----------------------------------------------------------------------
diff --git a/docs/python/development.html b/docs/python/development.html
index bfc4110..2146146 100644
--- a/docs/python/development.html
+++ b/docs/python/development.html
@@ -226,10 +226,27 @@ pyarrow/tests/test_tensor.py ................

Windows

-

First, make sure you can build the C++ library.

-

Now, we need to build and install the C++ libraries someplace.

+

First, we bootstrap a conda environment similar to the C++ build instructions. This +includes all the dependencies for Arrow and the Apache Parquet C++ libraries.

+

First, starting from fresh clones of Apache Arrow and parquet-cpp:

+
git clone https://github.com/apache/arrow.git
+git clone https://github.com/apache/parquet-cpp.git
+
+
+
conda create -n arrow-dev cmake git boost-cpp ^
+      flatbuffers snappy zlib brotli thrift-cpp rapidjson
+activate arrow-dev
+
+
+

As one git housekeeping item, we must run this command in our Arrow clone:

+
cd arrow
+git config core.symlinks true
+
+
+

Now, we build and install Arrow C++ libraries

mkdir cpp\build
 cd cpp\build
+set ARROW_BUILD_TOOLCHAIN=%CONDA_PREFIX%\Library
 set ARROW_HOME=C:\thirdparty
 cmake -G "Visual Studio 14 2015 Win64" ^
       -DCMAKE_INSTALL_PREFIX=%ARROW_HOME% ^
@@ -240,13 +257,31 @@ cmake --build . --target INSTALL --config Release
 cd ..\..
+

Now, we build parquet-cpp and install the result in the same place:

+
mkdir ..\parquet-cpp\build
+pushd ..\parquet-cpp\build
+set PARQUET_BUILD_TOOLCHAIN=%CONDA_PREFIX%\Library
+set PARQUET_HOME=C:\thirdparty
+cmake -G "Visual Studio 14 2015 Win64" ^
+      -DCMAKE_INSTALL_PREFIX=%PARQUET_HOME% ^
+      -DCMAKE_BUILD_TYPE=Release ^
+      -DPARQUET_ZLIB_VENDORED=off ^
+      -DPARQUET_BUILD_TESTS=off ..
+cmake --build . --target INSTALL --config Release
+popd
+
+

After that, we must put the install directory’s bin path in our %PATH%:

set PATH=%ARROW_HOME%\bin;%PATH%

Now, we can build pyarrow:

cd python
-python setup.py build_ext --inplace
+python setup.py build_ext --inplace --with-parquet
+
+
+

Then run the unit tests with:

+
py.test pyarrow -v
http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.BufferOutputStream.html ---------------------------------------------------------------------- diff --git a/docs/python/generated/pyarrow.BufferOutputStream.html b/docs/python/generated/pyarrow.BufferOutputStream.html new file mode 100644 index 0000000..25f5794 --- /dev/null +++ b/docs/python/generated/pyarrow.BufferOutputStream.html @@ -0,0 +1,164 @@ + + + + + + + + pyarrow.BufferOutputStream — pyarrow documentation + + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

pyarrow.BufferOutputStream

+
+
+class pyarrow.BufferOutputStream
+
+
+__init__()
+

Initialize self. See help(type(self)) for accurate signature.

+
+ +

Methods

+ ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
close(self)
download(self, stream_or_path[, buffer_size])Read file completely to local path (rather than reading completely into memory).
get_result(self)
read(self[, nbytes])
read_buffer(self[, nbytes])
seek(self, int64_t position)
size(self)
tell(self)
upload(self, stream[, buffer_size])Pipe file-like object to file
write(self, data)Write byte from any object implementing buffer protocol (bytes,
+
+ +
+ + +
+
+
+
+
+ + + + \ No newline at end of file http://git-wip-us.apache.org/repos/asf/arrow-site/blob/820f0af6/docs/python/generated/pyarrow.BufferReader.html ---------------------------------------------------------------------- diff --git a/docs/python/generated/pyarrow.BufferReader.html b/docs/python/generated/pyarrow.BufferReader.html index 197f273..e2b2615 100644 --- a/docs/python/generated/pyarrow.BufferReader.html +++ b/docs/python/generated/pyarrow.BufferReader.html @@ -27,7 +27,7 @@ - + @@ -38,7 +38,7 @@ index
  • - next |
  • pyarrow.Buffer

    Next topic

    -

    pyarrow.InMemoryOutputStream

    +

    pyarrow.BufferOutputStream

    This Page

      @@ -90,7 +90,7 @@ -Parameters:obj (Python bytes or pyarrow.io.Buffer) – +Parameters:obj (Python bytes or pyarrow.Buffer) – @@ -153,7 +153,7 @@ index
    • - next |
    • - +
    • - previous |
    • pyarrow documentation »
    • API Reference »
      • @@ -50,8 +50,8 @@