arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rollo Konig-Brock <>
Subject [pyarrow] Unable to write an array if the schema is nullable
Date Wed, 22 Jan 2020 18:10:38 GMT
Hey all,

I am unable to write an array (without any nulls) if the schema
explicitly sets nullable to false.

A quick example:

import io
import pyarrow

schema = pyarrow.schema([
    pyarrow.field('_strangle', 'string', nullable=False)]

batch_array = pyarrow.array(['abc', 'åbc', None])

output = io.BytesIO()

writer = pyarrow.RecordBatchStreamWriter(output, schema)

batch = pyarrow.RecordBatch.from_arrays(
    [batch_array], schema.names

# This fails with pyarrow.lib.ArrowInvalid: Tried to write record batch
with different schema

Any ideas here? It seems quite strange that I can't find an interface for
writing not-nullable arrays.

All the best,

View raw message