arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@apache.org>
Subject Say no to zero length batches...
Date Fri, 14 Apr 2017 21:55:33 GMT
Hey All,

I had a quick comment on ARROW-783 that Wes responded to and I wanted to
elevate the conversation here for a moment.

My suggestion there was that we should disallow zero-length batches.

Wes thought that should be an application level concern. I wanted to see
what others thought.

My general perspective is that zero-length batches are meaningless and
better to disallow than make every application have special handling for
them. In the jira Wes noted that they have to deal with zero-length
dataframes. Despite that, I don't think there is a requirement that there
should be a 1:1 mapping between arrow record batches and dataframes. If
someone wants to communicate empty things, no need for them to use Arrow.

What do others think?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message