spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chenzhao Guo (JIRA)" <>
Subject [jira] [Created] (SPARK-21412) Reset BufferHolder while initialize an UnsafeRowWriter
Date Fri, 14 Jul 2017 03:20:00 GMT
Chenzhao Guo created SPARK-21412:

             Summary: Reset BufferHolder while initialize an UnsafeRowWriter
                 Key: SPARK-21412
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.1.1
            Reporter: Chenzhao Guo

UnsafeRowWriter's construtor should contain BufferHolder.reset to make the writer out of the

While writing UnsafeRow using UnsafeRowWriter, developers should manually call BufferHolder.reset
to make the BufferHolder's cursor(which indicates where to write variable length portion)
right in order to write variable length fields like UTF8String, byte[], etc.

If a developer doesn't reuse the BufferHolder so maybe he never noticed reset and the comments
in code, the UnsafeRow won't be correct if he also writes variable length UTF8String. This
API design doesn't make sense. We should reset the BufferHolder to make UnsafeRowWriter out
of the box, but not let user read the code and manually call it.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message