spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-26001) Reduce memory copy when writing decimal
Date Sat, 10 Nov 2018 06:38:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-26001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-26001:
------------------------------------

    Assignee: Apache Spark

> Reduce memory copy when writing decimal
> ---------------------------------------
>
>                 Key: SPARK-26001
>                 URL: https://issues.apache.org/jira/browse/SPARK-26001
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.5.0
>            Reporter: caoxuewen
>            Assignee: Apache Spark
>            Priority: Major
>
> this PR fix 2 here:
> - when writing non-null decimals, we not zero-out all the 16 allocated bytes. if the
number of bytes needed for a decimal is greater than 8. then we not need zero-out between
0-byte and 8-byte. The first 8-byte will be covered when writing decimal.
> - when writing null decimals, we not zero-out all the 16 allocated bytes. BitSetMethods.set
the label for null and the length of decimal to 0. when we get the decimal, will not access
the 16 byte memory value, so this is safe.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message