hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10656) Optimize conversion of byte arrays back to path string
Date Tue, 15 Nov 2016 00:19:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665493#comment-15665493
] 

Xiao Chen commented on HDFS-10656:
----------------------------------

Thank you for the nice optimization [~daryn]!

Sorry for posting my late questions here:
- It seems the new check on range doesn't cover the scenario that {{length}} < 0. So {{length
< 0 && offset + length >=0}} would be a valid input. Should we worry about this?
- It seems the old behavior is to always return {{""}} if the byte array is 0-length, without
any input validation on {{offset}}/{{length}}. New behavior will throw {{IndexOutOfBoundsException}}
from the precondition check.
I'm only asking from a code review perspective. And I guess this is okay since {{DFSUtil}}
is private? Not sure why the old behavior is as such.


> Optimize conversion of byte arrays back to path string
> ------------------------------------------------------
>
>                 Key: HDFS-10656
>                 URL: https://issues.apache.org/jira/browse/HDFS-10656
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>             Fix For: 2.8.0, 2.9.0, 2.7.4, 3.0.0-alpha1
>
>         Attachments: HDFS-10656.patch
>
>
> {{DFSUtil.byteArray2PathString}} generates excessive object allocation.
> # each byte array is encoded to a string (copy)
> # string appended to a builder which extracts the chars from the intermediate string
(copy) and adds to its own char array
> # builder's char array is re-alloced if over 16 chars (copy)
> # builder's toString creates another string (copy)
> Instead of allocating all these objects and performing multiple byte/char encoding/decoding
conversions, the byte array can be built in-place with a single final conversion to a string.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message