hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10694) Remove synchronized input streams from Writable deserialization
Date Fri, 10 Jul 2015 05:16:05 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621740#comment-14621740
] 

Gopal V commented on HADOOP-10694:
----------------------------------

[~ozawa]: I think the synchronization is unintentional, since two threads using DataInputBuffer
readers without synchronization would cause corruption (something like readInt() would result
in interleaved corruption, because each byte read() is independently synchronized).

> Remove synchronized input streams from Writable deserialization
> ---------------------------------------------------------------
>
>                 Key: HADOOP-10694
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10694
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: io
>            Reporter: Gopal V
>            Assignee: Gopal V
>              Labels: BB2015-05-TBR
>         Attachments: HADOOP-10694.1.patch, writable-read-sync.png
>
>
> Writable deserialization is slowing down due to a synchronized block within DataInputBuffer$Buffer.
> ByteArrayInputStream::read() is synchronized and this shows up as a slow uncontested
lock.
> Hive ships with its own faster thread-unsafe version with hive.common.io.NonSyncByteArrayInputStream.
> !writable-read-sync.png!
> The DataInputBuffer and Writable deserialization should not require a lock per readInt()/read().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message