hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read
Date Thu, 04 Jun 2015 01:03:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571929#comment-14571929

Jing Zhao commented on HDFS-8319:

bq. So looks like the inputOffsets doesn't work?

The inputOffsets are derived from the given buffer's position, which does not work in the
slice scenario. E.g., suppose we have a (64KB * 6) underlying buffer, and the input buffers
are slices of it (i.e., they are generated by {{ByteBuffer#slice}} call). This means all the
input buffers' array() return the same (64KB * 6) byte array, while their position are totally
independent and can be all 0.

Currently the stateful read uses direct buffer thus the patch does not depend on the fix.
We can fix it in a separate jira.

bq. How about: 1) add clear Javadoc in the APIs? 2) add codes to check the requirement, like
ensureTheSameBuffer similar to ensureLength?

Yeah, sounds good to me. One question is whether the mixed scenario breaks the functionality?
If not maybe we can still allow it. But we can discuss this in the separate jira.

For now can we just leave the direct buffer detection change there as a temporary workaround?
We can change it after we have clearer thoughts about the buffer handling.

> Erasure Coding: support decoding for stateful read
> --------------------------------------------------
>                 Key: HDFS-8319
>                 URL: https://issues.apache.org/jira/browse/HDFS-8319
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>         Attachments: HDFS-8319.001.patch, HDFS-8319.002.patch, HDFS-8319.003.patch
> HDFS-7678 adds the decoding functionality for pread. This jira plans to add decoding
to stateful read.

This message was sent by Atlassian JIRA

View raw message