hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8382) Remove chunkSize parameter from initialize method of raw erasure coder
Date Fri, 22 May 2015 06:53:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555688#comment-14555688
] 

Vinayakumar B commented on HDFS-8382:
-------------------------------------

[~drankye] and [~aw], I think the problem is, when the jenkins  is running *Precommit-HDFS-Build*
on the branch HDFS-7285 with running hadoop-hdfs module tests, it depends on hadoop-common
and hadoop-hdfs-client module's jars present in local maven repo. 

At the same time, these jars can be replaced by another hadoop job of *Precommit-HADOOP-Build*
or any other Hadoop project running on different branch.

So for the HDFS-7285's tests, extra classes added in same branch and in some other module
(hadoop-common/hadoop-hdfs-client) will be missing.

Till now we have seen failures with missing classes from hadoop-common and hadoop-hdfs-client
modules while running hadoop-hdfs tests.

I think the solution will be like having a separate maven repo for each jenkins project to
avoid the collisions, even though results in duplicate contents of repo. What you say [~aw]
?

Similar problem would have been experienced earlier when the patch involved multiple module
changes.

> Remove chunkSize parameter from initialize method of raw erasure coder
> ----------------------------------------------------------------------
>
>                 Key: HDFS-8382
>                 URL: https://issues.apache.org/jira/browse/HDFS-8382
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>         Attachments: HDFS-8382-HDFS-7285-v1.patch, HDFS-8382-HDFS-7285-v2.patch, HDFS-8382-HDFS-7285-v3.patch,
HDFS-8382-HDFS-7285-v4.patch, HDFS-8382-HDFS-7285-v5.patch
>
>
> Per discussion in HDFS-8347, we need to support encoding/decoding variable width units
data instead of predefined fixed width like {{chunkSize}}. Have this issue to remove chunkSize
in the general raw erasure coder API. Specific coder will support fixed chunkSize using hard-coded
or specific schema customizing way if necessary, like HitchHiker coder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message