hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-1795) Port 0.20-append changes onto 0.20-security-203
Date Thu, 15 Mar 2012 06:45:42 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eli Collins resolved HDFS-1795.
-------------------------------

       Resolution: Fixed
    Fix Version/s: 0.20.205.0
    
> Port 0.20-append changes onto 0.20-security-203
> -----------------------------------------------
>
>                 Key: HDFS-1795
>                 URL: https://issues.apache.org/jira/browse/HDFS-1795
>             Project: Hadoop HDFS
>          Issue Type: Task
>            Reporter: Andrew Purtell
>             Fix For: 0.20.205.0
>
>         Attachments: security-append-patches.zip
>
>
> Port 0.20-append changes onto 0.20-security-203.
> I started with a Git repository cloned from git://git.apache.org/hadoop-common.git .
Branch 'branch-0.20-security-203' was used as the starting point for the work. I then enumerated
over the 0.20-append specific patches in 'branch-0.20-append'. Each was applied if not already
via cherry pick except for as noted below. This process in effect replayed the evolution of
0.20-append branch on top of 0.20-security-203.
> The specific functional changes that HBase absolutely relies upon are specially mentioned.
> Generally I ran the full test suite after each change. There were a couple of exceptions
where pairs of adjacent change sets were strongly related, in which case I applied them in
sequence, then ran the test suite. During this process I encountered no test failures except
for one test in TestFileAppend4, a test brought in from the append branch, and I still need
to dig in to see if this is a real problem or if the test needs to be changed to work on top
of security-203. 
> {noformat}
>   commit b9ad012eaf3915c2169a02a7130b54cbcc1d8a89
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Fri Jun 4 07:20:10 2010 +0000
>     HDFS-200. Support append and sync for hadoop 0.20 branch.
>  
>     Required for HBase
>   commit c968e11b5a60fc6f28e4e43fbbc8a99e7e49a659
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Wed Jun 9 23:09:07 2010 +0000
>     HDFS-101. DFSClient correctly detects second datanode failure in write
>     pipeline. (Nicolas Spiegelberg via dhruba)
>   Excluded
>   Already in 0.20-security-203 according to search of Git change log
>   commit 9f7e5ed2ff47444a1dcd12ed34796929d5b9f7d5
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Wed Jun 9 23:12:21 2010 +0000
>     HDFS-988. Fix bug where savenameSpace can corrupt edits log.
>     (Nicolas Spiegelberg via dhruba)
>   commit dfbbd6fbadaa95c54a1040b4fe8854b1b858d7a5
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Thu Jun 10 18:46:03 2010 +0000
>     HDFS-826. Allow a mechanism for an application to detect that
>     datanode(s) have died in the write pipeline. (dhruba)
>   Required for HBase
>   Commit be8d32503d30208a2d7772b3b4b2a270938a4004
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Thu Jun 10 22:25:39 2010 +0000
>     HDFS-142. Blocks that are being written by a client are stored in the
>     blocksBeingWritten directory.
>     (Dhruba Borthakur, Nicolas Spiegelberg, Todd Lipcon via dhruba)
>   commit 856efc2e95aaacc597d669c1b053634ff752dbec
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Fri Jun 11 00:48:41 2010 +0000
>     HDFS-630. Client can exclude specific nodes in the write pipeline.
>     (Nicolas Spiegelberg via dhruba)
>   Required for HBase
>   commit 2da1a05fc0cc0429229e87694977bae2ba370625
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Fri Jun 11 01:02:13 2010 +0000
>     HDFS-457. Better handling of volume failure in DataNode Storage.
>     (Nicolas Spiegelberg via dhruba)
>   Excluded
>   Already in 0.20-security-203 according to search of Git change log
>   commit bd42393cd3a3a731ea98b25ddb528ad03a1ab4af
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Fri Jun 11 23:37:38 2010 +0000
>     HDFS-1054. remove sleep before retry for allocating a block.
>     (Todd Lipcon via dhruba)
>   commit 120441b9e571a5703ac39b47608e87182f0f4972
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Wed Jun 16 20:53:12 2010 +0000
>     HDFS-445. pread should refetch block locations when necessary.
>     (Todd Lipcon via dhruba)
>   Excluded
>   Already in 0.20-security-203 according to search of Git change log
>   commit 2004aa453ba6b7ee2045093ba313ef8551a7f8da
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Wed Jun 16 20:59:10 2010 +0000
>     HDFS-561. Fix write pipeline
>   commit 2a8227b0e6be8937fc4a654899be2a22c1f6efbe
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Wed Jun 16 21:13:24 2010 +0000
>     HDFS-927. DFSInputStream retries too many times for new block
>     locations. (Todd Lipcon via dhruba)
>   Excluded
>   Already in 0.20-security-203 according to search of Git change log
>   commit b1e49dbf50a429cf01b636caa2666ff81ed2a016
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Wed Jun 16 21:21:45 2010 +0000
>     HDFS-1215. Fix unti test TestNodeCount.
>     (Todd Lipcon via dhruba)
>   Excluded
>   TestNodeCount not failing on current working branch
>   commit b5078c57478deda9974ed40897045d38b100b9fc
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Wed Jun 16 21:25:38 2010 +0000
>     HDFS-606. Fix ConcurrentModificationException in
>     invalidateCorruptReplicas. (Todd Lipcon via dhruba)
>   commit c3d35f8cdd761cbcd0a4ef851e37ccdb2d25d7e0
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Wed Jun 16 21:42:53 2010 +0000
>     HDFS-1216. Upgrade to junit 4.5
>     (Todd Lipcon via dhruba)
>   Excluded
>   Already using Junit 4.5 on current working branch
>   commit c21b6834e982b0bf2edaf61dae8a0028076ccb60
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Thu Jun 17 01:34:56 2010 +0000
>     HDFS-1207. FSNamesystem.stallReplicationWork should be volatile.
>     (Todd Lipcon via dhruba)
>   commit e8a1a09fec716bb013f94f1c14f4ae8f8409ee68
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Wed Jun 16 21:57:39 2010 +0000
>     HDFS-1141. completeFile does not check lease ownership.
>     (Todd Lipcon via dhruba)
>   commit ca458a07a6d8cb5cda8c589b1a23dabffacef7fe
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Thu Jun 17 01:44:32 2010 +0000
>     HDFS-1204. Lease expiration should recover single files,
>     not entire lease holder (Sam Rash via dhruba)
>   commit 62441fbd516ec9132619d448a1051554d29d2dba
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Thu Jun 17 01:52:50 2010 +0000
>     HDFS-1210. DFSClient should log exception when block recovery fails.
>     (Todd Lipcon via dhruba)
>   Not a required change but a minor useful one
>   commit 08b67cdf9ff7dada8aa86a2443a0ffc1a0a83c20
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Thu Jun 17 01:59:30 2010 +0000
>     HDFS-1211. Block receiver should not log "rewind" packets at INFO
>     level. (Todd Lipcon via dhruba)
>   Not a required change but a minor useful one
>   commit 0c9911a623891e82c40d4c1c3a523f3a85a85216
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Sun Jun 20 08:31:54 2010 +0000
>     HDFS-1118. Fix socketleak on DFSClient.
>     (Zheng Shao via dhruba)
>   commit db9989c091b37446e4f4d468b24c4bd08b1ea337
>   Author: Tsz-wo Sze <szetszwo@apache.org>
>   Date:   Mon Jun 21 21:26:45 2010 +0000
>     MAPREDUCE-1880. Fix BigDecimal.divide(..) in the pi example.
>   Excluded
>   Not an HDFS change
>   commit 11fa325540526da57a9e178463017853058ba921
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Tue Jun 22 05:55:48 2010 +0000
>     HDFS-1254. Support append/sync via the default configuration.
>     (dhruba)
>   commit 9e405ca177aba719b04cfd7d33d400d311bf8db7
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Thu Jul 1 08:37:30 2010 +0000
>     HDFS-1057.  Concurrent readers hit ChecksumExceptions if following
>     a writer to very end of file (Sam Rash via dhruba)
>   commit 6512b527cda8efa0c622a8657af96ac785625773
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Tue Jul 6 06:52:20 2010 +0000
>     HDFS-1202.  DataBlockScanner throws NPE when updated before
>     initialized. (Todd Lipcon via dhruba)
>   commit 116e71fea80c31042520f364625ba7b1647cdb09
>   Author: Tsz-wo Sze <szetszwo@apache.org>
>   Date:   Tue Jul 6 17:39:25 2010 +0000
>     HDFS-1258. Clearing namespace quota on "/" corrupts fs image.  
>   commit ca88978b319debe146e9988d882393033b1d2d62
>   Author: Tsz-wo Sze <szetszwo@apache.org>
>   Date:   Wed Sep 22 20:52:25 2010 +0000
>     MAPREDUCE-1280. Update Eclipse plugin to the new eclipse.jdt API.
>   Excluded
>   Not an HDFS change
>   commit a68b163fc31c3d82c63b69ae99e9954fef4df4af
>   Author: Hairong Kuang <hairong@apache.org>
>   Date:   Mon Sep 27 18:14:20 2010 +0000
>     HDFS-1346. DFSClient receives out of order packet ack.
>   commit 7083b46208f56776be9ada2d0e60d95cb4fdb67e
>   Author: Hairong Kuang <hairong@apache.org>
>   Date:   Mon Nov 1 06:17:40 2010 +0000
>     HDFS-724.  Use a bidirectional heartbeat to detect stuck pipeline.
>   commit eb2346ea7726c33715f3e651f4babcb6e6c3f0c2
>   Author: Hairong Kuang <hairong@apache.org>
>   Date:   Tue Nov 16 17:53:17 2010 +0000
>     Fix incorrect heartbeat ack serialization in HDFS-724 for append 0.20.
>   Combined with 7083b462
>   commit 0e996bdba420fd0031490457e27dda67f58de18f
>   Author: Eli Collins <eli@apache.org>
>   Date:   Tue Nov 9 01:59:27 2010 +0000
>     HDFS-1240. svn merge -c 1032810 from branch-0.20.
>   Excluded
>   TestDFSShell not failing on working branch
>   Commit eb2346ea7726c33715f3e651f4babcb6e6c3f0c2
>   Author: Hairong Kuang <hairong@apache.org>
>   Date:   Tue Nov 16 17:53:17 2010 +0000
>     HDFS-895. Allow hflush/sync to occur in parallel with new writes to
>     the file
>   Required to mitigate HLog flush penalty
>   Need to raise default heap for tests for HDFS-895 unit tests to pass
>   commit 9a3094f3a085817d377ce69fb8eb002f966f992b
>   Author: Dhruba Borthakur <dhruba@apache.org>
>   Date:   Fri Dec 10 05:38:06 2010 +0000
>     HDFS-1520. Lightweight NameNode operation recoverLease to trigger
>     lease recovery. (Hairong Kuang via dhruba)
>   Required by HBase for master to seize ownership of and quickly split
>   HLogs of failed regionservers
>   commit f9436bf6bd2f5bc4f8cebe2781c5ce628db31239
>   Author: Hairong Kuang <hairong@apache.org>
>   Date:   Fri Jan 7 20:11:38 2011 +0000
>     HDFS-1555. Disallow pipeline recovery if a file is already being lease
>   Excluded
>   Bad commit
>   commit 966ef38cb13d75ada40de2b9951496ddcb6918a4
>   Author: Hairong Kuang <hairong@apache.org>
>   Date:   Fri Jan 7 20:19:40 2011 +0000
>     Revert the change made to HDFS-1555:
>   Excluded
>   commit e499be8c74a871ef06aa8a74c621e68d50167d3d
>   Author: Hairong Kuang <hairong@apache.org>
>   Date:   Fri Jan 7 20:25:23 2011 +0000
>     HDFS-1555. Disallow pipelien recovery if a file is already being
>     lease recovered
>   commit df0d79cc2b09438c079fdf10b913936492117917
>   Author: Hairong Kuang <hairong@apache.org>
>   Date:   Mon Jan 10 19:01:36 2011 +0000
>     HDFS-1554. New semantics for recoverLease.
>   Required by HBase for master to seize ownership of and quickly split
>   HLogs of failed regionservers
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message