Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 1AB2E200CE7 for ; Fri, 1 Sep 2017 22:01:12 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 196D416DA4F; Fri, 1 Sep 2017 20:01:12 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 4205116DA48 for ; Fri, 1 Sep 2017 22:01:10 +0200 (CEST) Received: (qmail 24428 invoked by uid 500); 1 Sep 2017 20:00:53 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 20310 invoked by uid 99); 1 Sep 2017 20:00:51 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Sep 2017 20:00:51 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 913E9F5654; Fri, 1 Sep 2017 20:00:50 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: aengineer@apache.org To: common-commits@hadoop.apache.org Date: Fri, 01 Sep 2017 20:01:24 -0000 Message-Id: <7396cc12bab642f7a3f9386794745b6f@git.apache.org> In-Reply-To: <20fd9b3290104dfda8af410f6d60d213@git.apache.org> References: <20fd9b3290104dfda8af410f6d60d213@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [36/74] [abbrv] [partial] hadoop git commit: HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build archived-at: Fri, 01 Sep 2017 20:01:12 -0000 http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/RELEASENOTES.0.20.2.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/RELEASENOTES.0.20.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/RELEASENOTES.0.20.2.md index 2ebfdc0..5243c7e 100644 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/RELEASENOTES.0.20.2.md +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/RELEASENOTES.0.20.2.md @@ -23,23 +23,16 @@ These release notes cover new developer and user-facing incompatibilities, impor --- -* [HADOOP-6498](https://issues.apache.org/jira/browse/HADOOP-6498) | *Blocker* | **IPC client bug may cause rpc call hang** - -Correct synchronization error in IPC where handler thread could hang if request reader got an error. - - ---- - -* [HADOOP-6460](https://issues.apache.org/jira/browse/HADOOP-6460) | *Blocker* | **Namenode runs of out of memory due to memory leak in ipc Server** +* [MAPREDUCE-826](https://issues.apache.org/jira/browse/MAPREDUCE-826) | *Trivial* | **harchive doesn't use ToolRunner / harchive returns 0 even if the job fails with exception** -If an IPC server response buffer has grown to than 1MB, it is replaced by a smaller buffer to free up the Java heap that was used. This will improve the longevity of the name service. +Use ToolRunner for archives job and return non zero error code on failure. --- -* [HADOOP-6428](https://issues.apache.org/jira/browse/HADOOP-6428) | *Major* | **HttpServer sleeps with negative values** +* [MAPREDUCE-112](https://issues.apache.org/jira/browse/MAPREDUCE-112) | *Blocker* | **Reduce Input Records and Reduce Output Records counters are not being set when using the new Mapreduce reducer API** -Corrected arithmetic error that made sleep times less than zero. +Updates of counters for reduce input and output records were added in the new API so they are available for jobs using the new API. --- @@ -51,23 +44,23 @@ Allow a general mechanism to disable the cache on a per filesystem basis by usin --- -* [HADOOP-6097](https://issues.apache.org/jira/browse/HADOOP-6097) | *Major* | **Multiple bugs w/ Hadoop archives** +* [MAPREDUCE-979](https://issues.apache.org/jira/browse/MAPREDUCE-979) | *Blocker* | **JobConf.getMemoryFor{Map\|Reduce}Task doesn't fallback to newer config knobs when mapred.taskmaxvmem is set to DISABLED\_MEMORY\_LIMIT of -1** -Bugs fixed for Hadoop archives: character escaping in paths, LineReader and file system caching. +Added support to fallback to new task memory configuration when deprecated memory configuration values are set to disabled. --- -* [HDFS-793](https://issues.apache.org/jira/browse/HDFS-793) | *Blocker* | **DataNode should first receive the whole packet ack message before it constructs and sends its own ack message for the packet** +* [HDFS-677](https://issues.apache.org/jira/browse/HDFS-677) | *Blocker* | **Rename failure due to quota results in deletion of src directory** -**WARNING: No release note provided for this incompatible change.** +Rename properly considers the case where both source and destination are over quota; operation will fail with error indication. --- -* [HDFS-781](https://issues.apache.org/jira/browse/HDFS-781) | *Blocker* | **Metrics PendingDeletionBlocks is not decremented** +* [HADOOP-6097](https://issues.apache.org/jira/browse/HADOOP-6097) | *Major* | **Multiple bugs w/ Hadoop archives** -Correct PendingDeletionBlocks metric to properly decrement counts. +Bugs fixed for Hadoop archives: character escaping in paths, LineReader and file system caching. --- @@ -79,9 +72,9 @@ Corrected an error when checking quota policy that resulted in a failure to read --- -* [HDFS-677](https://issues.apache.org/jira/browse/HDFS-677) | *Blocker* | **Rename failure due to quota results in deletion of src directory** +* [MAPREDUCE-1068](https://issues.apache.org/jira/browse/MAPREDUCE-1068) | *Major* | **In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present** -Rename properly considers the case where both source and destination are over quota; operation will fail with error indication. +Fix streaming job to show proper message if file is is not present, for -file option. --- @@ -93,44 +86,44 @@ Memory leak in function hdfsFreeFileInfo in libhdfs. This bug affects fuse-dfs s --- -* [MAPREDUCE-1182](https://issues.apache.org/jira/browse/MAPREDUCE-1182) | *Blocker* | **Reducers fail with OutOfMemoryError while copying Map outputs** +* [MAPREDUCE-1147](https://issues.apache.org/jira/browse/MAPREDUCE-1147) | *Blocker* | **Map output records counter missing for map-only jobs in new API** -Modifies shuffle related memory parameters to use 'long' from 'int' so that sizes greater than maximum integer size are handled correctly +Adds a counter to track the number of records emitted by map writing directly to HDFS i.e map tasks of job with 0 reducers. --- -* [MAPREDUCE-1147](https://issues.apache.org/jira/browse/MAPREDUCE-1147) | *Blocker* | **Map output records counter missing for map-only jobs in new API** +* [MAPREDUCE-1182](https://issues.apache.org/jira/browse/MAPREDUCE-1182) | *Blocker* | **Reducers fail with OutOfMemoryError while copying Map outputs** -Adds a counter to track the number of records emitted by map writing directly to HDFS i.e map tasks of job with 0 reducers. +Modifies shuffle related memory parameters to use 'long' from 'int' so that sizes greater than maximum integer size are handled correctly --- -* [MAPREDUCE-1068](https://issues.apache.org/jira/browse/MAPREDUCE-1068) | *Major* | **In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present** +* [HDFS-781](https://issues.apache.org/jira/browse/HDFS-781) | *Blocker* | **Metrics PendingDeletionBlocks is not decremented** -Fix streaming job to show proper message if file is is not present, for -file option. +Correct PendingDeletionBlocks metric to properly decrement counts. --- -* [MAPREDUCE-979](https://issues.apache.org/jira/browse/MAPREDUCE-979) | *Blocker* | **JobConf.getMemoryFor{Map\|Reduce}Task doesn't fallback to newer config knobs when mapred.taskmaxvmem is set to DISABLED\_MEMORY\_LIMIT of -1** +* [HDFS-793](https://issues.apache.org/jira/browse/HDFS-793) | *Blocker* | **DataNode should first receive the whole packet ack message before it constructs and sends its own ack message for the packet** -Added support to fallback to new task memory configuration when deprecated memory configuration values are set to disabled. +**WARNING: No release note provided for this change.** --- -* [MAPREDUCE-826](https://issues.apache.org/jira/browse/MAPREDUCE-826) | *Trivial* | **harchive doesn't use ToolRunner / harchive returns 0 even if the job fails with exception** +* [HADOOP-6428](https://issues.apache.org/jira/browse/HADOOP-6428) | *Major* | **HttpServer sleeps with negative values** -Use ToolRunner for archives job and return non zero error code on failure. +Corrected arithmetic error that made sleep times less than zero. --- -* [MAPREDUCE-623](https://issues.apache.org/jira/browse/MAPREDUCE-623) | *Major* | **Resolve javac warnings in mapred** +* [HADOOP-6460](https://issues.apache.org/jira/browse/HADOOP-6460) | *Blocker* | **Namenode runs of out of memory due to memory leak in ipc Server** -Removes javac warnings by either resolving them or suppressing them (wherever resolution is not possible) +If an IPC server response buffer has grown to than 1MB, it is replaced by a smaller buffer to free up the Java heap that was used. This will improve the longevity of the name service. --- @@ -142,9 +135,16 @@ Resolves the test failure by modifying the test to base it on spill counters rat --- -* [MAPREDUCE-112](https://issues.apache.org/jira/browse/MAPREDUCE-112) | *Blocker* | **Reduce Input Records and Reduce Output Records counters are not being set when using the new Mapreduce reducer API** +* [HADOOP-6498](https://issues.apache.org/jira/browse/HADOOP-6498) | *Blocker* | **IPC client bug may cause rpc call hang** -Updates of counters for reduce input and output records were added in the new API so they are available for jobs using the new API. +Correct synchronization error in IPC where handler thread could hang if request reader got an error. + + +--- + +* [MAPREDUCE-623](https://issues.apache.org/jira/browse/MAPREDUCE-623) | *Major* | **Resolve javac warnings in mapred** + +Removes javac warnings by either resolving them or suppressing them (wherever resolution is not possible) http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/CHANGES.0.20.203.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/CHANGES.0.20.203.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/CHANGES.0.20.203.0.md index b45f75a..72c0742 100644 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/CHANGES.0.20.203.0.md +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/CHANGES.0.20.203.0.md @@ -20,16 +20,6 @@ ## Release 0.20.203.0 - 2011-05-11 -### INCOMPATIBLE CHANGES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### IMPORTANT ISSUES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | ### NEW FEATURES: @@ -44,58 +34,40 @@ | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [HADOOP-7247](https://issues.apache.org/jira/browse/HADOOP-7247) | Fix documentation to reflect new jar names | Major | . | Owen O'Malley | Owen O'Malley | -| [HDFS-1626](https://issues.apache.org/jira/browse/HDFS-1626) | Make BLOCK\_INVALIDATE\_LIMIT configurable | Minor | namenode | Arun C Murthy | Tsz Wo Nicholas Sze | +| [MAPREDUCE-478](https://issues.apache.org/jira/browse/MAPREDUCE-478) | separate jvm param for mapper and reducer | Minor | . | Koji Noguchi | Arun C Murthy | | [HDFS-457](https://issues.apache.org/jira/browse/HDFS-457) | better handling of volume failure in Data Node storage | Major | datanode | Boris Shkolnik | Boris Shkolnik | -| [MAPREDUCE-2355](https://issues.apache.org/jira/browse/MAPREDUCE-2355) | Add an out of band heartbeat damper | Major | jobtracker | Owen O'Malley | Arun C Murthy | +| [HDFS-1626](https://issues.apache.org/jira/browse/HDFS-1626) | Make BLOCK\_INVALIDATE\_LIMIT configurable | Minor | namenode | Arun C Murthy | Tsz Wo Nicholas Sze | | [MAPREDUCE-2316](https://issues.apache.org/jira/browse/MAPREDUCE-2316) | Update docs for CapacityScheduler | Major | capacity-sched, documentation | Arun C Murthy | Arun C Murthy | +| [HADOOP-7247](https://issues.apache.org/jira/browse/HADOOP-7247) | Fix documentation to reflect new jar names | Major | . | Owen O'Malley | Owen O'Malley | | [MAPREDUCE-1943](https://issues.apache.org/jira/browse/MAPREDUCE-1943) | Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes | Major | . | Mahadev konar | Mahadev konar | -| [MAPREDUCE-478](https://issues.apache.org/jira/browse/MAPREDUCE-478) | separate jvm param for mapper and reducer | Minor | . | Koji Noguchi | Arun C Murthy | +| [MAPREDUCE-2355](https://issues.apache.org/jira/browse/MAPREDUCE-2355) | Add an out of band heartbeat damper | Major | jobtracker | Owen O'Malley | Arun C Murthy | ### BUG FIXES: | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [HADOOP-7259](https://issues.apache.org/jira/browse/HADOOP-7259) | contrib modules should include build.properties from parent. | Major | build | Owen O'Malley | Owen O'Malley | -| [HADOOP-7258](https://issues.apache.org/jira/browse/HADOOP-7258) | Gzip codec should not return null decompressors | Major | . | Owen O'Malley | Owen O'Malley | -| [HADOOP-7253](https://issues.apache.org/jira/browse/HADOOP-7253) | Fix default config | Major | . | Owen O'Malley | Owen O'Malley | -| [HADOOP-7246](https://issues.apache.org/jira/browse/HADOOP-7246) | The default log4j configuration causes warnings about EventCounter | Major | . | Owen O'Malley | Luke Lu | -| [HADOOP-7243](https://issues.apache.org/jira/browse/HADOOP-7243) | Fix contrib unit tests (fairshare, hdfsproxy, datajoin, streaming) | Major | . | Owen O'Malley | Owen O'Malley | -| [HADOOP-7232](https://issues.apache.org/jira/browse/HADOOP-7232) | Fix javadoc warnings | Blocker | documentation | Owen O'Malley | Owen O'Malley | -| [HADOOP-7215](https://issues.apache.org/jira/browse/HADOOP-7215) | RPC clients must connect over a network interface corresponding to the host name in the client's kerberos principal key | Blocker | security | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6907](https://issues.apache.org/jira/browse/HADOOP-6907) | Rpc client doesn't use the per-connection conf to figure out server's Kerberos principal | Major | ipc, security | Kan Zhang | Kan Zhang | +| [MAPREDUCE-1280](https://issues.apache.org/jira/browse/MAPREDUCE-1280) | Eclipse Plugin does not work with Eclipse Ganymede (3.4) | Major | . | Aaron Kimball | Alex Kozlov | | [HADOOP-7190](https://issues.apache.org/jira/browse/HADOOP-7190) | Put metrics v1 back into the hadoop-20-security branch | Major | metrics | Owen O'Malley | Owen O'Malley | -| [HADOOP-7163](https://issues.apache.org/jira/browse/HADOOP-7163) | "java.net.SocketTimeoutException: 60000 millis timeout" happens a lot | Major | ipc | Owen O'Malley | Devaraj Das | +| [HADOOP-7232](https://issues.apache.org/jira/browse/HADOOP-7232) | Fix javadoc warnings | Blocker | documentation | Owen O'Malley | Owen O'Malley | | [HADOOP-7143](https://issues.apache.org/jira/browse/HADOOP-7143) | Hive Hadoop20SShims depends on removed HadoopArchives | Major | fs | Joep Rottinghuis | Joep Rottinghuis | -| [HADOOP-7040](https://issues.apache.org/jira/browse/HADOOP-7040) | DiskChecker:mkdirsWithExistsCheck swallows FileNotFoundException. | Major | . | Boris Shkolnik | Boris Shkolnik | -| [HADOOP-6907](https://issues.apache.org/jira/browse/HADOOP-6907) | Rpc client doesn't use the per-connection conf to figure out server's Kerberos principal | Major | ipc, security | Kan Zhang | Kan Zhang | -| [HADOOP-5647](https://issues.apache.org/jira/browse/HADOOP-5647) | TestJobHistory fails if /tmp/\_logs is not writable to. Testcase should not depend on /tmp | Major | test | Ravi Gummadi | Ravi Gummadi | | [HDFS-1822](https://issues.apache.org/jira/browse/HDFS-1822) | Editlog opcodes overlap between 20 security and later releases | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7246](https://issues.apache.org/jira/browse/HADOOP-7246) | The default log4j configuration causes warnings about EventCounter | Major | . | Owen O'Malley | Luke Lu | +| [HADOOP-7253](https://issues.apache.org/jira/browse/HADOOP-7253) | Fix default config | Major | . | Owen O'Malley | Owen O'Malley | | [HDFS-1022](https://issues.apache.org/jira/browse/HDFS-1022) | Merge under-10-min tests specs into one file | Major | test | Erik Steffl | Erik Steffl | +| [HADOOP-7259](https://issues.apache.org/jira/browse/HADOOP-7259) | contrib modules should include build.properties from parent. | Major | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-7215](https://issues.apache.org/jira/browse/HADOOP-7215) | RPC clients must connect over a network interface corresponding to the host name in the client's kerberos principal key | Blocker | security | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7258](https://issues.apache.org/jira/browse/HADOOP-7258) | Gzip codec should not return null decompressors | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-5647](https://issues.apache.org/jira/browse/HADOOP-5647) | TestJobHistory fails if /tmp/\_logs is not writable to. Testcase should not depend on /tmp | Major | test | Ravi Gummadi | Ravi Gummadi | +| [HADOOP-7040](https://issues.apache.org/jira/browse/HADOOP-7040) | DiskChecker:mkdirsWithExistsCheck swallows FileNotFoundException. | Major | . | Boris Shkolnik | Boris Shkolnik | | [MAPREDUCE-2365](https://issues.apache.org/jira/browse/MAPREDUCE-2365) | Add counters for FileInputFormat (BYTES\_READ) and FileOutputFormat (BYTES\_WRITTEN) | Major | . | Owen O'Malley | Siddharth Seth | +| [HADOOP-7163](https://issues.apache.org/jira/browse/HADOOP-7163) | "java.net.SocketTimeoutException: 60000 millis timeout" happens a lot | Major | ipc | Owen O'Malley | Devaraj Das | +| [HADOOP-7243](https://issues.apache.org/jira/browse/HADOOP-7243) | Fix contrib unit tests (fairshare, hdfsproxy, datajoin, streaming) | Major | . | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-323](https://issues.apache.org/jira/browse/MAPREDUCE-323) | Improve the way job history files are managed | Critical | jobtracker | Amar Kamat | Dick King | | [MAPREDUCE-2278](https://issues.apache.org/jira/browse/MAPREDUCE-2278) | DistributedCache shouldn't hold a ref to JobConf | Major | distributed-cache, tasktracker | Arun C Murthy | Chris Douglas | | [MAPREDUCE-1699](https://issues.apache.org/jira/browse/MAPREDUCE-1699) | JobHistory shouldn't be disabled for any reason | Major | jobtracker | Arun C Murthy | Krishna Ramachandran | -| [MAPREDUCE-1280](https://issues.apache.org/jira/browse/MAPREDUCE-1280) | Eclipse Plugin does not work with Eclipse Ganymede (3.4) | Major | . | Aaron Kimball | Alex Kozlov | -| [MAPREDUCE-1233](https://issues.apache.org/jira/browse/MAPREDUCE-1233) | Incorrect Waiting maps/reduces in Jobtracker metrics | Major | jobtracker | V.Karthikeyan | Luke Lu | | [MAPREDUCE-1118](https://issues.apache.org/jira/browse/MAPREDUCE-1118) | Capacity Scheduler scheduling information is hard to read / should be tabular format | Major | capacity-sched | Allen Wittenauer | Krishna Ramachandran | -| [MAPREDUCE-323](https://issues.apache.org/jira/browse/MAPREDUCE-323) | Improve the way job history files are managed | Critical | jobtracker | Amar Kamat | Dick King | - - -### TESTS: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### SUB-TASKS: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### OTHER: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-1233](https://issues.apache.org/jira/browse/MAPREDUCE-1233) | Incorrect Waiting maps/reduces in Jobtracker metrics | Major | jobtracker | V.Karthikeyan | Luke Lu | http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/RELEASENOTES.0.20.203.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/RELEASENOTES.0.20.203.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/RELEASENOTES.0.20.203.0.md index 4138475..57a6807 100644 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/RELEASENOTES.0.20.203.0.md +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/RELEASENOTES.0.20.203.0.md @@ -23,16 +23,20 @@ These release notes cover new developer and user-facing incompatibilities, impor --- -* [HADOOP-5647](https://issues.apache.org/jira/browse/HADOOP-5647) | *Major* | **TestJobHistory fails if /tmp/\_logs is not writable to. Testcase should not depend on /tmp** - -Removed dependency of testcase on /tmp and made it to use test.build.data directory instead. - - ---- +* [MAPREDUCE-478](https://issues.apache.org/jira/browse/MAPREDUCE-478) | *Minor* | **separate jvm param for mapper and reducer** -* [HDFS-1626](https://issues.apache.org/jira/browse/HDFS-1626) | *Minor* | **Make BLOCK\_INVALIDATE\_LIMIT configurable** +Allow map and reduce jvm parameters, environment variables and ulimit to be set separately. -Added a new configuration property dfs.block.invalidate.limit for FSNamesystem.blockInvalidateLimit. +Configuration changes: + add mapred.map.child.java.opts + add mapred.reduce.child.java.opts + add mapred.map.child.env + add mapred.reduce.child.ulimit + add mapred.map.child.env + add mapred.reduce.child.ulimit + deprecated mapred.child.java.opts + deprecated mapred.child.env + deprecated mapred.child.ulimit --- @@ -44,27 +48,16 @@ Datanode can continue if a volume for replica storage fails. Previously a datano --- -* [MAPREDUCE-1118](https://issues.apache.org/jira/browse/MAPREDUCE-1118) | *Major* | **Capacity Scheduler scheduling information is hard to read / should be tabular format** +* [HDFS-1626](https://issues.apache.org/jira/browse/HDFS-1626) | *Minor* | **Make BLOCK\_INVALIDATE\_LIMIT configurable** -Add CapacityScheduler servlet to enhance web UI for queue information. +Added a new configuration property dfs.block.invalidate.limit for FSNamesystem.blockInvalidateLimit. --- -* [MAPREDUCE-478](https://issues.apache.org/jira/browse/MAPREDUCE-478) | *Minor* | **separate jvm param for mapper and reducer** - -Allow map and reduce jvm parameters, environment variables and ulimit to be set separately. +* [HADOOP-5647](https://issues.apache.org/jira/browse/HADOOP-5647) | *Major* | **TestJobHistory fails if /tmp/\_logs is not writable to. Testcase should not depend on /tmp** -Configuration changes: - add mapred.map.child.java.opts - add mapred.reduce.child.java.opts - add mapred.map.child.env - add mapred.reduce.child.ulimit - add mapred.map.child.env - add mapred.reduce.child.ulimit - deprecated mapred.child.java.opts - deprecated mapred.child.env - deprecated mapred.child.ulimit +Removed dependency of testcase on /tmp and made it to use test.build.data directory instead. --- @@ -114,4 +107,9 @@ The new API described above, which can be used to programmatically obtain histor throws IOException +--- + +* [MAPREDUCE-1118](https://issues.apache.org/jira/browse/MAPREDUCE-1118) | *Major* | **Capacity Scheduler scheduling information is hard to read / should be tabular format** + +Add CapacityScheduler servlet to enhance web UI for queue information. http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGES.0.20.203.1.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGES.0.20.203.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGES.0.20.203.1.md index a288a36..66bce3f 100644 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGES.0.20.203.1.md +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGES.0.20.203.1.md @@ -18,30 +18,8 @@ --> # Apache Hadoop Changelog -## Release 0.20.203.1 - Unreleased (as of 2016-03-04) +## Release 0.20.203.1 - Unreleased (as of 2017-08-28) -### INCOMPATIBLE CHANGES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### IMPORTANT ISSUES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### NEW FEATURES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### IMPROVEMENTS: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | ### BUG FIXES: @@ -52,21 +30,3 @@ | [HADOOP-7297](https://issues.apache.org/jira/browse/HADOOP-7297) | Error in the documentation regarding Checkpoint/Backup Node | Trivial | documentation | arnaud p | Harsh J | -### TESTS: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### SUB-TASKS: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### OTHER: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/CHANGES.0.20.204.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/CHANGES.0.20.204.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/CHANGES.0.20.204.0.md index a9d18ef..d08c867 100644 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/CHANGES.0.20.204.0.md +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/CHANGES.0.20.204.0.md @@ -27,19 +27,13 @@ | [HADOOP-6255](https://issues.apache.org/jira/browse/HADOOP-6255) | Create an rpm integration project | Major | . | Owen O'Malley | Eric Yang | -### IMPORTANT ISSUES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - ### NEW FEATURES: | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [HADOOP-7398](https://issues.apache.org/jira/browse/HADOOP-7398) | create a mechanism to suppress the HADOOP\_HOME deprecated warning | Major | . | Owen O'Malley | Owen O'Malley | | [HADOOP-7144](https://issues.apache.org/jira/browse/HADOOP-7144) | Expose JMX with something like JMXProxyServlet | Major | . | Luke Lu | Robert Joseph Evans | | [MAPREDUCE-2558](https://issues.apache.org/jira/browse/MAPREDUCE-2558) | Add queue-level metrics 0.20-security branch | Major | jobtracker | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [HADOOP-7398](https://issues.apache.org/jira/browse/HADOOP-7398) | create a mechanism to suppress the HADOOP\_HOME deprecated warning | Major | . | Owen O'Malley | Owen O'Malley | | [MAPREDUCE-1938](https://issues.apache.org/jira/browse/MAPREDUCE-1938) | Ability for having user's classes take precedence over the system classes for tasks' classpath | Blocker | job submission, task, tasktracker | Devaraj Das | Krishna Ramachandran | @@ -47,87 +41,81 @@ | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [HADOOP-7277](https://issues.apache.org/jira/browse/HADOOP-7277) | Add Eclipse launch tasks for the 0.20-security branch | Minor | build | Jeffrey Naisbitt | Jeffrey Naisbitt | -| [HADOOP-7248](https://issues.apache.org/jira/browse/HADOOP-7248) | Have a way to automatically update Eclipse .classpath file when new libs are added to the classpath through Ivy for 0.20-\* based sources | Minor | build | Konstantin Boudnik | Thomas Graves | | [HDFS-1773](https://issues.apache.org/jira/browse/HDFS-1773) | Remove a datanode from cluster if include list is not empty and this datanode is removed from both include and exclude lists | Minor | namenode | Tanping Wang | Tanping Wang | -| [MAPREDUCE-2524](https://issues.apache.org/jira/browse/MAPREDUCE-2524) | Backport trunk heuristics for failing maps when we get fetch failures retrieving map output during shuffle | Minor | tasktracker | Thomas Graves | Thomas Graves | -| [MAPREDUCE-2495](https://issues.apache.org/jira/browse/MAPREDUCE-2495) | The distributed cache cleanup thread has no monitoring to check to see if it has died for some reason | Minor | distributed-cache | Robert Joseph Evans | Robert Joseph Evans | -| [MAPREDUCE-2490](https://issues.apache.org/jira/browse/MAPREDUCE-2490) | Log blacklist debug count | Trivial | jobtracker | Jonathan Eagles | Jonathan Eagles | -| [MAPREDUCE-2479](https://issues.apache.org/jira/browse/MAPREDUCE-2479) | Backport MAPREDUCE-1568 to hadoop security branch | Major | tasktracker | Robert Joseph Evans | Robert Joseph Evans | | [MAPREDUCE-2456](https://issues.apache.org/jira/browse/MAPREDUCE-2456) | Show the reducer taskid and map/reduce tasktrackers for "Failed fetch notification #\_ for task attempt..." log messages | Trivial | jobtracker | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [HADOOP-7248](https://issues.apache.org/jira/browse/HADOOP-7248) | Have a way to automatically update Eclipse .classpath file when new libs are added to the classpath through Ivy for 0.20-\* based sources | Minor | build | Konstantin Boudnik | Thomas Graves | +| [HADOOP-7277](https://issues.apache.org/jira/browse/HADOOP-7277) | Add Eclipse launch tasks for the 0.20-security branch | Minor | build | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2479](https://issues.apache.org/jira/browse/MAPREDUCE-2479) | Backport MAPREDUCE-1568 to hadoop security branch | Major | tasktracker | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2490](https://issues.apache.org/jira/browse/MAPREDUCE-2490) | Log blacklist debug count | Trivial | jobtracker | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-2495](https://issues.apache.org/jira/browse/MAPREDUCE-2495) | The distributed cache cleanup thread has no monitoring to check to see if it has died for some reason | Minor | distributed-cache | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2524](https://issues.apache.org/jira/browse/MAPREDUCE-2524) | Backport trunk heuristics for failing maps when we get fetch failures retrieving map output during shuffle | Minor | tasktracker | Thomas Graves | Thomas Graves | ### BUG FIXES: | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [HADOOP-7475](https://issues.apache.org/jira/browse/HADOOP-7475) | hadoop-setup-single-node.sh is broken | Blocker | . | Eric Yang | Eric Yang | -| [HADOOP-7373](https://issues.apache.org/jira/browse/HADOOP-7373) | Tarball deployment doesn't work with {start,stop}-{dfs,mapred} | Major | . | Owen O'Malley | Owen O'Malley | -| [HADOOP-7364](https://issues.apache.org/jira/browse/HADOOP-7364) | TestMiniMRDFSCaching fails if test.build.dir is set to something other than build/test | Major | test | Thomas Graves | Thomas Graves | -| [HADOOP-7356](https://issues.apache.org/jira/browse/HADOOP-7356) | RPM packages broke bin/hadoop script for hadoop 0.20.205 | Blocker | . | Eric Yang | Eric Yang | -| [HADOOP-7330](https://issues.apache.org/jira/browse/HADOOP-7330) | The metrics source mbean implementation should return the attribute value instead of the object | Major | metrics | Luke Lu | Luke Lu | -| [HADOOP-7324](https://issues.apache.org/jira/browse/HADOOP-7324) | Ganglia plugins for metrics v2 | Blocker | metrics | Luke Lu | Priyo Mustafi | -| [HADOOP-7274](https://issues.apache.org/jira/browse/HADOOP-7274) | CLONE - IOUtils.readFully and IOUtils.skipFully have typo in exception creation's message | Minor | util | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-118](https://issues.apache.org/jira/browse/MAPREDUCE-118) | Job.getJobID() will always return null | Blocker | client | Amar Kamat | Amareshwari Sriramadasu | +| [HDFS-1258](https://issues.apache.org/jira/browse/HDFS-1258) | Clearing namespace quota on "/" corrupts FS image | Blocker | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1377](https://issues.apache.org/jira/browse/HDFS-1377) | Quota bug for partial blocks allows quotas to be violated | Blocker | namenode | Eli Collins | Eli Collins | +| [HDFS-1750](https://issues.apache.org/jira/browse/HDFS-1750) | fs -ls hftp://file not working | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1758](https://issues.apache.org/jira/browse/HDFS-1758) | Web UI JSP pages thread safety issue | Minor | tools | Tanping Wang | Tanping Wang | +| [MAPREDUCE-2411](https://issues.apache.org/jira/browse/MAPREDUCE-2411) | When you submit a job to a queue with no ACLs you get an inscrutible NPE | Minor | . | Dick King | Dick King | +| [HDFS-1189](https://issues.apache.org/jira/browse/HDFS-1189) | Quota counts missed between clear quota and set quota | Major | namenode | Kang Xiao | John George | +| [MAPREDUCE-2443](https://issues.apache.org/jira/browse/MAPREDUCE-2443) | Fix FI build - broken after MR-2429 | Minor | test | Siddharth Seth | Siddharth Seth | | [HADOOP-7232](https://issues.apache.org/jira/browse/HADOOP-7232) | Fix javadoc warnings | Blocker | documentation | Owen O'Malley | Owen O'Malley | -| [HDFS-2057](https://issues.apache.org/jira/browse/HDFS-2057) | Wait time to terminate the threads causing unit tests to take longer time | Major | datanode | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1822](https://issues.apache.org/jira/browse/HDFS-1822) | Editlog opcodes overlap between 20 security and later releases | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [MAPREDUCE-2451](https://issues.apache.org/jira/browse/MAPREDUCE-2451) | Log the reason string of healthcheck script | Trivial | jobtracker | Thomas Graves | Thomas Graves | +| [HADOOP-7274](https://issues.apache.org/jira/browse/HADOOP-7274) | CLONE - IOUtils.readFully and IOUtils.skipFully have typo in exception creation's message | Minor | util | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-2514](https://issues.apache.org/jira/browse/MAPREDUCE-2514) | ReinitTrackerAction class name misspelled RenitTrackerAction in task tracker log | Trivial | tasktracker | Jonathan Eagles | Jonathan Eagles | +| [HDFS-1592](https://issues.apache.org/jira/browse/HDFS-1592) | Datanode startup doesn't honor volumes.tolerated | Major | . | Bharath Mundlapudi | Bharath Mundlapudi | +| [HADOOP-7330](https://issues.apache.org/jira/browse/HADOOP-7330) | The metrics source mbean implementation should return the attribute value instead of the object | Major | metrics | Luke Lu | Luke Lu | | [HDFS-2023](https://issues.apache.org/jira/browse/HDFS-2023) | Backport of NPE for File.list and File.listFiles | Major | datanode | Bharath Mundlapudi | Bharath Mundlapudi | +| [MAPREDUCE-2529](https://issues.apache.org/jira/browse/MAPREDUCE-2529) | Recognize Jetty bug 1342 and handle it | Major | tasktracker | Thomas Graves | Thomas Graves | +| [HADOOP-7364](https://issues.apache.org/jira/browse/HADOOP-7364) | TestMiniMRDFSCaching fails if test.build.dir is set to something other than build/test | Major | test | Thomas Graves | Thomas Graves | | [HDFS-1878](https://issues.apache.org/jira/browse/HDFS-1878) | TestHDFSServerPorts unit test failure - race condition in FSNamesystem.close() causes NullPointerException without serious consequence | Minor | namenode | Matt Foley | Matt Foley | -| [HDFS-1822](https://issues.apache.org/jira/browse/HDFS-1822) | Editlog opcodes overlap between 20 security and later releases | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | -| [HDFS-1758](https://issues.apache.org/jira/browse/HDFS-1758) | Web UI JSP pages thread safety issue | Minor | tools | Tanping Wang | Tanping Wang | -| [HDFS-1750](https://issues.apache.org/jira/browse/HDFS-1750) | fs -ls hftp://file not working | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2057](https://issues.apache.org/jira/browse/HDFS-2057) | Wait time to terminate the threads causing unit tests to take longer time | Major | datanode | Bharath Mundlapudi | Bharath Mundlapudi | +| [HADOOP-7373](https://issues.apache.org/jira/browse/HADOOP-7373) | Tarball deployment doesn't work with {start,stop}-{dfs,mapred} | Major | . | Owen O'Malley | Owen O'Malley | | [HDFS-1692](https://issues.apache.org/jira/browse/HDFS-1692) | In secure mode, Datanode process doesn't exit when disks fail. | Major | datanode | Bharath Mundlapudi | Bharath Mundlapudi | -| [HDFS-1592](https://issues.apache.org/jira/browse/HDFS-1592) | Datanode startup doesn't honor volumes.tolerated | Major | . | Bharath Mundlapudi | Bharath Mundlapudi | -| [HDFS-1377](https://issues.apache.org/jira/browse/HDFS-1377) | Quota bug for partial blocks allows quotas to be violated | Blocker | namenode | Eli Collins | Eli Collins | -| [HDFS-1258](https://issues.apache.org/jira/browse/HDFS-1258) | Clearing namespace quota on "/" corrupts FS image | Blocker | namenode | Aaron T. Myers | Aaron T. Myers | -| [HDFS-1189](https://issues.apache.org/jira/browse/HDFS-1189) | Quota counts missed between clear quota and set quota | Major | namenode | Kang Xiao | John George | +| [HADOOP-7324](https://issues.apache.org/jira/browse/HADOOP-7324) | Ganglia plugins for metrics v2 | Blocker | metrics | Luke Lu | Priyo Mustafi | +| [MAPREDUCE-2409](https://issues.apache.org/jira/browse/MAPREDUCE-2409) | Distributed Cache does not differentiate between file /archive for files with the same path | Major | distributed-cache | Siddharth Seth | Siddharth Seth | +| [HADOOP-7475](https://issues.apache.org/jira/browse/HADOOP-7475) | hadoop-setup-single-node.sh is broken | Blocker | . | Eric Yang | Eric Yang | +| [MAPREDUCE-2447](https://issues.apache.org/jira/browse/MAPREDUCE-2447) | Set JvmContext sooner for a task - MR2429 | Minor | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2621](https://issues.apache.org/jira/browse/MAPREDUCE-2621) | TestCapacityScheduler fails with "Queue "q1" does not exist" | Minor | . | Sherry Chen | Sherry Chen | +| [MAPREDUCE-2651](https://issues.apache.org/jira/browse/MAPREDUCE-2651) | Race condition in Linux Task Controller for job log directory creation | Major | task-controller | Bharath Mundlapudi | Bharath Mundlapudi | +| [HADOOP-7356](https://issues.apache.org/jira/browse/HADOOP-7356) | RPM packages broke bin/hadoop script for hadoop 0.20.205 | Blocker | . | Eric Yang | Eric Yang | | [MAPREDUCE-2846](https://issues.apache.org/jira/browse/MAPREDUCE-2846) | a small % of all tasks fail with DefaultTaskController | Blocker | task, task-controller, tasktracker | Allen Wittenauer | Owen O'Malley | | [MAPREDUCE-2804](https://issues.apache.org/jira/browse/MAPREDUCE-2804) | "Creation of symlink to attempt log dir failed." message is not useful | Blocker | . | Allen Wittenauer | Owen O'Malley | -| [MAPREDUCE-2651](https://issues.apache.org/jira/browse/MAPREDUCE-2651) | Race condition in Linux Task Controller for job log directory creation | Major | task-controller | Bharath Mundlapudi | Bharath Mundlapudi | -| [MAPREDUCE-2621](https://issues.apache.org/jira/browse/MAPREDUCE-2621) | TestCapacityScheduler fails with "Queue "q1" does not exist" | Minor | . | Sherry Chen | Sherry Chen | -| [MAPREDUCE-2555](https://issues.apache.org/jira/browse/MAPREDUCE-2555) | JvmInvalidate errors in the gridmix TT logs | Minor | tasktracker | Thomas Graves | Thomas Graves | -| [MAPREDUCE-2529](https://issues.apache.org/jira/browse/MAPREDUCE-2529) | Recognize Jetty bug 1342 and handle it | Major | tasktracker | Thomas Graves | Thomas Graves | -| [MAPREDUCE-2514](https://issues.apache.org/jira/browse/MAPREDUCE-2514) | ReinitTrackerAction class name misspelled RenitTrackerAction in task tracker log | Trivial | tasktracker | Jonathan Eagles | Jonathan Eagles | -| [MAPREDUCE-2451](https://issues.apache.org/jira/browse/MAPREDUCE-2451) | Log the reason string of healthcheck script | Trivial | jobtracker | Thomas Graves | Thomas Graves | -| [MAPREDUCE-2447](https://issues.apache.org/jira/browse/MAPREDUCE-2447) | Set JvmContext sooner for a task - MR2429 | Minor | . | Siddharth Seth | Siddharth Seth | -| [MAPREDUCE-2443](https://issues.apache.org/jira/browse/MAPREDUCE-2443) | Fix FI build - broken after MR-2429 | Minor | test | Siddharth Seth | Siddharth Seth | -| [MAPREDUCE-2429](https://issues.apache.org/jira/browse/MAPREDUCE-2429) | Check jvmid during task status report | Major | tasktracker | Arun C Murthy | Siddharth Seth | -| [MAPREDUCE-2418](https://issues.apache.org/jira/browse/MAPREDUCE-2418) | Errors not shown in the JobHistory servlet (specifically Counter Limit Exceeded) | Minor | . | Siddharth Seth | Siddharth Seth | -| [MAPREDUCE-2411](https://issues.apache.org/jira/browse/MAPREDUCE-2411) | When you submit a job to a queue with no ACLs you get an inscrutible NPE | Minor | . | Dick King | Dick King | -| [MAPREDUCE-2409](https://issues.apache.org/jira/browse/MAPREDUCE-2409) | Distributed Cache does not differentiate between file /archive for files with the same path | Major | distributed-cache | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2358](https://issues.apache.org/jira/browse/MAPREDUCE-2358) | MapReduce assumes HDFS as the default filesystem | Major | . | Owen O'Malley | Krishna Ramachandran | +| [MAPREDUCE-517](https://issues.apache.org/jira/browse/MAPREDUCE-517) | The capacity-scheduler should assign multiple tasks per heartbeat | Critical | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2357](https://issues.apache.org/jira/browse/MAPREDUCE-2357) | When extending inputsplit (non-FileSplit), all exceptions are ignored | Major | task | Owen O'Malley | Luke Lu | | [MAPREDUCE-2366](https://issues.apache.org/jira/browse/MAPREDUCE-2366) | TaskTracker can't retrieve stdout and stderr from web UI | Major | tasktracker | Owen O'Malley | Dick King | -| [MAPREDUCE-2364](https://issues.apache.org/jira/browse/MAPREDUCE-2364) | Shouldn't hold lock on rjob while localizing resources. | Major | tasktracker | Owen O'Malley | Devaraj Das | | [MAPREDUCE-2362](https://issues.apache.org/jira/browse/MAPREDUCE-2362) | Unit test failures: TestBadRecords and TestTaskTrackerMemoryManager | Major | test | Owen O'Malley | Greg Roelofs | +| [MAPREDUCE-2356](https://issues.apache.org/jira/browse/MAPREDUCE-2356) | A task succeeded even though there were errors on all attempts. | Major | . | Owen O'Malley | Luke Lu | +| [MAPREDUCE-2555](https://issues.apache.org/jira/browse/MAPREDUCE-2555) | JvmInvalidate errors in the gridmix TT logs | Minor | tasktracker | Thomas Graves | Thomas Graves | | [MAPREDUCE-2360](https://issues.apache.org/jira/browse/MAPREDUCE-2360) | Pig fails when using non-default FileSystem | Major | client | Owen O'Malley | | | [MAPREDUCE-2359](https://issues.apache.org/jira/browse/MAPREDUCE-2359) | Distributed cache doesn't use non-default FileSystems correctly | Major | . | Owen O'Malley | Krishna Ramachandran | -| [MAPREDUCE-2358](https://issues.apache.org/jira/browse/MAPREDUCE-2358) | MapReduce assumes HDFS as the default filesystem | Major | . | Owen O'Malley | Krishna Ramachandran | -| [MAPREDUCE-2357](https://issues.apache.org/jira/browse/MAPREDUCE-2357) | When extending inputsplit (non-FileSplit), all exceptions are ignored | Major | task | Owen O'Malley | Luke Lu | -| [MAPREDUCE-2356](https://issues.apache.org/jira/browse/MAPREDUCE-2356) | A task succeeded even though there were errors on all attempts. | Major | . | Owen O'Malley | Luke Lu | -| [MAPREDUCE-517](https://issues.apache.org/jira/browse/MAPREDUCE-517) | The capacity-scheduler should assign multiple tasks per heartbeat | Critical | . | Arun C Murthy | Arun C Murthy | -| [MAPREDUCE-118](https://issues.apache.org/jira/browse/MAPREDUCE-118) | Job.getJobID() will always return null | Blocker | client | Amar Kamat | Amareshwari Sriramadasu | +| [MAPREDUCE-2418](https://issues.apache.org/jira/browse/MAPREDUCE-2418) | Errors not shown in the JobHistory servlet (specifically Counter Limit Exceeded) | Minor | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2429](https://issues.apache.org/jira/browse/MAPREDUCE-2429) | Check jvmid during task status report | Major | tasktracker | Arun C Murthy | Siddharth Seth | +| [MAPREDUCE-2364](https://issues.apache.org/jira/browse/MAPREDUCE-2364) | Shouldn't hold lock on rjob while localizing resources. | Major | tasktracker | Owen O'Malley | Devaraj Das | ### TESTS: | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [HDFS-2218](https://issues.apache.org/jira/browse/HDFS-2218) | Disable TestHdfsProxy.testHdfsProxyInterface in 0.20-security and branch-1 until HDFS-2217 is fixed | Blocker | contrib/hdfsproxy, test | Matt Foley | Matt Foley | | [HDFS-2044](https://issues.apache.org/jira/browse/HDFS-2044) | TestQueueProcessingStatistics failing automatic test due to timing issues | Major | test | Matt Foley | Matt Foley | +| [HDFS-2218](https://issues.apache.org/jira/browse/HDFS-2218) | Disable TestHdfsProxy.testHdfsProxyInterface in 0.20-security and branch-1 until HDFS-2217 is fixed | Blocker | contrib/hdfsproxy, test | Matt Foley | Matt Foley | ### SUB-TASKS: | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [HDFS-1767](https://issues.apache.org/jira/browse/HDFS-1767) | Namenode should ignore non-initial block reports from datanodes when in safemode during startup | Major | datanode | Matt Foley | Matt Foley | -| [HDFS-1541](https://issues.apache.org/jira/browse/HDFS-1541) | Not marking datanodes dead When namenode in safemode | Major | namenode | Hairong Kuang | Hairong Kuang | | [HDFS-1445](https://issues.apache.org/jira/browse/HDFS-1445) | Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file | Major | datanode | Matt Foley | Matt Foley | -| [MAPREDUCE-2415](https://issues.apache.org/jira/browse/MAPREDUCE-2415) | Distribute TaskTracker userlogs onto multiple disks | Major | task-controller, tasktracker | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1541](https://issues.apache.org/jira/browse/HDFS-1541) | Not marking datanodes dead When namenode in safemode | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1767](https://issues.apache.org/jira/browse/HDFS-1767) | Namenode should ignore non-initial block reports from datanodes when in safemode during startup | Major | datanode | Matt Foley | Matt Foley | | [MAPREDUCE-2413](https://issues.apache.org/jira/browse/MAPREDUCE-2413) | TaskTracker should handle disk failures at both startup and runtime | Major | task-controller, tasktracker | Bharath Mundlapudi | Ravi Gummadi | - - -### OTHER: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-2415](https://issues.apache.org/jira/browse/MAPREDUCE-2415) | Distribute TaskTracker userlogs onto multiple disks | Major | task-controller, tasktracker | Bharath Mundlapudi | Bharath Mundlapudi | http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/RELEASENOTES.0.20.204.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/RELEASENOTES.0.20.204.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/RELEASENOTES.0.20.204.0.md index ae774bd..10d5ba6 100644 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/RELEASENOTES.0.20.204.0.md +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/RELEASENOTES.0.20.204.0.md @@ -23,37 +23,30 @@ These release notes cover new developer and user-facing incompatibilities, impor --- -* [HADOOP-6255](https://issues.apache.org/jira/browse/HADOOP-6255) | *Major* | **Create an rpm integration project** - -Added RPM/DEB packages to build system. - - ---- - -* [HDFS-2218](https://issues.apache.org/jira/browse/HDFS-2218) | *Blocker* | **Disable TestHdfsProxy.testHdfsProxyInterface in 0.20-security and branch-1 until HDFS-2217 is fixed** +* [HDFS-1445](https://issues.apache.org/jira/browse/HDFS-1445) | *Major* | **Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file** -Test case TestHdfsProxy.testHdfsProxyInterface has been temporarily disabled for this release, due to failure in the Hudson automated test environment. +Batch hardlinking during "upgrade" snapshots, cutting time from aprx 8 minutes per volume to aprx 8 seconds. Validated in both Linux and Windows. Depends on prior integration with patch for HADOOP-7133. --- -* [HDFS-1445](https://issues.apache.org/jira/browse/HDFS-1445) | *Major* | **Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file** +* [MAPREDUCE-2479](https://issues.apache.org/jira/browse/MAPREDUCE-2479) | *Major* | **Backport MAPREDUCE-1568 to hadoop security branch** -Batch hardlinking during "upgrade" snapshots, cutting time from aprx 8 minutes per volume to aprx 8 seconds. Validated in both Linux and Windows. Depends on prior integration with patch for HADOOP-7133. +Added mapreduce.tasktracker.distributedcache.checkperiod to the task tracker that defined the period to wait while cleaning up the distributed cache. The default is 1 min. --- -* [MAPREDUCE-2846](https://issues.apache.org/jira/browse/MAPREDUCE-2846) | *Blocker* | **a small % of all tasks fail with DefaultTaskController** +* [HADOOP-6255](https://issues.apache.org/jira/browse/HADOOP-6255) | *Major* | **Create an rpm integration project** -Fixed a race condition in writing the log index file that caused tasks to 'fail'. +Added RPM/DEB packages to build system. --- -* [MAPREDUCE-2804](https://issues.apache.org/jira/browse/MAPREDUCE-2804) | *Blocker* | **"Creation of symlink to attempt log dir failed." message is not useful** +* [MAPREDUCE-2524](https://issues.apache.org/jira/browse/MAPREDUCE-2524) | *Minor* | **Backport trunk heuristics for failing maps when we get fetch failures retrieving map output during shuffle** -Removed duplicate chmods of job log dir that were vulnerable to race conditions between tasks. Also improved the messages when the symlinks failed to be created. +Added a new configuration option: mapreduce.reduce.shuffle.maxfetchfailures, and removed a no longer used option: mapred.reduce.copy.backoff. --- @@ -68,16 +61,23 @@ mapreduce.reduce.shuffle.catch.exception.message.regex --- -* [MAPREDUCE-2524](https://issues.apache.org/jira/browse/MAPREDUCE-2524) | *Minor* | **Backport trunk heuristics for failing maps when we get fetch failures retrieving map output during shuffle** +* [HDFS-2218](https://issues.apache.org/jira/browse/HDFS-2218) | *Blocker* | **Disable TestHdfsProxy.testHdfsProxyInterface in 0.20-security and branch-1 until HDFS-2217 is fixed** -Added a new configuration option: mapreduce.reduce.shuffle.maxfetchfailures, and removed a no longer used option: mapred.reduce.copy.backoff. +Test case TestHdfsProxy.testHdfsProxyInterface has been temporarily disabled for this release, due to failure in the Hudson automated test environment. --- -* [MAPREDUCE-2479](https://issues.apache.org/jira/browse/MAPREDUCE-2479) | *Major* | **Backport MAPREDUCE-1568 to hadoop security branch** +* [MAPREDUCE-2846](https://issues.apache.org/jira/browse/MAPREDUCE-2846) | *Blocker* | **a small % of all tasks fail with DefaultTaskController** -Added mapreduce.tasktracker.distributedcache.checkperiod to the task tracker that defined the period to wait while cleaning up the distributed cache. The default is 1 min. +Fixed a race condition in writing the log index file that caused tasks to 'fail'. + + +--- + +* [MAPREDUCE-2804](https://issues.apache.org/jira/browse/MAPREDUCE-2804) | *Blocker* | **"Creation of symlink to attempt log dir failed." message is not useful** + +Removed duplicate chmods of job log dir that were vulnerable to race conditions between tasks. Also improved the messages when the symlinks failed to be created. http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/CHANGES.0.20.204.1.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/CHANGES.0.20.204.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/CHANGES.0.20.204.1.md deleted file mode 100644 index 56c3f85..0000000 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/CHANGES.0.20.204.1.md +++ /dev/null @@ -1,64 +0,0 @@ - - -# Apache Hadoop Changelog - -## Release 0.20.204.1 - Unreleased - -### INCOMPATIBLE CHANGES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### NEW FEATURES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### IMPROVEMENTS: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### BUG FIXES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### TESTS: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### SUB-TASKS: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### OTHER: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/RELEASENOTES.0.20.204.1.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/RELEASENOTES.0.20.204.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/RELEASENOTES.0.20.204.1.md deleted file mode 100644 index db70b82..0000000 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/RELEASENOTES.0.20.204.1.md +++ /dev/null @@ -1,24 +0,0 @@ - - -# Apache Hadoop 0.20.204.1 Release Notes - -These release notes cover new developer and user-facing incompatibilities, features, and major improvements. - - - --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org For additional commands, e-mail: common-commits-help@hadoop.apache.org