hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-538) Implement a nio's 'direct buffer' based wrapper over zlib to improve performance of java.util.zip.{De|In}flater as a 'custom codec'
Date Fri, 10 Nov 2006 20:03:40 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12448840 ] 
Doug Cutting commented on HADOOP-538:

This is looking really close.  A few comments:

1. There's some debug output in bin/hadoop, an 'echo'.

2. The patch should not include binaries: these should be built somehow by build.xml.

3. I don't like custom/prebuilt distinction.  How about we just put prebuilt stuff in lib/,
and custom be in build/lib/?  A deploy-native target can copy things from build/lib/ to lib/,
and, if that part of lib/ is stored in subversion, these files will be committed.  The "package"
target will always place whatever's in build/lib into the lib/ directory of the packaged code.

4. We should define a "compile.native" property in build.xml that determines whether the native
code is compiled, tested and packaged.  By default this will be unset, although we may want
to set it for releases and nightly builds.

The only reason to check any binaries into subversion is for folks who: (1) build from trunk
rather than using a release; and (2) don't have the C compliation environment installed.

> Implement a nio's 'direct buffer' based wrapper over zlib to improve performance of java.util.zip.{De|In}flater
as a 'custom codec'
> -----------------------------------------------------------------------------------------------------------------------------------
>                 Key: HADOOP-538
>                 URL: http://issues.apache.org/jira/browse/HADOOP-538
>             Project: Hadoop
>          Issue Type: Improvement
>    Affects Versions: 0.6.1
>            Reporter: Arun C Murthy
>         Assigned To: Arun C Murthy
>             Fix For: 0.9.0
>         Attachments: HADOOP-538.patch, HADOOP-538_20061005.tgz, HADOOP-538_20061011.tgz,
HADOOP-538_20061026.tgz, HADOOP-538_20061030.tgz, HADOOP-538_20061107.tgz, HADOOP-538_20061108.tgz,
> There has been more than one instance where java.util.zip's {De|In}flater classes perform
unreliably, a simple wrapper over zlib-1.2.3 (latest stable) using java.nio.ByteBuffer (i.e.
direct buffers) should go a long way in alleviating these woes.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message