Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Mon, 28 Nov 2011 23:33:40 +0000 (UTC)
From: "Jonathan Hsieh (Commented) (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: 
 <101015318.20219.1322523220231.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HADOOP-6840) Support non-recursive create() in
 FileSystem & SequenceFile.Writer
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158932#comment-13158932 ] 

Jonathan Hsieh commented on HADOOP-6840:
----------------------------------------

The calls to createNonRecursive(...) seems fishy -- the new boolean createParent variable from the signature isn't used at all...  Is this desired behavior?

{code}
+  public static Writer
+    createWriter(FileSystem fs, Configuration conf, Path name,
+                 Class keyClass, Class valClass, int bufferSize,
+                 short replication, long blockSize, boolean createParent,
+                 CompressionType compressionType, CompressionCodec codec,
+                 Metadata metadata) throws IOException {
+    if ((codec instanceof GzipCodec) &&
+        !NativeCodeLoader.isNativeCodeLoaded() &&
+        !ZlibFactory.isNativeZlibLoaded(conf)) {
+      throw new IllegalArgumentException("SequenceFile doesn't work with " +
+                                         "GzipCodec without native-hadoop code!");
+    }
+
+    switch (compressionType) {
+    case NONE:
+      return new Writer(conf, 
+          fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
+          keyClass, valClass, metadata).ownStream();
+    case RECORD:
+      return new RecordCompressWriter(conf, 
+          fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
+          keyClass, valClass, codec, metadata).ownStream();
+    case BLOCK:
+      return new BlockCompressWriter(conf,
+          fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
+          keyClass, valClass, codec, metadata).ownStream();
+    default:
+      return null;
+    }
+  } 
+  
{code}
                
> Support non-recursive create() in FileSystem & SequenceFile.Writer
> ------------------------------------------------------------------
>
>                 Key: HADOOP-6840
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6840
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs, io
>    Affects Versions: 0.20-append, 0.21.0
>            Reporter: Nicolas Spiegelberg
>            Assignee: Jitendra Nath Pandey
>            Priority: Minor
>             Fix For: 1.0.0
>
>         Attachments: HADOOP-6840-branch-0.20-security.patch, HADOOP-6840_0.20-append.patch, HADOOP-6840_0.21-2.patch, HADOOP-6840_0.21.patch
>
>
> The proposed solution for HBASE-2312 requires the sequence file to handle a non-recursive create.  This is already supported by HDFS, but needs to have an equivalent FileSystem & SequenceFile.Writer API.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira