hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5438) Merge FileSystem.create and FileSystem.append
Date Wed, 25 Mar 2009 20:22:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689229#action_12689229
] 

Jakob Homan commented on HADOOP-5438:
-------------------------------------

The Java-centric way to do C-style bit packing is with EnumSets (http://www.jakobhoman.com/2008/08/javas-enumset-fun-for-whole-family.html),
which is future-proof but a bit verbose.
{code}
  enum CreateFlag {
    OVERWRITE ((short)0x01),
    APPEND    ((short)0x02);

    CreateFlag(short mode) { this.mode = mode; }
    CreateFlag() { this.mode = (short)0x01; }
    
    public short getMode() { return mode; }
    private short mode;

  }
  
  public static createFile(Path p, EnumSet<CreateFlag> flags) {
    if(flags.contains(CreateFlag.APPEND))
      // blah
 }
{code}
We ran into a similar issue with create flags on Zookeeper (ZOOKEEPER-132), and ended up using
them but had to enumerate all the possible values for creating actual integer flags.


> Merge FileSystem.create and FileSystem.append
> ---------------------------------------------
>
>                 Key: HADOOP-5438
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5438
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: He Yongqiang
>
> Currently, when a user wants to modify a file, the user first calls exists() to know
if this file is already there. And then uses create() or append() according to whether the
file exists or not.
> the code looks like:
> {code}
> FSDataOutputStream out_1 = null;
> if (fs.exists(path_1))
>    out_1 = fs.append(path_1);
> else
>    out_1 = fs.create(path_1);
> {code}
> . On the performace side,It involes two RPCs. On the easy-of-use side, it is not very
convient in contrast to the traditional open interface.
> It will more complicate if there is a overwrite parameter specified. I donot know whether
there is a bug about 'overwrite' in 0.19, some times it takes a long time for overwrite creates
to reture. So i make the write file code with overwrite param works like:
> {code}
> boolean exists = fs.exists(name);
> if (overwrite) {
>     if (exists)
>        fs.delete(name, true);
>      this.out = fs.create(name, overwrite, bufferSize, replication,
> 				    blockSize, progress);
>      this.currentRowID = 0;
>  } else {
>    if (!exists)
> 	this.out = fs.create(name, overwrite, bufferSize,
> 					replication, blockSize, progress);
>    else
> 	this.out = fs.append(name, bufferSize, progress);
> {code}
> Some code statements there are really redundant and not needed, especialy with the delete().
But without deleting first, the overwrite takes a long time to reture.
> BTW, i will create another issue about the overwrite problem. If it is not a bug at all
or a duplicate, someone please close it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message