hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olivier Smadja <osma...@gmail.com>
Subject Re: Appending to a file / updating a file
Date Thu, 28 May 2009 19:29:45 GMT
Thanks Damien.

And can i update a file with hadoop or just create it and read it later?

Olivier

On Thu, May 28, 2009 at 1:31 PM, Damien Cooke <Damien.Cooke@sun.com> wrote:

> Olivier,
> Append is not supported or recommended at this point.  You can turn it on
> via dfs.support.append in hdfs-site.xml under 0.20.0.  There have been some
> issues making it reliable.  If this is not production code or a production
> job then turning it on will probably have no detrimental effect, but be
> aware it might destroy your data as it is not recommended at this point.
>
> Regards
> Damien
>
> On 28/05/2009, at 6:46 AM, Olivier Smadja wrote:
>
>  Hello,
>>
>> I'm trying hadoop for the first time and I'm just trying to create a file
>> and append some text in it with the following code:
>>
>>
>> import java.io.IOException;
>>
>> import org.apache.hadoop.conf. Configuration;
>> import org.apache.hadoop.fs.FSDataOutputStream;
>> import org.apache.hadoop.fs.FileSystem;
>> import org.apache.hadoop.fs.Path;
>>
>> /**
>> * @author olivier
>> *
>> */
>> public class HadoopIO {
>>
>>   public static void main(String[] args) throws IOException {
>>
>>
>>       String directory = "/Users/olivier/tmp/hadoop-data";
>>       Configuration conf = new Configuration(true);
>>       Path path = new Path(directory);
>>       // Create the File system
>>       FileSystem fs = path.getFileSystem(conf);
>>       // Sets the working directory
>>       fs.setWorkingDirectory(path);
>>
>>       System.out.println(fs.getWorkingDirectory());
>>
>>       // Creates a files
>>       FSDataOutputStream out = fs.create(new Path("test.txt"));
>>       out.writeBytes("Testing hadoop - first line");
>>       out.close();
>>       // then try to append something
>>       out = fs.append(new Path("test.txt"));
>>       out.writeBytes("Testing hadoop - second line");
>>       out.close();
>>
>>       fs.close();
>>
>>
>>   }
>>
>> }
>>
>>
>> but I receive the following exception:
>>
>> Exception in thread "main" java.io.IOException: Not supported
>>   at
>>
>> org.apache.hadoop.fs.ChecksumFileSystem.append(ChecksumFileSystem.java:290)
>>   at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:525)
>>   at com.neodatis.odb.hadoop.HadoopIO.main(HadoopIO.java:38)
>>
>>
>> 1) Can someone tell me what am i doing wrong?
>>
>> 2) How can I update the file (for example, just update the first 10 bytes
>> of
>> the file)?
>>
>>
>> Thanks,
>> Olivier
>>
>
>
> Damien Cooke
> Open Scalable Solutions Performance
> Performance & Applications Engineering
>
> Sun Microsystems
> Level 2, East Wing 50 Grenfell Street, Adelaide
> SA 5000 Australia
> Phone x58315 (x7058315 US callers)
> Email Damien.Cooke@Sun.Com
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message