Return-Path: Delivered-To: apmail-lucene-hadoop-commits-archive@locus.apache.org Received: (qmail 54011 invoked from network); 7 Feb 2006 19:43:03 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 7 Feb 2006 19:43:03 -0000 Received: (qmail 47263 invoked by uid 500); 7 Feb 2006 19:43:02 -0000 Delivered-To: apmail-lucene-hadoop-commits-archive@lucene.apache.org Received: (qmail 47247 invoked by uid 500); 7 Feb 2006 19:43:02 -0000 Mailing-List: contact hadoop-commits-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-commits@lucene.apache.org Received: (qmail 47238 invoked by uid 99); 7 Feb 2006 19:43:02 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Feb 2006 11:43:02 -0800 X-ASF-Spam-Status: No, hits=-9.4 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [209.237.227.194] (HELO minotaur.apache.org) (209.237.227.194) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 07 Feb 2006 11:43:00 -0800 Received: (qmail 53917 invoked by uid 65534); 7 Feb 2006 19:42:40 -0000 Message-ID: <20060207194240.53914.qmail@minotaur.apache.org> Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r375671 - in /lucene/hadoop/trunk/src/java/org/apache/hadoop: dfs/ io/ Date: Tue, 07 Feb 2006 19:42:19 -0000 To: hadoop-commits@lucene.apache.org From: cutting@apache.org X-Mailer: svnmailer-1.0.6 X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Author: cutting Date: Tue Feb 7 11:42:16 2006 New Revision: 375671 URL: http://svn.apache.org/viewcvs?rev=375671&view=rev Log: Add WritableFactory mechanism, to permit one to use ObjectWritable with non-public classes. Register factories for DFS implementation classes and make them non-public. This greatly simplifies the javadoc. Added: lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactories.java lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactory.java Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/Block.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/BlockCommand.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/ClientProtocol.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFile.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFileInfo.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeInfo.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeProtocol.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSConstants.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDirectory.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSNamesystem.java lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/LocatedBlock.java lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ArrayWritable.java lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ObjectWritable.java Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/Block.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/Block.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/Block.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/Block.java Tue Feb 7 11:42:16 2006 @@ -26,7 +26,16 @@ * * @author Mike Cafarella **************************************************/ -public class Block implements Writable, Comparable { +class Block implements Writable, Comparable { + + static { // register a ctor + WritableFactories.setFactory + (Block.class, + new WritableFactory() { + public Writable newInstance() { return new Block(); } + }); + } + static Random r = new Random(); /** Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/BlockCommand.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/BlockCommand.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/BlockCommand.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/BlockCommand.java Tue Feb 7 11:42:16 2006 @@ -25,7 +25,16 @@ * * @author Michael Cafarella ****************************************************/ -public class BlockCommand implements Writable { +class BlockCommand implements Writable { + + static { // register a ctor + WritableFactories.setFactory + (BlockCommand.class, + new WritableFactory() { + public Writable newInstance() { return new BlockCommand(); } + }); + } + boolean transferBlocks = false; boolean invalidateBlocks = false; Block blocks[]; Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/ClientProtocol.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/ClientProtocol.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/ClientProtocol.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/ClientProtocol.java Tue Feb 7 11:42:16 2006 @@ -23,7 +23,7 @@ * * @author Mike Cafarella **********************************************************************/ -public interface ClientProtocol { +interface ClientProtocol { /** * Open an existing file. Get back block and datanode info Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java Tue Feb 7 11:42:16 2006 @@ -31,7 +31,7 @@ * Connects to a namenode daemon. * @author Mike Cafarella, Tessa MacDuff ********************************************************/ -public class DFSClient implements FSConstants { +class DFSClient implements FSConstants { public static final Logger LOG = LogFormatter.getLogger("org.apache.hadoop.fs.DFSClient"); static int MAX_BLOCK_ACQUIRE_FAILURES = 10; ClientProtocol namenode; Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFile.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFile.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFile.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFile.java Tue Feb 7 11:42:16 2006 @@ -24,7 +24,7 @@ * * @author Mike Cafarella *****************************************************************/ -public class DFSFile extends File { +class DFSFile extends File { DFSFileInfo info; /** Separator used in DFS filenames. */ Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFileInfo.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFileInfo.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFileInfo.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DFSFileInfo.java Tue Feb 7 11:42:16 2006 @@ -25,7 +25,15 @@ * * @author Mike Cafarella ******************************************************/ -public class DFSFileInfo implements Writable { +class DFSFileInfo implements Writable { + static { // register a ctor + WritableFactories.setFactory + (DFSFileInfo.class, + new WritableFactory() { + public Writable newInstance() { return new DFSFileInfo(); } + }); + } + UTF8 path; long len; long contentsLen; Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeInfo.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeInfo.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeInfo.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeInfo.java Tue Feb 7 11:42:16 2006 @@ -25,7 +25,16 @@ * * @author Mike Cafarella **************************************************/ -public class DatanodeInfo implements Writable, Comparable { +class DatanodeInfo implements Writable, Comparable { + + static { // register a ctor + WritableFactories.setFactory + (DatanodeInfo.class, + new WritableFactory() { + public Writable newInstance() { return new DatanodeInfo(); } + }); + } + private UTF8 name; private long capacityBytes, remainingBytes, lastUpdate; private volatile TreeSet blocks; Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeProtocol.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeProtocol.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeProtocol.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/DatanodeProtocol.java Tue Feb 7 11:42:16 2006 @@ -24,7 +24,7 @@ * * @author Michael Cafarella **********************************************************************/ -public interface DatanodeProtocol { +interface DatanodeProtocol { public void sendHeartbeat(String sender, long capacity, long remaining) throws IOException; public Block[] blockReport(String sender, Block blocks[]) throws IOException; Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSConstants.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSConstants.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSConstants.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSConstants.java Tue Feb 7 11:42:16 2006 @@ -22,7 +22,7 @@ * * @author Mike Cafarella ************************************/ -public interface FSConstants { +interface FSConstants { public static int BLOCK_SIZE = 32 * 1000 * 1000; //public static int BLOCK_SIZE = 19; Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java Tue Feb 7 11:42:16 2006 @@ -27,7 +27,7 @@ * * @author Mike Cafarella ***************************************************/ -public class FSDataset implements FSConstants { +class FSDataset implements FSConstants { static final double USABLE_DISK_PCT = 0.98; /** Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDirectory.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDirectory.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDirectory.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDirectory.java Tue Feb 7 11:42:16 2006 @@ -30,7 +30,7 @@ * * @author Mike Cafarella *************************************************/ -public class FSDirectory implements FSConstants { +class FSDirectory implements FSConstants { static String FS_IMAGE = "fsimage"; static String NEW_FS_IMAGE = "fsimage.new"; static String OLD_FS_IMAGE = "fsimage.old"; Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSNamesystem.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSNamesystem.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSNamesystem.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSNamesystem.java Tue Feb 7 11:42:16 2006 @@ -32,7 +32,7 @@ * 4) machine --> blocklist (inverted #2) * 5) LRU cache of updated-heartbeat machines ***************************************************/ -public class FSNamesystem implements FSConstants { +class FSNamesystem implements FSConstants { public static final Logger LOG = LogFormatter.getLogger("org.apache.hadoop.fs.FSNamesystem"); Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/LocatedBlock.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/LocatedBlock.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/LocatedBlock.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/LocatedBlock.java Tue Feb 7 11:42:16 2006 @@ -25,7 +25,16 @@ * * @author Michael Cafarella ****************************************************/ -public class LocatedBlock implements Writable { +class LocatedBlock implements Writable { + + static { // register a ctor + WritableFactories.setFactory + (LocatedBlock.class, + new WritableFactory() { + public Writable newInstance() { return new LocatedBlock(); } + }); + } + Block b; DatanodeInfo locs[]; Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ArrayWritable.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ArrayWritable.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ArrayWritable.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ArrayWritable.java Tue Feb 7 11:42:16 2006 @@ -78,14 +78,7 @@ public void readFields(DataInput in) throws IOException { values = new Writable[in.readInt()]; // construct values for (int i = 0; i < values.length; i++) { - Writable value; // construct value - try { - value = (Writable)valueClass.newInstance(); - } catch (InstantiationException e) { - throw new RuntimeException(e.toString()); - } catch (IllegalAccessException e) { - throw new RuntimeException(e.toString()); - } + Writable value = WritableFactories.newInstance(valueClass); value.readFields(in); // read a value values[i] = value; // store it in values } Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ObjectWritable.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ObjectWritable.java?rev=375671&r1=375670&r2=375671&view=diff ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ObjectWritable.java (original) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/io/ObjectWritable.java Tue Feb 7 11:42:16 2006 @@ -227,18 +227,12 @@ instance = UTF8.readString(in); } else { // Writable - try { - Writable writable = (Writable)declaredClass.newInstance(); - if(writable instanceof Configurable) { - ((Configurable) writable).setConf(conf); - } - writable.readFields(in); - instance = writable; - } catch (InstantiationException e) { - throw new RuntimeException(e); - } catch (IllegalAccessException e) { - throw new RuntimeException(e); + Writable writable = WritableFactories.newInstance(declaredClass); + if(writable instanceof Configurable) { + ((Configurable) writable).setConf(conf); } + writable.readFields(in); + instance = writable; } if (objectWritable != null) { // store values Added: lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactories.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactories.java?rev=375671&view=auto ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactories.java (added) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactories.java Tue Feb 7 11:42:16 2006 @@ -0,0 +1,55 @@ +/** + * Copyright 2005 The Apache Software Foundation + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.io; + +import java.util.HashMap; + +/** Factories for non-public writables. Defining a factory permits {@link + * ObjectWritable} to be able to construct instances of non-public classes. */ +public class WritableFactories { + private static final HashMap CLASS_TO_FACTORY = new HashMap(); + + private WritableFactories() {} // singleton + + /** Define a factory for a class. */ + public static synchronized void setFactory(Class c, WritableFactory factory) { + CLASS_TO_FACTORY.put(c, factory); + } + + /** Define a factory for a class. */ + public static synchronized WritableFactory getFactory(Class c) { + return (WritableFactory)CLASS_TO_FACTORY.get(c); + } + + /** Create a new instance of a class with a defined factory. */ + public static Writable newInstance(Class c) { + WritableFactory factory = WritableFactories.getFactory(c); + if (factory != null) { + return factory.newInstance(); + } else { + try { + return (Writable)c.newInstance(); + } catch (InstantiationException e) { + throw new RuntimeException(e); + } catch (IllegalAccessException e) { + throw new RuntimeException(e); + } + } + } + +} + Added: lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactory.java URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactory.java?rev=375671&view=auto ============================================================================== --- lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactory.java (added) +++ lucene/hadoop/trunk/src/java/org/apache/hadoop/io/WritableFactory.java Tue Feb 7 11:42:16 2006 @@ -0,0 +1,26 @@ +/** + * Copyright 2005 The Apache Software Foundation + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.io; + +/** A factory for a class of Writable. + * @see WritableFactories + */ +public interface WritableFactory { + /** Return a new instance. */ + Writable newInstance(); +} +