hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leif Wickland <leifwickl...@gmail.com>
Subject Re: Is there a reason mapreduce.TableOutputFormat doesn't support Increment?
Date Tue, 21 Jun 2011 18:35:49 GMT
My patch to add support for Increment to TableOutputFormat follows.  (I did
the svn diff in trunk/src/main/java/org/apache/hadoop/hbase)

One point I was unsure about was whether I should duplicate the TimeRange in
the Increment's copy constructor.  TimeRange is immutable except for its
Writeable.readFields() implementation.

Index: mapreduce/TableOutputFormat.java
===================================================================
--- mapreduce/TableOutputFormat.java    (revision 1138076)
+++ mapreduce/TableOutputFormat.java    (working copy)
@@ -30,6 +30,7 @@
 import org.apache.hadoop.hbase.client.Delete;
 import org.apache.hadoop.hbase.client.HConnectionManager;
 import org.apache.hadoop.hbase.client.HTable;
+import org.apache.hadoop.hbase.client.Increment;
 import org.apache.hadoop.hbase.client.Put;
 import org.apache.hadoop.hbase.zookeeper.ZKUtil;
 import org.apache.hadoop.io.Writable;
@@ -41,8 +42,8 @@

 /**
  * Convert Map/Reduce output and write it to an HBase table. The KEY is
ignored
- * while the output value <u>must</u> be either a {@link Put} or a
- * {@link Delete} instance.
 + * while the output value <u>must</u> be a {@link Put},
+ * {@link Delete}, or {@link Increment} instance.
  *
  * @param <KEY>  The type of the key. Ignored in this class.
  */
@@ -119,8 +120,9 @@
     public void write(KEY key, Writable value)
     throws IOException {
       if (value instanceof Put) this.table.put(new Put((Put)value));
+      else if (value instanceof Increment) this.table.increment(new
Increment((Increment)value));
       else if (value instanceof Delete) this.table.delete(new
Delete((Delete)value));
-      else throw new IOException("Pass a Delete or a Put");
+      else throw new IOException("Pass a Delete, Increment or a Put");
     }
   }

Index: client/Increment.java
===================================================================
--- client/Increment.java       (revision 1138076)
+++ client/Increment.java       (working copy)
@@ -101,6 +101,19 @@
     return this;
   }

+  /**
+   * Copy constructor.  Creates an Increment operation cloned from the
specified
+   * Increment.
+   * @param incrementToCopy increment to copy
+   */
+  public Increment(final Increment incrementToCopy) {
+    this.row = incrementToCopy.getRow();
+    this.lockId = incrementToCopy.getLockId();
+    this.writeToWAL = incrementToCopy.getWriteToWAL();
+    this.tr = incrementToCopy.getTimeRange();
+    this.familyMap.putAll(incrementToCopy.getFamilyMap());
+  }
+
   /* Accessors */

   /**


On Fri, Jun 17, 2011 at 2:57 PM, Stack <stack@duboce.net> wrote:

> Go for it!
> St.Ack
>
> On Fri, Jun 17, 2011 at 1:43 PM, Leif Wickland <leifwickland@gmail.com>
> wrote:
> > I tried to use TableMapper and TableOutputFormat in
> > from org.apache.hadoop.hbase.mapreduce to write a map-reduce which
> > incremented some columns.  I noticed that TableOutputFormat.write()
> doesn't
> > support Increment, only Put and Delete.
> >
> > Is there a reason that TableOutputFormat shouldn't support increment?
> >
> > I think adding support for increment would only require adding a copy
> > constructor to Increment and a few lines to TableOutputFormat:  I'd be
> > willing to give writing the patch a try if there's no objection.
> >
> > Leif Wickland
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message