hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: [jira] Commented: (HBASE-1557) incrementColumnValue edits are not persisted
Date Mon, 22 Jun 2009 04:31:34 GMT
no, you dont need to use the same jar with hadoop, because it has a small
limited but import client-side only bug fix.

I don't understand your 'cube' and 'ETL'... How to import/export data out of
hbase?  You can use map-reduces of course...


On Sun, Jun 21, 2009 at 9:02 PM, Irfan Mohammed <irfan.ma@gmail.com> wrote:

> If I use this hadoop-0.20.0-plus4681-core.jar within hbase, should my core
> hadoop also use the same jar?
>
> ----- Original Message -----
> From: "Irfan Mohammed" <irfan.ma@gmail.com>
> To: hbase-dev@hadoop.apache.org
> Sent: Sunday, June 21, 2009 11:56:53 PM GMT -05:00 US/Canada Eastern
> Subject: Re: [jira] Commented: (HBASE-1557) incrementColumnValue edits are
> not  persisted
>
> I am actually using hbase/lib/hadoop-0.20.0.jar from the hadoop-0.20.0
> download because I installed hadoop-0.20.0 and ran into some jar
> incompatibilities [don't remember exactly which one, sorry]. I had assumed [
> incorrectly in hindsight ] that it should not matter.
>
> [irfan@localhost lib]$ pwd
> /home/irfan/apps/hbase-latest/lib
> [irfan@localhost lib]$ ls -l hadoop-0.20.0-*
> -rw-r--r--  1 qwapi qwapi 2585066 Jun 20 18:01 hadoop-0.20.0-core.jar
> -rw-r--r--  1 qwapi qwapi 1440518 Jun 20 18:01 hadoop-0.20.0-test.jar
>
> Regarding stop/starting the server, the data is not going into the cubes [
> effectively I don't have my ETL's running ]. Is there a way to export/import
> data?
>
> ----- Original Message -----
> From: "Ryan Rawson" <ryanobjc@gmail.com>
> To: hbase-dev@hadoop.apache.org
> Sent: Sunday, June 21, 2009 11:42:06 PM GMT -05:00 US/Canada Eastern
> Subject: Re: [jira] Commented: (HBASE-1557) incrementColumnValue edits are
> not  persisted
>
> Can you verify what is in hbase/lib/hadoop*.jar ?  It should be the variant
> like so:
> hadoop-0.20.0-plus4681-core.jar
>
> I wouldn't stop the server until you can do your demo, since I'm not 100%
> sure if you are seeing flushes of your ICV data or not.  If this
> table/column family was only created with ICV, then you may be in good
> shape.
>
> Thanks for testing the alpha!
> -ryan
>
> On Sun, Jun 21, 2009 at 8:38 PM, Irfan Mohammed (JIRA) <jira@apache.org
> >wrote:
>
> >
> >    [
> >
> https://issues.apache.org/jira/browse/HBASE-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722470#action_12722470
> ]
> >
> > Irfan Mohammed commented on HBASE-1557:
> > ---------------------------------------
> >
> > here is the snip of the "hadoop dfs -lsr hdfs://app48:11000/hbase"
> >
> > drwxr-xr-x   - qwapi supergroup          0 2009-06-21 22:53
> > /hbase/transaction_impression/1684039776/gender
> > -rw-r--r--   1 qwapi supergroup     698515 2009-06-21 09:10
> > /hbase/transaction_impression/1684039776/gender/1775984732506941500
> > -rw-r--r--   1 qwapi supergroup     147422 2009-06-21 11:20
> > /hbase/transaction_impression/1684039776/gender/3121373883819729703
> > -rw-r--r--   1 qwapi supergroup     119241 2009-06-21 12:04
> > /hbase/transaction_impression/1684039776/gender/3146097270073712628
> > -rw-r--r--   1 qwapi supergroup     211137 2009-06-21 22:53
> > /hbase/transaction_impression/1684039776/gender/5773081374290155305
> > -rw-r--r--   1 qwapi supergroup     174108 2009-06-21 18:52
> > /hbase/transaction_impression/1684039776/gender/6345196659527815961
> > -rw-r--r--   1 qwapi supergroup     219050 2009-06-21 10:36
> > /hbase/transaction_impression/1684039776/gender/6999125414335797495
> > -rw-r--r--   1 qwapi supergroup    1005833 2009-06-21 03:35
> > /hbase/transaction_impression/1684039776/gender/7883801781648008956
> >
> >
> > > incrementColumnValue edits are not persisted
> > > --------------------------------------------
> > >
> > >                 Key: HBASE-1557
> > >                 URL: https://issues.apache.org/jira/browse/HBASE-1557
> > >             Project: Hadoop HBase
> > >          Issue Type: Bug
> > >            Reporter: Andrew Purtell
> > >             Fix For: 0.20.0, 0.19.4
> > >
> > >
> > > From Irfan up on hbase-users@
> > > I shutdown hbase using the following command
> > > {{$ HBASE_HOME/bin/stop-hbase.sh}}
> > > and it shuts down properly after a few seconds.
> > > here is my observation since last night. If I user TableRecordWriter1
> > uses HTable.incrementColumnValue, then the table data is not persisted
> > across restarts but if I use TableRecordWriter2 then the table data is
> > persisted across the restarts.
> > > Any clues ...
> > > Thanks,
> > > Irfan
> > > {code}
> > >     protected static class TableRecordWriter extends
> > RecordWriter<ImmutableBytesWritable, Put> {
> > >         ...
> > >         ...
> > >         ...
> > >         /**
> > >         * {@inheritDoc}
> > >         */
> > >         @Override
> > >         public void write(ImmutableBytesWritable key, Put put) throws
> > IOException
> > >         {
> > >             byte[] row = put.getRow();
> > >             for (Map.Entry<byte[], List<KeyValue>> familyEntry
:
> > put.getFamilyMap().entrySet()) {
> > >                 byte[] family = familyEntry.getKey();
> > >                 for (KeyValue keyValue : familyEntry.getValue()) {
> > >                     byte[] qualifier = keyValue.getQualifier();
> > >                     long amount = Bytes.toLong(keyValue.getValue());
> > >                     this.table_.incrementColumnValue(row, family,
> > qualifier, amount);
> > >                 }
> > >             }
> > >         }
> > >     }
> > > {code}
> > > ------
> > > {code}
> > >     protected static class TableRecordWriter2 extends
> > RecordWriter<ImmutableBytesWritable, Put> {
> > >         ...
> > >         ...
> > >         ...
> > >         /**
> > >         * {@inheritDoc}
> > >         */
> > >         public void write(ImmutableBytesWritable key, Put put) throws
> > IOException {
> > >             this.table_.put(new Put(put));
> > >         }
> > >     }
> > > {code}
> >
> > --
> > This message is automatically generated by JIRA.
> > -
> > You can reply to this email to add a comment to the issue online.
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message