hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Replacing a full Row content in HBase
Date Thu, 20 Nov 2014 17:05:46 GMT
bq. talk about the lack of transactions

As Anoop mentioned, mutateRow() provides transactional semantics for Benjamin's
use case.

Cheers

On Thu, Nov 20, 2014 at 8:55 AM, Michael Segel <michael_segel@hotmail.com>
wrote:

> Hi,
>
> Lets take a step back… OP’s initial goal is to replace all of the
> fields/cells on a row at the same time.
>
> Thought about doing a delete prior to the put().
>
> Is now a good time to remind people about what happens during a delete and
> how things can happen out of order?
>
> And should we talk about the lack of transactions and that RLL in HBase
> isn’t the same RLL in terms of a RDBMS?
>
> Theory before mechanics?  ;-)
>
> -Mike
>
>
> On Nov 20, 2014, at 3:17 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Sznajder:
> > You're using the following ctor:
> >
> >  public Delete(byte [] row) {
> >
> >    this(row, HConstants.LATEST_TIMESTAMP);
> >
> > You're letting server determine the actual timestamp (same for your
> Put's).
> >
> > As Anoop said:
> >
> > You should provide ts from client side.
> >
> > Cheers
> >
> > On Thu, Nov 20, 2014 at 6:47 AM, Sznajder ForMailingList <
> > bs4mailinglist@gmail.com> wrote:
> >
> >> Thanks for your answers!
> >>
> >> I added the setAutoFlush to be true....
> >>
> >> For "security", I also added table.flushCommits(); after every Put(),
> and
> >> Delete() call.
> >>
> >> And I still get some unpredictable results:
> >>
> >>
> >>     [java] Iteration 0
> >>     [java] Before put                                  *Empty result*
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put                            *Empty result*
> >>     [java]
> >>     [java]
> >>     [java] Iteration 1
> >>     [java] Before put                                  *Empty result*
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put                            *Empty result*
> >>     [java]
> >>     [java]
> >>     [java] Iteration 2
> >>     [java] Before put                                 * Empty result*
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put
> {colfam1,qual1,val}
> >>     [java]
> >>     [java]
> >>     [java] Iteration 3
> >>     [java] Before put
> {colfam1,qual1,val}
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put
> {colfam1,qual1,val}
> >>     [java]
> >>     [java]
> >>     [java] Iteration 4
> >>     [java] Before put
> {colfam1,qual1,val}
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put
> {colfam1,qual1,val}
> >>     [java]
> >>     [java]
> >>     [java] Iteration 5
> >>     [java] Before put
> {colfam1,qual1,val}
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put
> {colfam1,qual1,val}
> >>     [java]
> >>     [java]
> >>     [java] Iteration 6
> >>     [java] Before put
> {colfam1,qual1,val}
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put
> {colfam1,qual1,val}
> >>     [java]
> >>     [java]
> >>     [java] Iteration 7
> >>     [java] Before put
> {colfam1,qual1,val}
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put                            *Empty result*
> >>     [java]
> >>     [java]
> >>     [java] Iteration 8
> >>     [java] Before put                                  *Empty result*
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put
> {colfam1,qual1,val}
> >>     [java]
> >>     [java]
> >>     [java] Iteration 9
> >>     [java] Before put
> {colfam1,qual1,val}
> >>     [java] After put - before delete
> >> {colfam1,qual1,val}{colfam1,qual2,val}
> >>     [java] After second put                          *  Empty result*
> >>
> >>
> >> On Thu, Nov 20, 2014 at 4:32 PM, Nick Dimiduk <ndimiduk@gmail.com>
> wrote:
> >>
> >>> Hit send too soon.
> >>>
> >>> I don't see you calling flush anywhere. Could be the edits are
> >> accumulated
> >>> in the local client write buffer but haven't been sent to the store.
> The
> >>> exact semantics will depend on what version and settings you're
> running.
> >>>
> >>> Try adding a table.flush() before each call to printTableContent().
> >>>
> >>> On Thu, Nov 20, 2014 at 3:29 PM, Nick Dimiduk <ndimiduk@gmail.com>
> >> wrote:
> >>>
> >>>> Are you flushing the edits so that they're actually written to the
> >> server
> >>>> before you send the gets?
> >>>>
> >>>> On Thu, Nov 20, 2014 at 2:43 PM, Sznajder ForMailingList <
> >>>> bs4mailinglist@gmail.com> wrote:
> >>>>
> >>>>> Sure
> >>>>>
> >>>>> Here is the sample code I used for testing.
> >>>>>
> >>>>> The call Delete and then Put return some weird content : some times
> >> the
> >>>>> table is just... empty!
> >>>>>
> >>>>> Benjamin
> >>>>>
> >>>>>
> >>>>> package db.hbase;
> >>>>>
> >>>>> import java.io.IOException;
> >>>>> import java.util.List;
> >>>>>
> >>>>> import org.apache.hadoop.conf.Configuration;
> >>>>> import org.apache.hadoop.hbase.HBaseConfiguration;
> >>>>> import org.apache.hadoop.hbase.HColumnDescriptor;
> >>>>> import org.apache.hadoop.hbase.HTableDescriptor;
> >>>>> import org.apache.hadoop.hbase.KeyValue;
> >>>>> import org.apache.hadoop.hbase.TableExistsException;
> >>>>> import org.apache.hadoop.hbase.client.Delete;
> >>>>> import org.apache.hadoop.hbase.client.Get;
> >>>>> import org.apache.hadoop.hbase.client.HBaseAdmin;
> >>>>> import org.apache.hadoop.hbase.client.HTable;
> >>>>> import org.apache.hadoop.hbase.client.Put;
> >>>>> import org.apache.hadoop.hbase.client.Result;
> >>>>> import org.apache.hadoop.hbase.util.Bytes;
> >>>>>
> >>>>> public class HBaseStandaloneTest {
> >>>>>
> >>>>>    private final static String NAME =
> >>>>> "dummy-test-"+System.currentTimeMillis();
> >>>>>    private final static byte[] ROW1 = Bytes.toBytes("row1");
> >>>>>    private final static byte[] COLFAM1 = Bytes.toBytes("colfam1");
> >>>>>    private final static byte[] QUAL1 = Bytes.toBytes("qual1");
> >>>>>    private final static byte[] QUAL2 = Bytes.toBytes("qual2");
> >>>>>    private final static byte[] VAL = Bytes.toBytes("val");
> >>>>>
> >>>>>    private static final String HBASE_CONFIGURATION_ZOOKEEPER_QUORUM
=
> >>>>> "hbase.zookeeper.quorum";
> >>>>>    private static final String
> >>> HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT =
> >>>>> "hbase.zookeeper.property.clientPort";
> >>>>>
> >>>>>
> >>>>>    private static HTable table;
> >>>>>    private static HBaseAdmin admin;
> >>>>>
> >>>>>    private static void init() throws IOException {
> >>>>>        Configuration conf = HBaseConfiguration.create();
> >>>>>        conf.set(HBASE_CONFIGURATION_ZOOKEEPER_QUORUM, IP_ADRESS);
> >>>>>        conf.set(HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT, "2181");
> >>>>>
> >>>>>        admin = new HBaseAdmin(conf);
> >>>>>        try {
> >>>>>            if (admin.tableExists(NAME)) {
> >>>>>                admin.disableTable(NAME);
> >>>>>                admin.deleteTable(NAME);
> >>>>>            }
> >>>>>        } catch (Exception e ) {
> >>>>>            //
> >>>>>        }
> >>>>>        HTableDescriptor desc = new HTableDescriptor(NAME);
> >>>>>        HColumnDescriptor coldef = new
> >>>>> HColumnDescriptor(Bytes.toString(COLFAM1));
> >>>>>        desc.addFamily(coldef);
> >>>>>        try {
> >>>>>            admin.createTable(desc);
> >>>>>        } catch (TableExistsException e) {
> >>>>>            System.err.println("table \'" + NAME + "\' already
> >> exists");
> >>>>>        }
> >>>>>
> >>>>>        table = new HTable(conf, NAME);
> >>>>>    }
> >>>>>
> >>>>>    private static void testUpdateRows() throws Exception{
> >>>>>
> >>>>>        for (int i = 0 ; i < 10; i++) {
> >>>>>            System.out.println("\n\nIteration " + i);
> >>>>>            printTableContent("Before put");
> >>>>>            putRow1Version1();
> >>>>>            printTableContent("After put - before delete");
> >>>>>            deleteRow1();
> >>>>>            //printTableContent("After delete - before 2nd put");
> >>>>>            putRow1Version2();
> >>>>>            printTableContent("After second put");
> >>>>>        }
> >>>>>    }
> >>>>>
> >>>>>    private static void putRow1Version2() throws Exception {
> >>>>>        Put put = new Put(ROW1);
> >>>>>        put.add(COLFAM1, QUAL1, VAL);
> >>>>>
> >>>>>        table.put(put);
> >>>>>
> >>>>>    }
> >>>>>
> >>>>>    private static void deleteRow1() throws Exception {
> >>>>>        Delete del = new Delete(ROW1);
> >>>>>        table.delete(del);
> >>>>>    }
> >>>>>
> >>>>>    private static void putRow1Version1() throws Exception {
> >>>>>        Put put = new Put(ROW1);
> >>>>>        put.add(COLFAM1, QUAL1, VAL);
> >>>>>        put.add(COLFAM1, QUAL2, VAL);
> >>>>>
> >>>>>        table.put(put);
> >>>>>    }
> >>>>>
> >>>>>    private static void printTableContent(String str) throws
> >>> IOException {
> >>>>>        System.out.print((str+"
> >>>>> ").substring(0,30));
> >>>>>        Result res = table.get(new Get(ROW1));
> >>>>>        List<KeyValue> list = res.list();
> >>>>>        if (list == null) {
> >>>>>            System.out.println("\t\tEmpty result");
> >>>>>        } else {
> >>>>>            StringBuilder strBlder = new StringBuilder();
> >>>>>            for (KeyValue keyVal : list) {
> >>>>>                String fam = Bytes.toString(keyVal.getFamily());
> >>>>>                String qual = Bytes.toString(keyVal.getQualifier());
> >>>>>                String data =
> >>>>> Bytes.toString(res.getValue(keyVal.getFamily(),
> >> keyVal.getQualifier()));
> >>>>>
> >>>>>                strBlder.append("{"+ fam + "," + qual + "," + data
> >>> +"}");
> >>>>>            }
> >>>>>            System.out.println("\t\t"+strBlder.toString());
> >>>>>        }
> >>>>>    }
> >>>>>
> >>>>>    public static void main(String[] args) throws Exception {
> >>>>>        init();
> >>>>>        try {
> >>>>>            testUpdateRows();
> >>>>>        } finally {
> >>>>>            cleanup();
> >>>>>        }
> >>>>>    }
> >>>>>
> >>>>>    private static void cleanup() throws IOException {
> >>>>>        table.close();
> >>>>>        admin.disableTable(NAME);
> >>>>>        admin.deleteTable(NAME);
> >>>>>    }
> >>>>> }
> >>>>>
> >>>>>
> >>>>> On Thu, Nov 20, 2014 at 1:48 PM, Nick Dimiduk <ndimiduk@gmail.com>
> >>> wrote:
> >>>>>
> >>>>>> Attachements are filtered from the list. Please include a link
if
> >>> you'd
> >>>>>> like to share some attachment.
> >>>>>>
> >>>>>> On Thu, Nov 20, 2014 at 11:46 AM, Sznajder ForMailingList <
> >>>>>> bs4mailinglist@gmail.com> wrote:
> >>>>>>
> >>>>>>> Hi Nick
> >>>>>>>
> >>>>>>> Many thanks for your rapid answer!
> >>>>>>>
> >>>>>>> By "unpredictable results", I mean that I do not get the
expected
> >>>>> value
> >>>>>> in
> >>>>>>> the row, after the new Put call (after Delete).
> >>>>>>>
> >>>>>>> I tried to call a delete before a put  (see the attached
file)
> >>>>>>>
> >>>>>>> And I get some weird results: after iterations, I start
to get
> >> some
> >>>>>> "empty
> >>>>>>> content".
> >>>>>>>
> >>>>>>> Apparently, "Delete" and "Put" on a same key are not well
> >>>>> supported....
> >>>>>>>
> >>>>>>> Benjamin
> >>>>>>>
> >>>>>>> On Thu, Nov 20, 2014 at 11:40 AM, Nick Dimiduk <
> >> ndimiduk@gmail.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> What does "unpredictable results" mean? If you know
all the
> >>> existing
> >>>>>>>> qualifiers, just provide new values of all of them in
a single
> >> put.
> >>>>> If
> >>>>>> you
> >>>>>>>> don't, you can use a delete family marker to clear visibility
of
> >> an
> >>>>>> entire
> >>>>>>>> family. I think you'll need to do this separately from
writing
> >> the
> >>>>> new
> >>>>>>>> values.
> >>>>>>>>
> >>>>>>>> On Thu, Nov 20, 2014 at 9:17 AM, Sznajder ForMailingList
<
> >>>>>>>> bs4mailinglist@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> I would like to **replace** the whole content of
a Row in HBase
> >>> by
> >>>>>> some
> >>>>>>>> new
> >>>>>>>>> content.
> >>>>>>>>> However:
> >>>>>>>>> * Simple *put* call just replaces the cells in the
new put row.
> >>>>>>>>> * I thought to call Delete and then Put, and I get
some very
> >>>>>>>> unpredictable
> >>>>>>>>> results...
> >>>>>>>>>
> >>>>>>>>> Is there a solution for replacing the whole content
of a Row?
> >>>>>>>>>
> >>>>>>>>> Many thanks!
> >>>>>>>>>
> >>>>>>>>> Benjamin
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message