hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: Replacing a full Row content in HBase
Date Thu, 20 Nov 2014 16:55:13 GMT
Hi, 

Lets take a step back… OP’s initial goal is to replace all of the fields/cells on a row
at the same time. 

Thought about doing a delete prior to the put().

Is now a good time to remind people about what happens during a delete and how things can
happen out of order? 

And should we talk about the lack of transactions and that RLL in HBase isn’t the same RLL
in terms of a RDBMS? 

Theory before mechanics?  ;-)

-Mike


On Nov 20, 2014, at 3:17 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Sznajder:
> You're using the following ctor:
> 
>  public Delete(byte [] row) {
> 
>    this(row, HConstants.LATEST_TIMESTAMP);
> 
> You're letting server determine the actual timestamp (same for your Put's).
> 
> As Anoop said:
> 
> You should provide ts from client side.
> 
> Cheers
> 
> On Thu, Nov 20, 2014 at 6:47 AM, Sznajder ForMailingList <
> bs4mailinglist@gmail.com> wrote:
> 
>> Thanks for your answers!
>> 
>> I added the setAutoFlush to be true....
>> 
>> For "security", I also added table.flushCommits(); after every Put(), and
>> Delete() call.
>> 
>> And I still get some unpredictable results:
>> 
>> 
>>     [java] Iteration 0
>>     [java] Before put                                  *Empty result*
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                            *Empty result*
>>     [java]
>>     [java]
>>     [java] Iteration 1
>>     [java] Before put                                  *Empty result*
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                            *Empty result*
>>     [java]
>>     [java]
>>     [java] Iteration 2
>>     [java] Before put                                 * Empty result*
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                            {colfam1,qual1,val}
>>     [java]
>>     [java]
>>     [java] Iteration 3
>>     [java] Before put                                  {colfam1,qual1,val}
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                            {colfam1,qual1,val}
>>     [java]
>>     [java]
>>     [java] Iteration 4
>>     [java] Before put                                  {colfam1,qual1,val}
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                            {colfam1,qual1,val}
>>     [java]
>>     [java]
>>     [java] Iteration 5
>>     [java] Before put                                  {colfam1,qual1,val}
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                            {colfam1,qual1,val}
>>     [java]
>>     [java]
>>     [java] Iteration 6
>>     [java] Before put                                  {colfam1,qual1,val}
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                            {colfam1,qual1,val}
>>     [java]
>>     [java]
>>     [java] Iteration 7
>>     [java] Before put                                  {colfam1,qual1,val}
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                            *Empty result*
>>     [java]
>>     [java]
>>     [java] Iteration 8
>>     [java] Before put                                  *Empty result*
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                            {colfam1,qual1,val}
>>     [java]
>>     [java]
>>     [java] Iteration 9
>>     [java] Before put                                  {colfam1,qual1,val}
>>     [java] After put - before delete
>> {colfam1,qual1,val}{colfam1,qual2,val}
>>     [java] After second put                          *  Empty result*
>> 
>> 
>> On Thu, Nov 20, 2014 at 4:32 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
>> 
>>> Hit send too soon.
>>> 
>>> I don't see you calling flush anywhere. Could be the edits are
>> accumulated
>>> in the local client write buffer but haven't been sent to the store. The
>>> exact semantics will depend on what version and settings you're running.
>>> 
>>> Try adding a table.flush() before each call to printTableContent().
>>> 
>>> On Thu, Nov 20, 2014 at 3:29 PM, Nick Dimiduk <ndimiduk@gmail.com>
>> wrote:
>>> 
>>>> Are you flushing the edits so that they're actually written to the
>> server
>>>> before you send the gets?
>>>> 
>>>> On Thu, Nov 20, 2014 at 2:43 PM, Sznajder ForMailingList <
>>>> bs4mailinglist@gmail.com> wrote:
>>>> 
>>>>> Sure
>>>>> 
>>>>> Here is the sample code I used for testing.
>>>>> 
>>>>> The call Delete and then Put return some weird content : some times
>> the
>>>>> table is just... empty!
>>>>> 
>>>>> Benjamin
>>>>> 
>>>>> 
>>>>> package db.hbase;
>>>>> 
>>>>> import java.io.IOException;
>>>>> import java.util.List;
>>>>> 
>>>>> import org.apache.hadoop.conf.Configuration;
>>>>> import org.apache.hadoop.hbase.HBaseConfiguration;
>>>>> import org.apache.hadoop.hbase.HColumnDescriptor;
>>>>> import org.apache.hadoop.hbase.HTableDescriptor;
>>>>> import org.apache.hadoop.hbase.KeyValue;
>>>>> import org.apache.hadoop.hbase.TableExistsException;
>>>>> import org.apache.hadoop.hbase.client.Delete;
>>>>> import org.apache.hadoop.hbase.client.Get;
>>>>> import org.apache.hadoop.hbase.client.HBaseAdmin;
>>>>> import org.apache.hadoop.hbase.client.HTable;
>>>>> import org.apache.hadoop.hbase.client.Put;
>>>>> import org.apache.hadoop.hbase.client.Result;
>>>>> import org.apache.hadoop.hbase.util.Bytes;
>>>>> 
>>>>> public class HBaseStandaloneTest {
>>>>> 
>>>>>    private final static String NAME =
>>>>> "dummy-test-"+System.currentTimeMillis();
>>>>>    private final static byte[] ROW1 = Bytes.toBytes("row1");
>>>>>    private final static byte[] COLFAM1 = Bytes.toBytes("colfam1");
>>>>>    private final static byte[] QUAL1 = Bytes.toBytes("qual1");
>>>>>    private final static byte[] QUAL2 = Bytes.toBytes("qual2");
>>>>>    private final static byte[] VAL = Bytes.toBytes("val");
>>>>> 
>>>>>    private static final String HBASE_CONFIGURATION_ZOOKEEPER_QUORUM =
>>>>> "hbase.zookeeper.quorum";
>>>>>    private static final String
>>> HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT =
>>>>> "hbase.zookeeper.property.clientPort";
>>>>> 
>>>>> 
>>>>>    private static HTable table;
>>>>>    private static HBaseAdmin admin;
>>>>> 
>>>>>    private static void init() throws IOException {
>>>>>        Configuration conf = HBaseConfiguration.create();
>>>>>        conf.set(HBASE_CONFIGURATION_ZOOKEEPER_QUORUM, IP_ADRESS);
>>>>>        conf.set(HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT, "2181");
>>>>> 
>>>>>        admin = new HBaseAdmin(conf);
>>>>>        try {
>>>>>            if (admin.tableExists(NAME)) {
>>>>>                admin.disableTable(NAME);
>>>>>                admin.deleteTable(NAME);
>>>>>            }
>>>>>        } catch (Exception e ) {
>>>>>            //
>>>>>        }
>>>>>        HTableDescriptor desc = new HTableDescriptor(NAME);
>>>>>        HColumnDescriptor coldef = new
>>>>> HColumnDescriptor(Bytes.toString(COLFAM1));
>>>>>        desc.addFamily(coldef);
>>>>>        try {
>>>>>            admin.createTable(desc);
>>>>>        } catch (TableExistsException e) {
>>>>>            System.err.println("table \'" + NAME + "\' already
>> exists");
>>>>>        }
>>>>> 
>>>>>        table = new HTable(conf, NAME);
>>>>>    }
>>>>> 
>>>>>    private static void testUpdateRows() throws Exception{
>>>>> 
>>>>>        for (int i = 0 ; i < 10; i++) {
>>>>>            System.out.println("\n\nIteration " + i);
>>>>>            printTableContent("Before put");
>>>>>            putRow1Version1();
>>>>>            printTableContent("After put - before delete");
>>>>>            deleteRow1();
>>>>>            //printTableContent("After delete - before 2nd put");
>>>>>            putRow1Version2();
>>>>>            printTableContent("After second put");
>>>>>        }
>>>>>    }
>>>>> 
>>>>>    private static void putRow1Version2() throws Exception {
>>>>>        Put put = new Put(ROW1);
>>>>>        put.add(COLFAM1, QUAL1, VAL);
>>>>> 
>>>>>        table.put(put);
>>>>> 
>>>>>    }
>>>>> 
>>>>>    private static void deleteRow1() throws Exception {
>>>>>        Delete del = new Delete(ROW1);
>>>>>        table.delete(del);
>>>>>    }
>>>>> 
>>>>>    private static void putRow1Version1() throws Exception {
>>>>>        Put put = new Put(ROW1);
>>>>>        put.add(COLFAM1, QUAL1, VAL);
>>>>>        put.add(COLFAM1, QUAL2, VAL);
>>>>> 
>>>>>        table.put(put);
>>>>>    }
>>>>> 
>>>>>    private static void printTableContent(String str) throws
>>> IOException {
>>>>>        System.out.print((str+"
>>>>> ").substring(0,30));
>>>>>        Result res = table.get(new Get(ROW1));
>>>>>        List<KeyValue> list = res.list();
>>>>>        if (list == null) {
>>>>>            System.out.println("\t\tEmpty result");
>>>>>        } else {
>>>>>            StringBuilder strBlder = new StringBuilder();
>>>>>            for (KeyValue keyVal : list) {
>>>>>                String fam = Bytes.toString(keyVal.getFamily());
>>>>>                String qual = Bytes.toString(keyVal.getQualifier());
>>>>>                String data =
>>>>> Bytes.toString(res.getValue(keyVal.getFamily(),
>> keyVal.getQualifier()));
>>>>> 
>>>>>                strBlder.append("{"+ fam + "," + qual + "," + data
>>> +"}");
>>>>>            }
>>>>>            System.out.println("\t\t"+strBlder.toString());
>>>>>        }
>>>>>    }
>>>>> 
>>>>>    public static void main(String[] args) throws Exception {
>>>>>        init();
>>>>>        try {
>>>>>            testUpdateRows();
>>>>>        } finally {
>>>>>            cleanup();
>>>>>        }
>>>>>    }
>>>>> 
>>>>>    private static void cleanup() throws IOException {
>>>>>        table.close();
>>>>>        admin.disableTable(NAME);
>>>>>        admin.deleteTable(NAME);
>>>>>    }
>>>>> }
>>>>> 
>>>>> 
>>>>> On Thu, Nov 20, 2014 at 1:48 PM, Nick Dimiduk <ndimiduk@gmail.com>
>>> wrote:
>>>>> 
>>>>>> Attachements are filtered from the list. Please include a link if
>>> you'd
>>>>>> like to share some attachment.
>>>>>> 
>>>>>> On Thu, Nov 20, 2014 at 11:46 AM, Sznajder ForMailingList <
>>>>>> bs4mailinglist@gmail.com> wrote:
>>>>>> 
>>>>>>> Hi Nick
>>>>>>> 
>>>>>>> Many thanks for your rapid answer!
>>>>>>> 
>>>>>>> By "unpredictable results", I mean that I do not get the expected
>>>>> value
>>>>>> in
>>>>>>> the row, after the new Put call (after Delete).
>>>>>>> 
>>>>>>> I tried to call a delete before a put  (see the attached file)
>>>>>>> 
>>>>>>> And I get some weird results: after iterations, I start to get
>> some
>>>>>> "empty
>>>>>>> content".
>>>>>>> 
>>>>>>> Apparently, "Delete" and "Put" on a same key are not well
>>>>> supported....
>>>>>>> 
>>>>>>> Benjamin
>>>>>>> 
>>>>>>> On Thu, Nov 20, 2014 at 11:40 AM, Nick Dimiduk <
>> ndimiduk@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>>> What does "unpredictable results" mean? If you know all the
>>> existing
>>>>>>>> qualifiers, just provide new values of all of them in a single
>> put.
>>>>> If
>>>>>> you
>>>>>>>> don't, you can use a delete family marker to clear visibility
of
>> an
>>>>>> entire
>>>>>>>> family. I think you'll need to do this separately from writing
>> the
>>>>> new
>>>>>>>> values.
>>>>>>>> 
>>>>>>>> On Thu, Nov 20, 2014 at 9:17 AM, Sznajder ForMailingList
<
>>>>>>>> bs4mailinglist@gmail.com> wrote:
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> I would like to **replace** the whole content of a Row
in HBase
>>> by
>>>>>> some
>>>>>>>> new
>>>>>>>>> content.
>>>>>>>>> However:
>>>>>>>>> * Simple *put* call just replaces the cells in the new
put row.
>>>>>>>>> * I thought to call Delete and then Put, and I get some
very
>>>>>>>> unpredictable
>>>>>>>>> results...
>>>>>>>>> 
>>>>>>>>> Is there a solution for replacing the whole content of
a Row?
>>>>>>>>> 
>>>>>>>>> Many thanks!
>>>>>>>>> 
>>>>>>>>> Benjamin
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>> 


Mime
View raw message