hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liu, Ming (HPIT-GADSC)" <ming.l...@hp.com>
Subject What is proper way to make a hbase connection? using HTable (conf,tbl) or createConnection? Zookeeper session run out.
Date Mon, 24 Nov 2014 08:33:17 GMT
Hello,

I am using HBase 0.98.5. In example hbase client programs, some use createConnection() and
some use HTable() directly, I found they behave different, I wrote two simple tests program
using these two different methods, each program will start two threads and do simple put.
 and I found:
One program will start only 1 zookeeper sessions shared in two threads while another will
start 2 zookeeper sessions. So I don't know why the program using createConnection  will do
more zookeeper requests than simply use HTable. Is it possible to use createConnection in
two threads but share the same zookeeper session?

Here is details:
Demo1 will make two zookeeper sessions, seems two HBase connections; but Demo2 will only make
one zookeeper session. My real program is using createConnection in multiple threads as in
demo1, since I have a very small zookeeper , it only allows 60 concurrent sessions, so my
program always fail when there are hundreds of threads started. But I saw if using HTable
directly, it will only consume 1 zookeeper session. But it will change a lot in my current
program, so I wish there is a way to use createConnection and behave same as using HTable,
is it possible?

Source code:

Demo1.java
class ClientThread extends Thread
{
        public static Configuration configuration;
        static {
                configuration = HBaseConfiguration.create();
        }
        public void run()
        {
        try {
                System.out.println("start insert data ......");
                HConnection con=HConnectionManager.createConnection(configuration);
                HTable table = (HTable)con.getTable("hbase_table1");
                Put put = new Put("1".getBytes());
                put.add("c1".getBytes(), null, "baidu".getBytes());
                put.add("c2".getBytes(), null, "http://www.baidu.com1".getBytes());
                try {
                        table.put(put);
                } catch (IOException e) {
                        e.printStackTrace();
                }
                System.out.println("end insert data ......");
        }
        catch  (Exception e) {
       }

        }
}
public class demo1 {

        public static void main(String[] args) throws Exception {
                Thread t1=new ClientThread();
                Thread t2=new ClientThread();
                t1.start();
                t2.start();
        }

}


Demo2.java
class ClientThread1 extends Thread
{
        public static Configuration configuration;
        static {
                configuration = HBaseConfiguration.create();
        }
        public void run()
        {
                System.out.println("start insert data ......");
                try {
                        HTableInterface table = new HTable(configuration, "hbase_table1");
                        Put put = new Put("1".getBytes());
                        put.add("c1".getBytes(), null, "baidu".getBytes());
                        put.add("c2".getBytes(), null, "http://www.baidu.com1".getBytes());
                        table.put(put);
                } catch (Exception e) {
                        e.printStackTrace();
                }
                System.out.println("end insert data ......");

        }


}
public class demo2 {

        public static void main(String[] args) throws Exception {
                Thread t1=new ClientThread1();
                Thread t2=new ClientThread1();
                t1.start();
                t2.start();
        }
}



This should be a very basic question, sorry, I really did some search but cannot find any
good explaination. Any help will be very appreciated.

Thanks,
Ming

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message