hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Haidinyak <phaidin...@local.com>
Subject RE: Truncate tables
Date Tue, 15 Feb 2011 17:35:28 GMT
Thanks, I added the new parameters and my client now runs as fast as the shell. This makes
it easier to debug import routines.

-Pete

-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Tuesday, February 15, 2011 7:10 AM
To: user@hbase.apache.org
Subject: Re: Truncate tables

What Andrey describes is how the shell makes itself more prompt
responding to changes.

See src/main/ruby/hbase/hbase.rb

        # Turn off retries in hbase and ipc.  Human doesn't want to
wait on N   retries.
        configuration.setInt("hbase.client.retries.number", 7)
        configuration.setInt("ipc.client.connect.max.retries", 3)

St.Ack


On Tue, Feb 15, 2011 at 3:36 AM, Andrey Stepachev <octo47@gmail.com> wrote:
> It is strage thing in hbase. Operations like create or drop are asyncronous,
> so immidiately after first rpc 'disable'
> hbase client try to check succesfull execution. Often it is not really
> complete yet, so hbase client pauses an
> amount of time configured in 'hbase.client.pause'. If you change it in you
> Configuration before connect to hbase to,
> for example 1000 you will super fast drop/disable/create cycle. But bevare,
> you should increment amount of tries,
> configured with 'hbase.client.retries.number', or you can fail, if real
> disable takes longer then 10 seconds (10 tryes by 1 second each).
>
> 2011/2/14 Peter Haidinyak <phaidinyak@local.com>
>
>> Each table is taking 30 seconds. Without looking at the HBase code I
>> assumed it was doing the same process as I am. Just quicker.
>>
>> Thanks
>>
>> -Pete
>>
>> -----Original Message-----
>> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of
>> Jean-Daniel Cryans
>> Sent: Monday, February 14, 2011 10:12 AM
>> To: user@hbase.apache.org
>> Subject: Re: Truncate tables
>>
>> Erm well could it be that your script is truncating a bunch of tables
>> whereas in the shell it only does one? Apart from that I can't think
>> of anything obvious, the truncate command really does the same thing
>> as you.
>>
>> J-D
>>
>> On Mon, Feb 14, 2011 at 9:43 AM, Peter Haidinyak <phaidinyak@local.com>
>> wrote:
>> > Hi,
>> >        I've setup a little utility to truncate my tables during
>> development but I found that if I truncate a table from the HBase shell it
>> takes 2.5 seconds, from my program it take 30 seconds. The code to truncate
>> is pretty simple. Any ideas on what I've done wrong?
>> >
>> > Thanks
>> >
>> > -Pete
>> >
>> >  private void truncateTables()
>> >    throws IOException
>> >  {
>> >    m_hbaseConnection = new HbaseConnection();
>> >    m_hbaseConnection.setupHBaseConnection();
>> >    final HBaseAdmin hBaseAdmin = m_hbaseConnection.getAdmin();
>> >
>> >    int counter = 1;
>> >    for (final String tableName : CLUSTER_TABLES)
>> >    {
>> >      final HTableDescriptor hTableDescriptor =
>> hBaseAdmin.getTableDescriptor(tableName.getBytes());
>> >      m_logger.info("Truncating Table {}.",tableName);
>> >      hBaseAdmin.disableTable(tableName);
>> >      hBaseAdmin.deleteTable(tableName);
>> >      hBaseAdmin.createTable(hTableDescriptor);
>> >      m_logger.info("Truncated Table {}.", tableName);
>> >      m_logger.info("Finished {} of {} Tables.", counter++,
>> CLUSTER_TABLES.length);
>> >
>> >    } // - End for each table.
>> >
>> >  } // - End truncateTables Method.
>> >
>> >
>> >  public void setupHBaseConnection()
>> >    throws IOException
>> >  {
>> >    m_hbaseConfiguration = HBaseConfiguration.create();
>> >    m_hbaseConfiguration.clear();
>> >
>> > // - Get the m_hbaseConfiguration properties from the Command line -D
>> parameters.
>> >    m_hbaseConfiguration.set("hbase.zookeeper.quorum",
>> System.getProperty("hbase.zookeeper.quorum"));
>> >    m_hbaseConfiguration.set("hbase.zookeeper.property.clientPort",
>> System.getProperty("hbase.zookeeper.property.clientPort"));
>> >
>> >    m_logger.info("Using hbase.zookeeper.quorum : {}",
>> m_hbaseConfiguration.get("hbase.zookeeper.quorum"));
>> >    m_logger.info("Using hbase.zookeeper.property.clientPort : {}",
>> m_hbaseConfiguration.get("hbase.zookeeper.property.clientPort"));
>> >
>> >  } // - End setupHBaseConnection Method.
>> >
>> >
>>
>

Mime
View raw message