hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Splits are not preserved during table copy using getAdmin().createTable(tableDescriptor);
Date Fri, 03 Oct 2014 16:23:01 GMT
You can simplify your code by utilizing the following from HTable:

  public byte [][] getStartKeys() throws IOException {

No need to sort the keys.

Cheers

On Fri, Oct 3, 2014 at 9:10 AM, Serega Sheypak <serega.sheypak@gmail.com>
wrote:

> Thanks, I'm already updated my end-to-end test, waiting for the result.
>
> Here is my code snippet, is it ok?
>
>  @SneakyThrows(IOException.class)
>     private byte[][] collectSplits(String tableName){
>         HTable table = new HTable(configuration, tableName);
>         int splitSize =
> table.getRegionLocations().descendingKeySet().size();
>
>         if(splitSize < 2){
>             LOG.info(String.format("There is only one region for a
> table[%s]. No splits to recreate", tableName));
>             return new byte[][]{};
>         }
>         LOG.info(String.format("There are [%s] regions for a table[%s].
> Skip pre-splitting", splitSize, tableName));
>
>         byte[][] splits = new byte[splitSize][];
>       *  Iterator<HRegionInfo> itr =
> table.getRegionLocations().descendingKeySet().iterator();*
> *        int index = 0;*
> *        while (itr.hasNext()){*
> *            splits[index] = itr.next().getStartKey();*
> *            index++;*
> *        }*
>
>         table.close();
>
>         Arrays.sort(splits, Bytes.BYTES_COMPARATOR);
>         return splits;
>     }
>
>
> 2014-10-03 20:02 GMT+04:00 Ted Yu <yuzhihong@gmail.com>:
>
> > Take a look at this method in HTable:
> >
> >   public Pair<byte[][],byte[][]> getStartEndKeys() throws IOException {
> >
> > You would see that the first dimension corresponds to the number of
> > regions.
> >
> >
> > Cheers
> >
> > On Fri, Oct 3, 2014 at 8:33 AM, Serega Sheypak <serega.sheypak@gmail.com
> >
> > wrote:
> >
> > > Ok, I found them:
> > > hTable.getRegionLocations().descendingKeySet()
> > > and
> > > HRegionInfo has startKey and endKey
> > > I have to prepare splits[][].
> > > I'm confused a little, how keys should be placed there? splits[][]
> shoud
> > by
> > > and array with length=region count and width = 2?
> > >
> > >
> > > 2014-10-03 17:57 GMT+04:00 Serega Sheypak <serega.sheypak@gmail.com>:
> > >
> > > > Hi, here is my code:
> > > >
> > > > public void dropIfExistsAndCreate(String sourceTableName, String
> > > > newTableName) throws IOException {
> > > >         LOG.info(String.format("Use [%s] to create [%s]",
> > > sourceTableName,
> > > > newTableName));
> > > >         HTableDescriptor descriptor = getDescriptor(sourceTableName);
> > > >         dropIfExists(newTableName); // doesn't matter
> > > >         createTable(descriptor, newTableName);
> > > >     }
> > > >
> > > >     private void createTable(HTableDescriptor descriptor, String
> > > > newTableName) throws IOException {
> > > >         descriptor.setName(Bytes.toBytes(newTableName));
> > > >         getAdmin().createTable(descriptor);
> > > >         LOG.info(String.format("Table created[%s]", newTableName));
> > > >     }
> > > >
> > > > My source table has 256 regions, newly created table has no splits.
> > > > Is there any possibility to copy splits from source table?
> > > >
> > > > I supposed, that HTableDescriptor encasulates splits, but it was bad
> > > > assumption.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message