Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of theo.bertozzi@gmail.com
 designates 209.85.212.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALte62xVGbu0iueN2jG6asEJigWUSZC42o6t1r-_wK25NnxUjw@mail.gmail.com>
References: 
 <CACA1tWJSjXnH=xDryrdzTfzVg96364s90Eo1xpNaX3VP8Yyt+w@mail.gmail.com>
 <CACA1tWJ+kUX+YPTKAu_1ineuKkexQbEnxfjaTbZc5AKFT8+xcw@mail.gmail.com>
 <CACA1tWK6Mg5ZaMBZOBrW_JywpChxVPKxu_aoYcqhj3EwwaeXJA@mail.gmail.com>
 <CAFWiQHZ=X2VPm7Uu5k6C1-qoEcj9X63pnZ34v_j8+_qjX+X-EA@mail.gmail.com>
 <CALte62xVGbu0iueN2jG6asEJigWUSZC42o6t1r-_wK25NnxUjw@mail.gmail.com>
From: Matteo Bertozzi <theo.bertozzi@gmail.com>
Date: Tue, 29 Jul 2014 20:08:06 +0100
Message-ID: 
 <CAFb6HiKCmrj23OC2BauF4-0B29tpEtENYTOfVv=Me5GcdStdhw@mail.gmail.com>
Subject: Re: Completebulkload with namespace option?
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=f46d043891d9a4aa1f04ff59c390

--f46d043891d9a4aa1f04ff59c390
Content-Type: text/plain; charset=UTF-8

we don't need the name to be related to the table name.
We can replace the getUniqueName() using something like this

  String getUniqueName(TableName tableName) {
    String name = UUID.randomUUID().toString().replaceAll("-", "") +
                  "," + regionCount.incrementAndGet();
    return name;
  }

Matteo


On Tue, Jul 29, 2014 at 4:52 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> TableName.valueOf() would return an instance of TableName whose String form
> would contain ':'.
>
> FYI
>
>
> On Tue, Jul 29, 2014 at 8:46 AM, Bharath Vissapragada <
> bharathv@cloudera.com
> > wrote:
>
> > Appears to be a bug. It should be TableName.valueOf(...) or something
> > similar. Mind filing a jira?
> >
> >
> > On Tue, Jul 29, 2014 at 12:22 PM, Jianshi Huang <jianshi.huang@gmail.com
> >
> > wrote:
> >
> > > I see why, looking at the source code of LoadIncrementalHFiles.java, it
> > > seems the temporary path created for splitting will contain ':',
> > >
> > > The error part should be this:
> > > String uniqueName = getUniqueName(table.getName());
> > >     HColumnDescriptor familyDesc =
> > > table.getTableDescriptor().getFamily(item.family);
> > >     Path botOut = new Path(tmpDir, uniqueName + ".bottom");
> > >     Path topOut = new Path(tmpDir, uniqueName + ".top");
> > >     splitStoreFile(getConf(), hfilePath, familyDesc, splitKey,
> > >         botOut, topOut);
> > >
> > > uniqueName will be "namespce:table" so new Path will fail.
> > >
> > > A bug right?
> > >
> > > Jianshi
> > >
> > >
> > > On Tue, Jul 29, 2014 at 2:42 PM, Jianshi Huang <
> jianshi.huang@gmail.com>
> > > wrote:
> > >
> > > > I'm using hbase 0.98 with HDP 2.1.
> > > >
> > > >
> > > > On Tue, Jul 29, 2014 at 2:39 PM, Jianshi Huang <
> > jianshi.huang@gmail.com>
> > > > wrote:
> > > >
> > > >> I'm using completebulkload to load 500GB of data to a table
> > > >> (presplitted). However, it reports the following errors:
> > > >>
> > > >> Looks like completebulkload didn't recognize the namespace part
> > > >> (namespace:table).
> > > >>
> > > >> Is there an option to do it? I can't find one in Google...
> > > >>
> > > >> Exception in thread "main" 14/07/28 23:32:19 INFO
> > > >> mapreduce.LoadIncrementalHFiles: Trying to load
> > > >> hfile=hdfs://xxx/vertices/PROP/f5cbf0965ff44cb8bdabd038e66485c3
> > > >> first=dc595cfe#cust#1812199228741466242
> > > >>  last=dc68cedc#cust#2251647837553603393
> > > >> java.lang.reflect.InvocationTargetException
> > > >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > >>         at
> > > >>
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > >>         at
> > > >>
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > >>         at java.lang.reflect.Method.invoke(Method.java:606)
> > > >>         at
> > org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:54)
> > > >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > >>         at
> > > >>
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > >>         at
> > > >>
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > >>         at java.lang.reflect.Method.invoke(Method.java:606)
> > > >>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> > > >> Caused by: java.lang.IllegalStateException:
> > > >> java.lang.IllegalArgumentException: java.net.URISyntaxException:
> > > Relative
> > > >> path in absolute URI: grapple:vertices,37.bottom
> > > >>         at
> > > >>
> > >
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplitPhase(LoadIncrementalHFiles.java:421)
> > > >>         at
> > > >>
> > >
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:291)
> > > >>         at
> > > >>
> > >
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:825)
> > > >>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > > >>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> > > >>         at
> > > >>
> > >
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:831)
> > > >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > >>         at
> > > >>
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > >>         at
> > > >>
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > >>         at java.lang.reflect.Method.invoke(Method.java:606)
> > > >>         at
> > > >>
> > >
> >
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> > > >>         at
> > > >> org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
> > > >>         at
> > > >> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:153)
> > > >>         ... 10 more
> > > >> Caused by: java.lang.IllegalArgumentException:
> > > >> java.net.URISyntaxException: Relative path in absolute URI:
> > > >> grapple:vertices,37.bottom
> > > >>         at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> > > >>         at org.apache.hadoop.fs.Path.<init>(Path.java:172)
> > > >>         at org.apache.hadoop.fs.Path.<init>(Path.java:94)
> > > >>         at
> > > >>
> > >
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.splitStoreFile(LoadIncrementalHFiles.java:450)
> > > >>         at
> > > >>
> > >
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:516)
> > > >>         at
> > > >>
> > >
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:400)
> > > >>         at
> > > >>
> > >
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:398)
> > > >>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> > > >>         at
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > >>         at
> > > >>
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > >>         at java.lang.Thread.run(Thread.java:724)
> > > >> Caused by: java.net.URISyntaxException: Relative path in absolute
> URI:
> > > >> grapple:vertices,37.bottom
> > > >>         at java.net.URI.checkPath(URI.java:1804)
> > > >>         at java.net.URI.<init>(URI.java:752)
> > > >>         at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> > > >>         ... 10 more
> > > >>
> > > >> --
> > > >> Jianshi Huang
> > > >>
> > > >> LinkedIn: jianshi
> > > >> Twitter: @jshuang
> > > >> Github & Blog: http://huangjs.github.com/
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Jianshi Huang
> > > >
> > > > LinkedIn: jianshi
> > > > Twitter: @jshuang
> > > > Github & Blog: http://huangjs.github.com/
> > > >
> > >
> > >
> > >
> > > --
> > > Jianshi Huang
> > >
> > > LinkedIn: jianshi
> > > Twitter: @jshuang
> > > Github & Blog: http://huangjs.github.com/
> > >
> >
> >
> >
> > --
> > Bharath Vissapragada
> > <http://www.cloudera.com>
> >
>

--f46d043891d9a4aa1f04ff59c390--