Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <C83BCDBE.94CB%vidhyash@yahoo-inc.com>
References: <C83BC4B6.94C5%vidhyash@yahoo-inc.com>
 <C83BCDBE.94CB%vidhyash@yahoo-inc.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Mon, 14 Jun 2010 12:32:37 -0700
Message-ID: <AANLkTimYw0fS5osyTPnQE7xnylD_2J-isd8TNPzh4v1_@mail.gmail.com>
Subject: Re: Bulk load problems..
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=000e0cd71f665454be0489028ec8

--000e0cd71f665454be0489028ec8
Content-Type: text/plain; charset=ISO-8859-1

On Mon, Jun 14, 2010 at 12:14 PM, Vidhyashankar Venkataraman <
vidhyash@yahoo-inc.com> wrote:

> >> Most likely you are not appending the correct metadata entries (in
> >> particular the log sequence ID)
> Since I am not creating any logs, the max log sequence ID should be -1,
> isnt it?
>
>
In trunk there's a feature whereby the metadata can include a special "this
is a bulk load" entry. In 0.20, you have to pick some sequence number - I'd
go with something like 0 for a bulk load. Check out what HFileOutputFormat
does and copy that :)

-Todd


>
> On 6/14/10 11:36 AM, "Vidhyashankar Venkataraman" <vidhyash@yahoo-inc.com>
> wrote:
>
> >> Most likely you are not appending the correct metadata entries (in
> >> particular the log sequence ID)
> Can you elaborate? What additional info do I need to add when I
> create/close Hfiles?
>
> Thank you
> vidhya
>
>
> On 6/14/10 11:22 AM, "Todd Lipcon" <todd@cloudera.com> wrote:
>
> On Mon, Jun 14, 2010 at 11:08 AM, Vidhyashankar Venkataraman <
> vidhyash@yahoo-inc.com> wrote:
>
> > I tried dumping my own Hfiles (similar to HFileOutputFormat: open an
> > Hfile.writer, append the key value pairs and then close the writer) and
> > tried loading them using the ruby script..  I had altered loadtable.rb to
> > modify the block size for the column family.
> >
> > The script reported no errors. But on checking with the web interface,
> >  (table's name is DocData)
> > http://b5120231.yst.yahoo.net:60010/table.jsp?name=DocData
> >
> > I got this:
> > org.apache.hadoop.hbase.client.NoServerForRegionException: No server
> > address listed in .META. for region DocData,,1276537980228
> >
> > And in each region server, I got this
> > 2010-06-14 17:57:48,846 ERROR
> > org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening
> > DocData,0000133832,1276537980224
> > java.lang.IllegalAccessError: Has not been initialized
> >        at
> >
> org.apache.hadoop.hbase.regionserver.StoreFile.getMaxSequenceId(StoreFile.java:216)
> >        at
> > org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:417)
> >        at
> org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:221)
> >        at
> >
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1549)
> >        at
> > org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:312)
> >        at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1564)
> >        at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1531)
> >        at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1451)
> >        at java.lang.Thread.run(Thread.java:619)
> >
> >
> > I am using Hbase 0.20.3... Have I made a mistake while dumping Hfiles?
> >
> >
> Most likely you are not appending the correct metadata entries (in
> particular the log sequence ID)
>
> -Todd
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
>
>


-- 
Todd Lipcon
Software Engineer, Cloudera

--000e0cd71f665454be0489028ec8--