hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jmozah <jmo...@gmail.com>
Subject Re: Hbase- Hadoop DFS size not decreased even after deleting a column family
Date Sat, 18 Aug 2012 16:19:27 GMT
Hi

Submitted a patch for this.
Should i call for a review for this?

i See lot of other testcases are missing for cases like this in handlers...
i think a Jira would be good for this to track.

./Zahoor
HBase Musings


On 13-Aug-2012, at 11:13 AM, J Mohamed Zahoor <jmozah@gmail.com> wrote:

> HBASE-6564.
> 
> I will try to take a stab on it this weekend.
> 
> ./zahoor
> 
> On Fri, Aug 10, 2012 at 12:47 PM, J Mohamed Zahoor <jmozah@gmail.com> wrote:
> Hi Lars,
> 
> Will file it... 
> ./Zahoor
> 
> 
> On Fri, Aug 10, 2012 at 12:00 AM, lars hofhansl <lhofhansl@yahoo.com> wrote:
> Hi zahoor,
> 
> could you file a jira with what you found? It looks like a bug.
> Thanks.
> 
> 
> -- Lars
> 
> 
> 
> ----- Original Message -----
> From: J Mohamed Zahoor <jmozah@gmail.com>
> To: user@hbase.apache.org
> Cc:
> Sent: Thursday, August 9, 2012 2:03 AM
> Subject: Re: Hbase- Hadoop DFS size not decreased even after deleting a column family
> 
> Hi
> 
> I see this atleast in trunk... in TableDeleteFamilyHandler's constructor..
> 
> public TableDeleteFamilyHandler(byte[] tableName, byte [] familyName,
> 
>       Server server, final MasterServices masterServices) throwsIOException {
> 
>     super(EventType.C_M_ADD_FAMILY, tableName, server, masterServices);
> 
>     HTableDescriptor htd = getTableDescriptor();
> 
>     this.familyName = hasColumnFamily(htd, familyName);
> 
>   }
> 
> 
> The event type is added as C_M_ADD_FAMILY....
> May be this is the problem and it is there in 0.94.1 also !!!
> 
> ./zahoor
> 
> 
> On Wed, Aug 8, 2012 at 5:33 PM, prem yadav <ipremyadav@gmail.com> wrote:
> 
> > Hi Stack,
> > we have got a lot of these in the logs after we deleted the column family.
> >
> > org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
> > family ol does not exist in region <region>
> >         at
> > org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:4759)
> >         at
> >
> > org.apache.hadoop.hbase.regionserver.HRegion.checkFamilies(HRegion.java:2639)
> >         at
> >
> > org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2065)
> >         at
> > org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:1962)
> >         at
> >
> > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3428)
> >         at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
> >         at
> >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:601)
> >         at
> >
> > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> >         at
> > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1386)
> > 2012-08-05 00:11:37,697 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> > rollbackMemstore rolled back 0 keyvalues from start:0 to end:1
> >
> >
> >
> >
> > On Tue, Aug 7, 2012 at 10:27 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > Looking at TableDeleteFamilyHandler, I see:
> > >
> > >   protected void handleTableOperation(List<HRegionInfo> hris) throws
> > > IOException {
> > >     // Update table descriptor in HDFS
> > >     HTableDescriptor htd =
> > >       this.masterServices.getMasterFileSystem().deleteColumn(tableName,
> > > familyName);
> > >     // Update in-memory descriptor cache
> > >     this.masterServices.getTableDescriptors().add(htd);
> > >
> > > MasterFileSystem.deleteColumn() calls:
> > >
> > >     HTableDescriptor htd =
> > > this.services.getTableDescriptors().get(tableName);
> > >     htd.removeFamily(familyName);
> > >
> > > I will dig some more.
> > >
> > > But looks like there is a bug.
> > >
> > > On Tue, Aug 7, 2012 at 9:21 AM, Stack <stack@duboce.net> wrote:
> > >
> > > > On Tue, Aug 7, 2012 at 9:59 AM, prem yadav <ipremyadav@gmail.com>
> > wrote:
> > > > > Hi Stack,
> > > > > yes . I can still see the column family.
> > > > >
> > > > > we ran the following command too:
> > > > >
> > > > > fs -dus /hbase/<table name>/<region hash>/<cf that
was deleted> and
> > it
> > > > > returned a large value which means that the space has not been
> > > reclaimed.
> > > > > The hbase version we are running is 0.94.1.
> > > > >
> > > > > is there  a manual way to force reclaim the space?
> > > > >
> > > >
> > > > You could delete the column family from hdfs under each region.
> > > > Anything in the logs around the remove of the column family?  Maybe we
> > > > just don't do the delete cleanup in hdfs when we drop a cf (I've not
> > > > checked)?
> > > >
> > > > St.Ack
> > > >
> > >
> >
> 
> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message