hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: Meeting about hbase 0.95 release
Date Mon, 06 May 2013 22:27:13 GMT
 I do not remember all the details but I believe the recovery lease issue
had to do with jimmy and the locality improvements was likely referring to
Deveraj's work.

Jon.


On Mon, May 6, 2013 at 3:21 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Jon:
> Thanks for the notes.
> Do you remember a bit more detail for the following two topics ?
>
> 'recover lease thing' and 'locality improvements'
>
> Cheers
>
> On Mon, May 6, 2013 at 11:43 AM, Jonathan Hsieh <jon@cloudera.com> wrote:
>
> > These are my rough notes from a meatspace meeting was held on 5/3/15
> >
> > tl;dr a large group of folks pushing on the 0.95 release went through the
> > myriad of jirae and hashed out which ones we should try to get into the
> > next 0.95 release, and bumped several issues from the 0.95.1 release.
> >
> > There was also a substantial discussion about getting hadoop2 tests to
> pass
> > (and understanding why the currently fail) and also about the new table
> > namespacing feature.
> >
> > Other topics: more people please review, BoF session for Hadoop Summit,
> > Integration testing suite.
> >
> > ----
> >
> > (notes get sparser towards the end).
> >
> > 5/3/13
> > Location: Hortonworks HQ, Palo Alto, CA
> > Who:  Stack, JD, Jon H , Jimmy X, Himanshu, Dave W, Elliott C, Ted Y,
> Enis,
> > Deveraj, Sergey, Jeff Z, (Matteo, Nicholas L on the phone).
> >
> > Went through blockers and critical lists (goal, resolve or nix before
> > release):
> >
> > Blocker:
> > HBASE-5995 - logrol on dn pipeline restart (in progress)
> >
> > Critical:
> > HBASE-8366 logs full trace stuff.
> > HBASE-5923 check and clean - nix
> > HBASE-8450 - hbase-defaults.xml updates - stack to do, everyone please
> look
> > at it
> > HBASE-7997 - on last class moves. - not critical
> > HBASE-6891 - hadoop2 - investigate speed regression is hadoop2 or
> hadoop1.
> > * make unit test pass right new.  regression investigation not blocking.
> > * HBASE-8337 Hadoop2  - SCR problem - can resolve.  have reasonable
> reason
> > HBASE-7006 - mttr - distributed log replay - give feature good name and
> > close it out.
> > HBASE-3787 - increment non-idepmottent - sergey - working on tests.
> > HBASE-8449 - recover lease "fun and games" - stack working on it.
> > HBASE-7932 - region locations in meta.
> > HBASE-7897 - cells and tags. change interface. cell interface must be
> > hardened in 0.96. - big for intel
> > HBASE-8483 - zk leak.
> >
> > Selected Majors to bump to critical?:
> >
> >
> > HBASE-8143 - short circuit read.  OOM.  every reader 1MB buffer, 600
> block
> > readers, then lots of memory used.
> > - move to critical - enis: maybe set to hfile block size
> >
> > Would be good to fix:
> > HBASE-8385 -
> > HBASE-7910 -
> > HBASE-7391 -
> > HBASE-7709 - infite loop in m/m replication - would be good to fix.
> > HBASE-7564 - replication refactor. -- if done done, if not leave out.
> > HBASE-7958 - stats per column family per-region - remove from 0.95.1
> > HBASE-6294 - usability, nice to fix.
> > HBASE-8479 - compile issue - generics problem.
> > HBASE-4050 - metrics to metric2 umbrella blocked by HBASE-7074 - docs for
> > hbase-4050
> > HBASE-6580 htable pool broken - leaving in for now.
> > HBASE-7839 - dead machine in integration - nice to have
> > HBASE-7840 - nice to have
> >
> > ----
> > Enis:
> >
> > HBASE-8015 namespaces  (discussion below)
> > - file system structure likely changes things, ideally before 0.6
> > HBASE-6721 RS grouping stuff- can do with coproc and load balancer
> plugins
> > (not blocking0.
> > HBASE-7999 system tables? (deprecated if namespaces added)
> >
> > namespace descriptor -
> > - distribute in cluster.  similar to zk permisison watcher
> >
> > '.' vs some other separator char for <namespace><sep><table>
> > - '.' is what db's use, but is valid hbase table name and breaks hbase
> > tables
> > - other chars would look weird but don't break existing hbase tables.
> >
> > the proposed dir structure move has impact on upgrades
> > - do a shutdown restart upgrade?
> > - Discussed potential approach where empty name space files don't move
> (and
> > don't break rolling upgrade)
> > - discussed how to deal with matteo's hfile pool approach (one dir with
> > hfiles, meta just has pointers, not dependent on dir strcuture)
> >
> > how does this interact with rolling upgrades?
> > how does this  interact with hdfs quotas?
> >
> > stack:
> > - not in 0.95 unless progress is made.
> > - Let's try to get it committed to a branch ala snapshots
> >
> > ----
> >
> > patch available - discussion
> > - please do more reviews.
> >
> > ----
> >
> > integration tests
> > - alex 94
> > - elliot 95
> >
> > run via maven against distributed cluster
> > - system test framework
> > - cherry-pick, run via command line tool.
> > - 0.94 chaos monkey - every 1 in 5 fails
> > - 0.96 - 1 in 10 fails.
> >
> > machines don't come backup.
> >
> > 4 tests are good
> > - test too many regions
> > - want tests to scale to scale
> > - want to kill less -- not completely random.
> >
> > bigtop does this
> > - put in maven.
> >
> > -----
> >
> > recover lease thing
> >
> > ----
> >
> > locality improvements
> >
> > ----
> >
> > Hadoop summit
> > - day before - birds of a feather
> >
> > ----
> >
> > compactions
> >
> > --
> > // Jonathan Hsieh (shay)
> > // Software Engineer, Cloudera
> > // jon@cloudera.com
> >
>



-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message