hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Wang <...@cloudera.com>
Subject Re: Some notes from the HBase developer Pow-Wow
Date Wed, 20 Feb 2013 01:47:23 GMT
Thanks for the summary Deveraj.

The webex recording for the meeting is at:

https://cloudera.webex.com/cloudera/ldr.php?AT=pb&SP=MC&rID=120418137&rKey=d058765b798c47c5

Please let me know if you cannot access it.

Regards,

- Dave


On Tue, Feb 19, 2013 at 5:30 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Awesome!
>
> Thanks a for those notes Devaraj!
>
> Very useful for the unfortunates who did not got the chance to join the
> meeting ...
> Le 19 févr. 2013 20:27, "Devaraj Das" <ddas@hortonworks.com> a écrit :
>
> > - Stabilizing 0.96 / CI
> >   -- not running continuously
> >   -- Roman:   is it a good idea to run IT under Bigtop
> >        - the HBase unit tests are good..
> >        - What about HBase as a backend for hive - tests for those
> >        - Do we think there is value with the integration with the rest
> > of the hadoop ecosystem
> >   -- Jon: What's needed to make this work
> >   -- Roman: Collective agreement that we want to solve this
> >   -- Stack: We need to run the IT tests from Enis and Sergey running
> > continuously
> >   -- Roman: If we all agree that this is something needs to be fixed
> > .. then yes we would talk about mechanics. Bigtop doesn't have
> > expertise in HBase and hence HBase folks would have to debug failures.
> >   -- Roman: Bigtop is committed to tests. Less than a dozen tests for
> > hbase currently..
> >   -- We have been running validation around RC time. Find all kinds of
> > issues - sometimes trivial (maybe a config issue),
> >   -- Roman can offer CI for trunk but will work for only hadoop-2 line.
> >   -- Roman does first line of triage.
> >   -- No issues to do with other ecosystem artifacts. Bigtop ensures
> > the right artifacts are in place.
> >   -- hadoop-2 is important but not particular about the version of
> > hadoop within 2. For example 2.0.2
> >   -- Gary: Will be good to run security tests
> >   -- Roman & Devaraj to talk on how this can be done/implemented
> >
> > On 0.96 branching
> > - Lars:
> >    -- We will have three branches to maintain.
> > - Stack: we need to stabilize quickly
> > - Enis: What about 0.95.
> > - Stack: Just do the snapshots thing. Every week, give a snapshot
> > - Enis: we've done a bunch of stuff in RPC.. If we have to break
> > something, we can if it is beta (0.96-beta).
> > - Agreement is there generally ... Debate on the name with snapshot
> > versus 95.0/96.0
> >   -- 0.95 experimental .. 0.96 will be stable
> >   -- If we go with 0.95, releasing will be easier as well..
> >   -- 0.95 will not be in production..
> >   -- 0.96 will be off 0.95 branch and not trunk based
> >
> > - How do we go about committing issues.. (issue commit rate is low)
> >  -- 0.94 is stable -  2 new commits and 2 new bugs a day
> >  -- 0.96 has lots of issues not reviewed
> >  -- Break up the patch into multiple smaller pieces to make review easier
> >  -- Branching on a big feature was suggested -
> >    --- Issues: committer needs to be there
> >          Sergey: If the rate of change is high on common code (to the
> > branches) then merging will be tough
> >    --- Jon: Refactor should be done in the main branch (since it
> > doesn't add any new funtionality)
> >    --- Release often to reduce #backports overall and issues with that..
> >
> > - Review process .. how to drive a review to closure. Effort goes
> > waste if the review process is not completed. The same reviewer should
> > continue to review the patch ..
> > - Hard to enforce any process
> > - Enis: there should be a summary of the patch and all that .. so that
> > the review process is helped.. Hard to understand the architecture of
> > the patch unless documented
> > - Jon: It should be easy to make a one-to-one correspondence between
> > the description and the patch
> > - The commit should have only the jira# as opposed to pages of
> description
> >
> > - Component owners: is this working. Committers need to be forthcoming
> > with reviews
> >   -- Maybe review the modules and add some more if needed.
> >
> >   -- Good that we have more contributions coming than we have
> > reviewers, but unless we keep up, we will plateau
> >   -- Mail on dev@ list if review doesn't happen
> >
> > - Dev co-ordination:
> >  -- How best can we pull together
> >  -- Priorities:
> >    --- Getting 0.96 out is priority
> >    --- Backports to 0.94 will happen .. until 0.96 is stable
> >    --- 0.92 release ? Any committer who wants to make a release can do
> > so (maybe with some backports, etc.)
> >    --- Backporting can be tough if there are bugs and the bugfix has
> > to be applied to all branches
> >
> > - HBASE-2600 - this requires a change in the client and the server.
> > They have to be changed in lock-step. Its hard to do this .. Jon
> > doesn't want to have the fix for 0.96. So 0.98 might be another
> > singular release. Maybe do a rewrite of the meta after taking a lock
> > on the meta, do a shim layer to handle the backward compat. between
> > 0.96 and 0.98
> >
> > - What do people want to get into 0.94
> >  -- The biggest thing - Snapshots, mostly new code, about a 3rd of the
> > stuff in 0.94 already
> >  -- Compactions improvments - no backport
> >
> > - How devs can better co-ordinate
> >   -- Snapshots co-ordination working well
> >   -- One page design is useful (makes it readable and all)
> >   -- How about handling the stripe compaction - where an idea leads to
> > a bunch of others
> >   -- Again write-up should be done
> >
> > - Should we change the description to match the comments
> >   -- Two ideas suggested:
> >    ---  We probably should have the description updated with the
> > "Date: new description" if the issue at hand is updated
> >    --- Should we have a summary after a bunch of comments - yes
> >
> > - The face-to-face meetings are useful. We should semd out the minutes
> > of the discussion to the dev list. We probably should have more
> > focused huddles. Discuss but don't decide (decide on the jira)!
> >
> > - Jon: Would people be amenable to merge sooner rather than later on
> > snapshots? Tested and being beaten up.
> > - Stack: Yes
> >
> > - What else goes in in 0.96:
> >   -- RPC refactor
> >   -- ROOT removal
> >   -- Compaction stuff
> >   -- Package name mailing list thread - there is now a jira on that.
> > We shouldn't break clients. Package name changes is not worth the
> > trouble.
> >
> > - A bunch of discussions on the RPC with KeyValue/Cells
> >
> > - What do we do about usability
> >   -- It'll be nice if we don't need to change configs..
> >   -- Maybe expose more metrics and then allow for online config
> > changes since automatic config is difficult and needs to be battle
> > tested and all.
> >
> > - Benchmarking of the release:
> >   -- We should measure the overhead of PB stuff
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message