hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@apache.org>
Subject Re: [DISCUSS] Multiplicity of repos and release tooling
Date Sat, 28 Sep 2019 18:12:56 GMT
On Sat, Sep 28, 2019 at 10:58 Stack <stack@duboce.net> wrote:

> hbase-operator-tools/create-release or a new repo altogether?
>

If we’re going to have one tool to release them all, I’m okay with it
staying in the main repo under dev-support or similar, so long as it’s on
only one branch (probably master, maybe it’s own). If the tools is itself
is going to be released and made installable (on a developer laptop or a
Jenkins worker) then I think it should have its own repo.

> On Wed, Sep 25, 2019 at 8:56 AM Peter Somogyi <psomogyi@apache.org> wrote:
> > >
> > > It is great that you brought up this topic Nick! I agree that the
> optimal
> > > solution would be to host all the release related scripts (RC build,
> > > verifier) in a common place.
> > >
> > > The RC making scrip in hbase-operator-tools that Stack finetuned is
> meant
> > > to work with different artifacts. The current version there gives the
> RM
> > a
> > > smooth experience. It emerged from HBase's create-release script and
> > > hopefully it can be used on other releases as well. There are some
> > > constraints of the tool like Jira versions should use
> > > `hbase-operator-tools` prefix.
> > >
> > > > This is a good point. I wonder if `dev-support` should be pruned or
> > purged
> > > from all branches other than master
> > >
> > > When we create branch-3 out of master then this becomes a problem
> again.
> > > Would it work if we use a specific branch for such tooling (e.g.
> > > dev-tools)? In this case RMs can just clone that branch and don't need
> > the
> > > whole HBase repository on their local machine.
> > >
> > > On Thu, Sep 19, 2019 at 5:40 PM Nick Dimiduk <ndimiduk@apache.org>
> > wrote:
> > >
> > > > On Wed, Sep 18, 2019 at 10:30 AM Sean Busbey <busbey@apache.org>
> > wrote:
> > > >
> > > > > If the tooling is in one place where will it live?
> > > > >
> > > > > As an RM I like not needing to checkout more then the repo I'm
> > trying to
> > > > > get a release out on.
> > > > >
> > > >
> > > > My initial thinking was all would be in the main repo, but this is
> > contrary
> > > > to your above statement. As I see it, everyone working on HBase has a
> > > > checkout of the main repo handy, so for releases spun on developer
> > machines
> > > > it's "no big deal." If we can ever get to releases spun through
> > automated
> > > > environments, it's all scripted anyway and thus "no big deal."
> > > >
> > > > In particular my release machine is slow on disk and so updates to
> the
> > main
> > > > > git repo when trying to release something not in the main repo will
> > be
> > > > > painful.
> > > >
> > > >
> > > > "slow on disk" as in iops rate or "low on disk" as in capacity?
> Either
> > way,
> > > > I'm surprised to hear about this as a barrier. Is there a
> > side-conversation
> > > > we can have about trimming the fat out of our repo(s)? Some git
> > maintenance
> > > > that can/needs to be done? I was recently shocked by the girth of our
> > repos
> > > > -- nearly 0.5g on the main repo and a whopping 4g on the site!
> > > >
> > > > For the most part this is also why I usually manually build RCs
> > > > > that I run for the main repo, because I can do a shallow clone of
> the
> > > > > release branch instead of needing to get updates to all the active
> > > > > branches.
> > > > >
> > > >
> > > > This makes me sad. Automate more, not less! Altering the automation
> to
> > make
> > > > shallow clones of targeted branches should be simple enough, no?
> > > >
> > > > For testing RCs I guess it's currently all some combination of the
> > > > > foundation policies that should be the same and maybe maven
> profiles?
> > > > >
> > > >
> > > > I agree that codification of foundation policy is the baseline. That
> > would
> > > > be enough for me as a first pass. After that, perhaps layers of
> > increasing
> > > > sophistication, perhaps repo-dependent. I don't follow your meaning
> re:
> > > > maven profiles.
> > > >
> > > > Iirc there was already some confusion about using the testing script
> > that
> > > > > came in the main repo source bundle vs the one on master.
> > > > >
> > > >
> > > > This is a good point. I wonder if `dev-support` should be pruned or
> > purged
> > > > from all branches other than master. Maybe the CI related stuff
> > branches
> > > > into it's own directory or directories, and we keep everything else
> > limited
> > > > to a single canonical copy on master. This strategy would eliminate
> > > > confusion re: what is authority in any given situation but perhaps is
> > too
> > > > constraining, given the number of active release branches we
> maintain.
> > > >
> > > > What issue are we trying to solve?
> > > > >
> > > >
> > > > Minimize contributor friction. Make it easy for every subscriber to
> > dev@
> > > > to
> > > > say "There's a new RC posted and I have an idle machine for an hour /
> > for
> > > > the evening / for the weekend, let's just kick the tires." Make it
> > easy for
> > > > everyone who's learned the RC tooling on one branch to pinch-hit in
> on
> > > > another branch or another repo. I hear a constant complaint of
> > "scarcity of
> > > > volunteer hours" and "I wish we had more people voting in RCs" and "I
> > wish
> > > > we could keep a monthly release cadence on every branch we're
> > maintaining".
> > > > So I'd like to see a focused effort on maximizing the volunteer human
> > and
> > > > machine time that's thrown our way.
> > > >
> > > > Thanks,
> > > > Nick
> > > >
> > > > On Wed, Sep 18, 2019, 11:50 Nick Dimiduk <ndimiduk@apache.org>
> wrote:
> > > > >
> > > > > > Heya,
> > > > > >
> > > > > > Looks like we have quite a few repos now, each of which must
> > produce
> > > > > > artifacts that follow the Apache protocols. I also see we have
> some
> > > > nice
> > > > > > tools built up in dev-support in the main repo for RM's who
build
> > > > release
> > > > > > candidates and community members to vote on them. I tried to
use
> > our
> > > > > > hbase-vote.sh on this operator-tools RC and found it mostly
> works.
> > I
> > > > > think
> > > > > > a few adjustments on each end would allow these tools to work
> > across
> > > > > repos,
> > > > > > so I filed HBASE-23048. However, I now see that operator-tools
> has
> > its
> > > > > own
> > > > > > dev-support/create-release, so I wonder what direction we want
to
> > take
> > > > > with
> > > > > > this automation, so here I come to the list.
> > > > > >
> > > > > > Do we want to have independent tooling for each repo? Are the
> > processes
> > > > > of
> > > > > > building RC's different enough to warrant separate tools? Is
a
> > single
> > > > > tool
> > > > > > that can build an RC for all of them not worth the trouble?
At
> the
> > very
> > > > > > least, I think the hbase-vote.sh can be made to work with
> releases
> > > > > > generated from each of the repos, as it's not doing all that
> much.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > Thanks,
> > > > > > Nick
> > > > > >
> > > > > > On Wed, Sep 18, 2019 at 9:42 AM Nick Dimiduk <
> ndimiduk@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > >  - Verified signatures.
> > > > > > >  - Verified checksums.
> > > > > > >  - Built from source tarball successfully.
> > > > > > >  - Ran unit tests from source tarball, pass.
> > > > > > >  - Ran rat check from source tarball, pass.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Sep 16, 2019 at 11:31 AM Peter Somogyi <
> > psomogyi@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Please vote on this Apache HBase Operator Tools Release
> > Candidate
> > > > > (RC),
> > > > > > >> 1.0.0.
> > > > > > >>
> > > > > > >> The VOTE will remain open for at least 72 hours.
> > > > > > >>
> > > > > > >> [ ] +1 Release this package as Apache HBase Operator
Tools
> 1.0.0
> > > > > > >> [ ] -1 Do not release this package because ...
> > > > > > >>
> > > > > > >> The tag to be voted on is 1.0.0RC0:
> > > > > > >>
> > > > > > >>  https://github.com/apache/hbase-operator-tools/tree/1.0.0RC0
> > > > > > >>
> > > > > > >> The release files, including signatures, digests, as
well as
> > > > > CHANGES.md
> > > > > > >> and RELEASENOTES.md included in this RC can be found
at:
> > > > > > >>
> > > > > > >>  https://dist.apache.org/repos/dist/dev/hbase/1.0.0RC0/
> > > > > > >>
> > > > > > >> Maven artifacts are available in a staging repository
at:
> > > > > > >>
> > > > > > >>
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehbase-1348/
> > > > > > >>
> > > > > > >> Artifacts were signed with the psomogyi@apache.org
key which
> > can be
> > > > > > found
> > > > > > >> in:
> > > > > > >>
> > > > > > >>  https://dist.apache.org/repos/dist/release/hbase/KEYS
> > > > > > >>
> > > > > > >> To learn more about Apache HBase Operator Tools, please
see
> > > > > > >> http://hbase.apache.org/
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Your HBase Release Manager
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message