From hdfs-dev-return-31748-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Tue Mar 6 00:55:16 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 24403180608 for ; Tue, 6 Mar 2018 00:55:15 +0100 (CET) Received: (qmail 10669 invoked by uid 500); 5 Mar 2018 23:55:09 -0000 Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-dev@hadoop.apache.org Received: (qmail 10643 invoked by uid 99); 5 Mar 2018 23:55:09 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Mar 2018 23:55:09 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id ACB3EC00B7 for ; Mon, 5 Mar 2018 23:55:08 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=cloudera.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id wQPiKW869itD for ; Mon, 5 Mar 2018 23:55:07 +0000 (UTC) Received: from mail-wm0-f41.google.com (mail-wm0-f41.google.com [74.125.82.41]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 242C75F3B9 for ; Mon, 5 Mar 2018 23:55:07 +0000 (UTC) Received: by mail-wm0-f41.google.com with SMTP id q83so19402047wme.5 for ; Mon, 05 Mar 2018 15:55:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudera.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ewiaSu6iA76vjs6eys4DayBO8DmVKrIP2K4QyRK3xlM=; b=Fs8x1MfL24KY5DPAlwWx1kAy0v2r7Uv7IXqvLufFfJpBs/3qIjeSyTynZ2SA15KgR3 ErJtwpIGduSXlqgv2h3jum+KXiQKa4vuGoW9yaiNbI26AztBVRT2e2HihxBduupv0OS1 oqrHlBHTo4ALILW/dSFC9EjvZysI0Aj4l+QEyTQZ6I7xv3UNH8j6im5B+4BXo2UaxFD9 lmg+i+5CbMuYGNQ/0eVV6A/vQgzFo50eXzUQWANVKOFjMRiSomvHo2usliMEv6VgSMGS F/AwRDaBWjsujnXl3TXXqF0eOyQCKKSlgswesPKSm+4T/ROnf/+ZsyOehy17Tx2MGvoE VUjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ewiaSu6iA76vjs6eys4DayBO8DmVKrIP2K4QyRK3xlM=; b=WLqhDF1JBBNJWEOfzNtE81jrWjEBvGJnTeE+CtPQXeZ3UATHNcsTpKd6GqSWOUZkXN znZAftWhQNKymyYUIqWf2zicW06z6M7svFVDGQTmOLzQh2j0aC5iZhW8f6PYpYGqS/8H nJNC3S+kHULY60K2Gw//w+lwl7Ru6KwLp22JS7VRpI9Fy/6ifHMHmTzHyvhcyZM0BHqF P1BgccsFK2sdX+6m4X15dN+pDTfU18SJpFUr9FGh2rgOUefi66kkFec8eWC2IMkCLNkq R3kf+yP+mVI/NhRTfPAVyjHPZ29FC2PGbopJBTHq7dWp+VT6XG1NK7cBenuQHdVcn9V0 Bctg== X-Gm-Message-State: APf1xPAAnPIWLHg8oCSsU9rSQWytUEZMXEZExnt1GeaO+cNv3LbZNyX/ AdpcnL1h9XLDIT9O8038+jhdXey7SzJgl8Dbhcc+5g== X-Google-Smtp-Source: AG47ELuSQbVEz6md2eIefN2aboXEawcnGxci+ZICrAxN5MHyCCo+ZqBKvfTvT6+Hh+eDA4u22301TxgBnkUVT8qCTS8= X-Received: by 10.80.137.213 with SMTP id h21mr21487142edh.39.1520294105867; Mon, 05 Mar 2018 15:55:05 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.167.131 with HTTP; Mon, 5 Mar 2018 15:54:45 -0800 (PST) In-Reply-To: References: <460F525D-20D1-42DB-A064-2512CC070614@hortonworks.com> From: Andrew Wang Date: Mon, 5 Mar 2018 15:54:45 -0800 Message-ID: Subject: Re: [VOTE] Merging branch HDFS-7240 to trunk To: Wangda Tan Cc: "Owen O'Malley" , Daryn Sharp , Jitendra Pandey , hdfs-dev , "common-dev@hadoop.apache.org" , "yarn-dev@hadoop.apache.org" , "mapreduce-dev@hadoop.apache.org" Content-Type: multipart/alternative; boundary="f403045c1834228fbb0566b30eee" --f403045c1834228fbb0566b30eee Content-Type: text/plain; charset="UTF-8" Hi Owen, Wangda, Thanks for clearly laying out the subproject options, that helps the discussion. I'm all onboard with the idea of regular releases, and it's something I tried to do with the 3.0 alphas and betas. The problem though isn't a lack of commitment from feature developers like Sanjay or Jitendra; far from it! I think every feature developer makes a reasonable effort to test their code before it's merged. Yet, my experience as an RM is that more code comes with more risk. I don't believe that Ozone is special or different in this regard. It comes with a maintenance cost, not a maintenance benefit. I'm advocating for #3: separate source, separate release. Since HDSL stability and FSN/BM refactoring are still a ways out, I don't want to incur a maintenance cost now. I sympathize with the sentiment that working cross-repo is harder than within same repo, but the right tooling can make this a lot easier (e.g. git submodule, Google's repo tool). We have experience doing this internally here at Cloudera, and I'm happy to share knowledge and possibly code. Best, Andrew On Fri, Mar 2, 2018 at 4:41 PM, Wangda Tan wrote: > I like the idea of same source / same release and put Ozone's source under > a different directory. > > Like Owen mentioned, It gonna be important for all parties to keep a > regular and shorter release cycle for Hadoop, e.g. 3-4 months between minor > releases. Users can try features and give feedbacks to stabilize feature > earlier; developers can be happier since efforts will be consumed by users > soon after features get merged. In addition to this, if features merged to > trunk after reasonable tests/review, Andrew's concern may not be a problem > anymore: > > bq. Finally, I earnestly believe that Ozone/HDSL itself would benefit from > being a separate project. Ozone could release faster and iterate more > quickly if it wasn't hampered by Hadoop's release schedule and security and > compatibility requirements. > > Thanks, > Wangda > > > On Fri, Mar 2, 2018 at 4:24 PM, Owen O'Malley > wrote: > >> On Thu, Mar 1, 2018 at 11:03 PM, Andrew Wang >> wrote: >> >> Owen mentioned making a Hadoop subproject; we'd have to >> > hash out what exactly this means (I assume a separate repo still >> managed by >> > the Hadoop project), but I think we could make this work if it's more >> > attractive than incubation or a new TLP. >> >> >> Ok, there are multiple levels of sub-projects that all make sense: >> >> - Same source tree, same releases - examples like HDFS & YARN >> - Same master branch, separate releases and release branches - Hive's >> Storage API vs Hive. It is in the source tree for the master branch, >> but >> has distinct releases and release branches. >> - Separate source, separate release - Apache Commons. >> >> There are advantages and disadvantages to each. I'd propose that we use >> the >> same source, same release pattern for Ozone. Note that we tried and later >> reverted doing Common, HDFS, and YARN as separate source, separate release >> because it was too much trouble. I like Daryn's idea of putting it as a >> top >> level directory in Hadoop and making sure that nothing in Common, HDFS, or >> YARN depend on it. That way if a Release Manager doesn't think it is ready >> for release, it can be trivially removed before the release. >> >> One thing about using the same releases, Sanjay and Jitendra are signing >> up >> to make much more regular bugfix and minor releases in the near future. >> For >> example, they'll need to make 3.2 relatively soon to get it released and >> then 3.3 somewhere in the next 3 to 6 months. That would be good for the >> project. Hadoop needs more regular releases and fewer big bang releases. >> >> .. Owen >> > > --f403045c1834228fbb0566b30eee--