parquet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From u..@apache.org
Subject parquet-cpp git commit: PARQUET-767: Add release scripts
Date Sat, 10 Dec 2016 08:29:39 GMT
Repository: parquet-cpp
Updated Branches:
  refs/heads/master 641984e6d -> a1517582f


PARQUET-767: Add release scripts

Based on the release scripts from Apache Aurora (as we don't have the luxury of the Maven
release plugin, we need a bit more code). Kudos to @StephanErb for pointing me to them.

- [x] Rename JIRA release tag from `cpp-0.1` to `cpp-0.1.0` ( @julienledem can you take care
of this?) This is then in line with all other Apache Parquet artifacts.
- [x] Test the scripts with dry-run mode (needs the above point)
- [x] Reference Apache Aurora in LICENSE/NOTICE
- [x] Add Guides
- [x] Add script to verify the release

Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Author: Uwe L. Korn <uwelk@xhochy.com>

Closes #199 from xhochy/PARQUET-767 and squashes the following commits:

0f3ca22 [Korn, Uwe] Add script to verify release
58a24f9 [Uwe L. Korn] Add guide
37c4877 [Korn, Uwe] Mention Apache Aurora in the License
be828bd [Korn, Uwe] Fix path to changelog script
1adf248 [Korn, Uwe] PARQUET-767: Add release scripts


Project: http://git-wip-us.apache.org/repos/asf/parquet-cpp/repo
Commit: http://git-wip-us.apache.org/repos/asf/parquet-cpp/commit/a1517582
Tree: http://git-wip-us.apache.org/repos/asf/parquet-cpp/tree/a1517582
Diff: http://git-wip-us.apache.org/repos/asf/parquet-cpp/diff/a1517582

Branch: refs/heads/master
Commit: a1517582fb2bcdde970e7c36be2e4a78cc9dbb5b
Parents: 641984e
Author: Korn, Uwe <Uwe.Korn@blue-yonder.com>
Authored: Sat Dec 10 09:29:24 2016 +0100
Committer: Uwe L. Korn <uwelk@xhochy.com>
Committed: Sat Dec 10 09:29:24 2016 +0100

----------------------------------------------------------------------
 .parquetcppversion                   |   1 +
 LICENSE.txt                          |  11 ++
 dev/committers-guide.md              | 118 ++++++++++++
 dev/release/changelog                | 155 +++++++++++++++
 dev/release/release                  | 250 ++++++++++++++++++++++++
 dev/release/release-candidate        | 308 ++++++++++++++++++++++++++++++
 dev/release/verify-release-candidate |  92 +++++++++
 7 files changed, 935 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/.parquetcppversion
----------------------------------------------------------------------
diff --git a/.parquetcppversion b/.parquetcppversion
new file mode 100644
index 0000000..b694fe3
--- /dev/null
+++ b/.parquetcppversion
@@ -0,0 +1 @@
+0.1.0-SNAPSHOT

http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/LICENSE.txt
----------------------------------------------------------------------
diff --git a/LICENSE.txt b/LICENSE.txt
index 1bf56ab..5e63e2b 100644
--- a/LICENSE.txt
+++ b/LICENSE.txt
@@ -244,3 +244,14 @@ Home page: http://impala.apache.org/
 License: http://www.apache.org/licenses/LICENSE-2.0
 
 --------------------------------------------------------------------------------
+
+This product includes code from Apache Aurora.
+
+* dev/release/{release,changelog,release-candidate} is based on the scripts from
+  Apache Aurora
+
+Copyright: 2016 The Apache Software Foundation.
+Home page: https://aurora.apache.org/
+License: http://www.apache.org/licenses/LICENSE-2.0
+
+--------------------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/committers-guide.md
----------------------------------------------------------------------
diff --git a/dev/committers-guide.md b/dev/committers-guide.md
new file mode 100644
index 0000000..9c15071
--- /dev/null
+++ b/dev/committers-guide.md
@@ -0,0 +1,118 @@
+<!---
+  Licensed under the Apache License, Version 2.0 (the "License");
+  you may not use this file except in compliance with the License.
+  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License. See accompanying LICENSE file.
+
+  Original imported from Apache Aurora.
+-->
+
+Committer's Guide
+=================
+
+Information for official Apache Parquet committers.
+
+Setting up your email account
+-----------------------------
+Once your Apache ID has been set up you can configure your account and add ssh keys and setup
an
+email forwarding address at
+
+    http://id.apache.org
+
+Additional instructions for setting up your new committer email can be found at
+
+    http://www.apache.org/dev/user-email.html
+
+The recommended setup is to configure all services (mailing lists, JIRA) to send
+emails to your @apache.org email address.
+
+
+Creating a gpg key for releases
+-------------------------------
+In order to create a release candidate you will need a gpg key published to an external key
server
+and that key will need to be added to our KEYS file as well.
+
+1. Create a key:
+
+               gpg --gen-key
+
+2. Add your gpg key to the Apache Parquet {format,mr,cpp} KEYS file:
+
+               git clone https://git-wip-us.apache.org/repos/asf/parquet-{format,mr,cpp}.git
+               (gpg --list-sigs <KEY ID> && gpg --armor --export <KEY ID>)
>> KEYS
+               git add KEYS && git commit -m "Adding gpg key for <APACHE ID>"
+               ./rbt post -o -g
+
+3. Publish the key to an external key server:
+
+               gpg --keyserver pgp.mit.edu --send-keys <KEY ID>
+
+4. Update the changes to the KEYS file to the Apache Parquet svn dist locations listed below:
+
+               https://dist.apache.org/repos/dist/dev/parquet/KEYS
+               https://dist.apache.org/repos/dist/release/parquet/KEYS
+
+5. Add your key to git config for use with the release scripts:
+
+               git config --global user.signingkey <KEY ID>
+
+
+Creating a release
+------------------
+The following will guide you through the steps to create a release candidate, vote, and finally
an
+official Apache Parquet C++ release. Before starting your gpg key should be in the KEYS file
and you
+must have access to commit to the dist.a.o repositories.
+
+1. Ensure that all issues resolved for this release candidate are tagged with the correct
Fix
+Version in JIRA, the changelog script will use this to generate the CHANGELOG in step #2.
+To assign the fix version:
+
+    * Look up the [previous release date](https://issues.apache.org/jira/browse/parquet/?selectedTab=com.atlassian.jira.jira-projects-plugin:versions-panel).
+    * Query all issues resolved after that release date: `project = PARQUET AND status in
(resolved, Closed) and fixVersion is empty and resolutiondate >= "YYYY/MM/DD"`
+    * In the upper right corner of the query result, select Tools > Bulk Edit.
+    * Select all issues > edit issue > set 'Change Fix Version/s' to the release version.
+    * Make sure to uncheck 'Send mail for this update' at the bottom.
+
+2. Prepare RELEASE-NOTES.md for the release. This just boils down to removing the "(Not yet
+released)" suffix from the impending release.
+
+2. Create a release candidate. This will automatically update the CHANGELOG and commit it,
create a
+branch and update the current version within the trunk. To create a minor version update
and publish
+it run
+
+               ./dev/release/release-candidate -l m -p
+
+3. Update, if necessary, the draft email created from the `release-candidate` script in step
#2 and
+send the [VOTE] email to the dev@ mailing list. You can verify the release signature and
checksums
+by running
+
+               ./dev/release/verify-release-candidate
+
+4. Wait for the vote to complete. If the vote fails close the vote by replying to the initial
[VOTE]
+email sent in step #3 by editing the subject to [RESULT][VOTE] ... and noting the failure
reason
+(example [here](http://markmail.org/message/d4d6xtvj7vgwi76f)). You'll also need to manually
revert
+the commits generated by the release candidate script that incremented the snapshot version
and
+updated the changelog. Once that is done, now address any issues and go back to step #1 and
run
+again, this time you will use the -r flag to increment the release candidate version. This
will
+automatically clean up the release candidate rc0 branch and source distribution.
+
+               ./dev/release/release-candidate -l m -r 1 -p
+
+5. Once the vote has successfully passed create the release
+
+**IMPORTANT: make sure to use the correct release at this final step (e.g.: `-r 1` if rc1
candidate
+has been voted for). Once the release tag is pushed it will be very hard to undo due to remote
+git pre-receive hook explicitly forbidding release tag manipulations.**
+
+               ./dev/release/release
+
+6. Update the draft email created fom the `release` script in step #5 to include the Apache
ID's for
+all binding votes and send the [RESULT][VOTE] email to the dev@ mailing list.
+

http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/release/changelog
----------------------------------------------------------------------
diff --git a/dev/release/changelog b/dev/release/changelog
new file mode 100755
index 0000000..d6ed9f2
--- /dev/null
+++ b/dev/release/changelog
@@ -0,0 +1,155 @@
+#!/usr/bin/env ruby
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Updates the CHANGELOG for the specified Apache Parquet C++ release, if no
+# version override is specified then .parquetcppversion will be read and used.
+# The version string will need to match the Apache Parquet C++ jira tickets
+# fixVersion to generate this list.
+#
+require 'rubygems'
+require 'json'
+require 'uri'
+require 'net/http'
+require 'net/https'
+require 'openssl'
+require 'tempfile'
+
+def http_post(uri, params={}, headers={}, body=nil, debug=false)
+  http = Net::HTTP.new(uri.host, uri.port)
+  http.set_debug_output($stdout) if debug
+  if uri.scheme == "https" || uri.port == 443
+    http.use_ssl = true
+    http.verify_mode = OpenSSL::SSL::VERIFY_NONE
+  end
+  response = nil
+
+  begin
+    request = Net::HTTP::Post.new(uri.request_uri)
+    request.set_form_data(params) if (params && params.size > 1)
+    request.basic_auth uri.user, uri.password if uri.user
+    request.body = body if body
+
+    headers.each_pair do |name, value|
+      request[name] = value
+    end
+
+    response = http.request(request)
+  rescue Exception => e
+    raise "Error posting to #{uri.to_s}. #{e}"
+  end
+
+  response
+end
+
+def get_jira_issues_for_query(url, jql, startAt=0, maxResults=100)
+  puts(jql)
+  begin
+    request_body = { :jql => jql, :startAt => startAt, :maxResults => maxResults}
+    response = http_post(
+      URI(url),
+      {},
+      {"content-type" => 'application/json', "accept" => "application/json"},
+      request_body.to_json
+    )
+  rescue StandardError => e
+    raise "Error executing jql query: #{jql}. #{e}"
+  end
+
+  begin
+    results = JSON.parse(response.body)
+  rescue StandardError => e
+    raise "Error parsing json result"
+  end
+
+  results
+end
+
+def get_all_jira_issues_for_query(url, jql)
+  startAt = 0
+  maxResults = 100
+
+  results = []
+  begin
+    res = get_jira_issues_for_query(url, jql, startAt, maxResults)
+    results.concat(res['issues']) if res && res.has_key?('issues')
+    startAt = startAt + maxResults
+  end until (results.size() == res['total'] )
+
+  results
+end
+
+base_dir = `git rev-parse --show-toplevel`.strip
+
+# Get the current version from the .parquetcppversion file if no version override is provided
+version = nil
+if ARGV[0].nil?
+  version = nil
+  Dir.chdir(base_dir) do
+    version = File.read('.parquetcppversion').strip
+  end
+else
+  version = ARGV[0]
+end
+
+raise "Unable to read .parquetcppversion" if version.nil?
+
+jira_base_url = "https://issues.apache.org/jira"
+jira_search_url = "/rest/api/2/search"
+jira_url = jira_base_url + jira_search_url
+
+jql="project=PARQUET " \
+    "AND fixVersion='cpp-#{version}' " \
+    "AND status = Resolved " \
+    "AND resolution in (Fixed, Done) " \
+    "ORDER BY issuetype DESC"
+
+# Fetch all the issues available for the given jql query
+results = get_all_jira_issues_for_query(jira_url, jql)
+
+changelog = {}
+# Loop through and add all results
+results.each do |issue|
+  key = issue['key']
+  summary = issue['fields']['summary']
+  type = issue['fields']['issuetype']['name']
+
+  changelog_entry = "[#{key}] - #{summary}"
+
+  if !changelog[type].nil?
+    changelog[type] << changelog_entry
+  else
+    changelog[type] = [changelog_entry]
+  end
+end
+
+# Merge the new updates and the existing changelog
+tmpfile = Tempfile.open('parquet-cpp.changelog')
+begin
+  tmpfile.puts "Parquet C++ #{version}", "-" * 80
+  changelog.keys.sort.each do |type|
+    tmpfile.puts "## #{type}"
+    changelog[type].each { |entry| tmpfile.puts "    * #{entry}" }
+    tmpfile.puts ""
+  end
+  # Append all the existing CHANGELOG entries and write the new CHANGELOG file
+  tmpfile.puts ""
+  changelog_file = File.join(base_dir, 'CHANGELOG')
+  tmpfile.write File.read(changelog_file) if File.exist?(changelog_file)
+  tmpfile.rewind
+  tmpfile.flush
+  File.open(changelog_file,"w+") {|f|  f.write(tmpfile.read) }
+ensure
+  tmpfile.close
+  tmpfile.unlink
+end

http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/release/release
----------------------------------------------------------------------
diff --git a/dev/release/release b/dev/release/release
new file mode 100755
index 0000000..61c294f
--- /dev/null
+++ b/dev/release/release
@@ -0,0 +1,250 @@
+#!/bin/bash
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#
+# This script is used to publish the official release after a successful
+# vote of a release-candidate.
+
+set -e
+set -o nounset
+
+parquetcpp_git_web_url='https://git-wip-us.apache.org/repos/asf?p=parquet-cpp.git'
+parquet_svn_dist_url='https://dist.apache.org/repos/dist/release/parquet'
+parquet_svn_dev_dist_url='https://dist.apache.org/repos/dist/dev/parquet'
+
+function print_help_and_exit {
+cat <<EOF
+Apache Parquet C++ release tool.
+
+Usage: $0 [-h] [-r #] [-p | publish]
+
+  -h   Print this help message and exit
+  -r   Release candidate number (default: 0)
+  -p   Publish (default: dry-run (does not publish anything))
+EOF
+exit 0
+}
+
+publish=0
+rc_tag_version=0
+while getopts ":hl:r:p" opt; do
+  case $opt in
+    r)
+      rc_tag_version=${OPTARG}
+      ;;
+    p)
+      publish=1
+      ;;
+    h)
+      print_help_and_exit
+      ;;
+    *  )
+      echo "Unknown option: -$OPTARG"
+      print_help_and_exit
+      ;;
+  esac
+done
+
+shift $(($OPTIND - 1))
+if [[ "${1:-dry-run}" == "publish" ]]; then
+  publish=1
+fi
+
+# Update local repository
+git fetch --all -q
+git fetch --tags -q
+
+# Ensure that a signing key is available
+if [[ -z "`git config user.signingkey`" ]]; then
+  cat <<EOF
+Error: No GPG signing key can be found within gitconfig.
+
+To configure one, find your code signing key's ID with
+
+   gpg --list-secret-keys
+
+Then configure it as the signing key for this repository with
+
+   git config --global user.signingkey YOUR_KEY_ID
+EOF
+  exit 1
+fi
+
+# Set the base dir for the script to be the top level of the repository
+base_dir=$(git rev-parse --show-toplevel)
+# Verify that this is a clean repository
+if [[ -n "`git status --porcelain`" ]]; then
+  echo "ERROR: Please run from a clean master."
+  exit 1
+elif [[ "`git rev-parse --abbrev-ref HEAD`" == "master" ]]; then
+  echo "ERROR: This script must be run from the released branch."
+  exit 1
+fi
+
+if [[ "$base_dir" != "$PWD" ]]; then
+  echo "Warrning: This script must be run from the root of the repository ${base_dir}"
+  cd $base_dir
+fi
+
+# Make sure that this is not on a snapshot release
+current_version=$(cat .parquetcppversion | tr '[a-z]' '[A-Z]')
+if [[ $current_version =~ .*-SNAPSHOT ]]; then
+  echo "ERROR: .parquetcppversion can not be a 'SNAPSHOT', it is ${current_version}"
+  exit 1
+else
+  major=`echo $current_version | cut -d. -f1`
+  minor=`echo $current_version | cut -d. -f2`
+  patch=`echo $current_version | cut -d. -f3 | cut -d- -f1`
+
+  current_version="${major}.${minor}.${patch}"
+fi
+
+previous_version_tag="${current_version}-rc${rc_tag_version}"
+current_version_tag="rel/${current_version}"
+
+# Make sure the tag does not exist
+if [[ $(git ls-remote --exit-code --tags origin refs/tags/${current_version} >/dev/null
2>&1) == 0 ]]; then
+  echo "ERROR: ${current_version} tag exists."
+  exit 1
+fi
+
+# All check are now complete, before we start alert if we are in dry-run
+if [[ $publish == 0 ]]; then
+  echo "Performing dry-run"
+fi
+
+dist_name="apache-parquet-cpp-${current_version}"
+
+if [[ $publish == 1 ]]; then
+  # Create release branch and tag and push them to the origin
+  echo "Creating ${current_version_tag} staging branch"
+  git checkout -b "stage_${current_version_tag}"
+
+  # Increment the version and create a branch
+  echo ${current_version} > .parquetcppversion
+  git add .parquetcppversion
+  git commit -m "Updating .parquetcppversion to release version ${current_version}."
+  git tag -s "${current_version_tag}" \
+    -m "Apache Parquet C++ ${current_version} release" HEAD
+  git push origin "${current_version_tag}"
+fi
+
+dist_dir=${base_dir}/dist
+rc_dir=${dist_dir}/rc
+release_dir=${dist_dir}/${current_version}
+mkdir -p ${rc_dir}
+cd ${dist_dir}
+
+# Checkout the release candidate
+svn co ${parquet_svn_dev_dist_url}/${previous_version_tag} ${rc_dir} >/dev/null 2>&1
+
+if [[ $publish == 1 ]]; then
+  echo "Publishing the release"
+  # Make the release dist directory and check it out
+  svn mkdir ${parquet_svn_dist_url}/${current_version} -m "parquet-cpp-${current_version}
release"
+  svn co --depth=empty ${parquet_svn_dist_url}/${current_version} ${release_dir}
+else
+  mkdir ${release_dir}
+fi
+
+cd ${release_dir}
+
+# Rename the .parquetcppversion from -RC[:digit:] to release current_version and repackage
the release
+tar -xzf ${rc_dir}/apache-parquet-cpp-*.tar.gz
+mv apache-parquet-cpp-* ${dist_name}
+echo ${current_version} > ${dist_name}/.parquetcppversion
+tar -czf ${dist_name}.tar.gz ${dist_name}
+rm -rf ${dist_name}
+
+# Sign the tarball.
+echo "Signing the distribution"
+gpg --armor --output ${dist_name}.tar.gz.asc --detach-sig ${dist_name}.tar.gz
+
+# Create the checksums
+echo "Creating checksums"
+gpg --print-md MD5 ${dist_name}.tar.gz > ${dist_name}.tar.gz.md5
+shasum ${dist_name}.tar.gz > ${dist_name}.tar.gz.sha
+
+if [[ $publish == 1 ]]; then
+  # Commit the release
+  svn add ${dist_name}*
+  svn ci -m "parquet-cpp-${current_version} release"
+fi
+
+cd ${base_dir}
+
+echo
+echo "Done creating the release. The following draft email has been created"
+echo "to send to the dev@parquet.apache.org mailing list."
+echo
+
+# Create the email template for the release to be sent to the mailing lists.
+MESSAGE=$(cat <<__EOF__
+To: dev@parquet.apache.org
+Subject: [RESULT][VOTE] Release Apache Parquet C++ ${current_version} RC${rc_tag_version}
+
+All,
+The vote to accept Apache Parquet C++ ${current_version} RC${rc_tag_version}
+as the official Apache Parquet C++ ${current_version} release has passed.
+
+
++1 (Binding)
+------------------------------
+
+
++1 (Non-binding)
+------------------------------
+
+
+There were no 0 or -1 votes. Thank you to all who helped make this release.
+
+
+Parquet C++ ${current_version} includes the following:
+---
+The CHANGELOG for the release is available at:
+${parquetcpp_git_web_url}&f=CHANGELOG&hb=${current_version_tag}
+
+The tag used to create the release with is ${current_version_tag}:
+${parquetcpp_git_web_url}&a=shortlog&h=refs/tags/${current_version_tag}
+
+The release is available at:
+${parquet_svn_dist_url}/${current_version}/${dist_name}.tar.gz
+
+The MD5 checksum of the release can be found at:
+${parquet_svn_dist_url}/${current_version}/${dist_name}.tar.gz.md5
+
+The signature of the release can be found at:
+${parquet_svn_dist_url}/${current_version}/${dist_name}.tar.gz.asc
+
+The GPG key used to sign the release are available at:
+${parquet_svn_dist_url}/KEYS
+
+__EOF__
+)
+echo "--------------------------------------------------------------------------------"
+echo
+echo "${MESSAGE}"
+echo
+echo "--------------------------------------------------------------------------------"
+echo
+
+# Print reset instructions if this was a dry-run
+if [[ $publish == 0 ]]; then
+  echo
+  echo "This is a dry run, nothing has been published."
+  echo
+  echo "To clean up run: rm -rf ${dist_dir}"
+fi
+
+exit 0

http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/release/release-candidate
----------------------------------------------------------------------
diff --git a/dev/release/release-candidate b/dev/release/release-candidate
new file mode 100755
index 0000000..9fe6027
--- /dev/null
+++ b/dev/release/release-candidate
@@ -0,0 +1,308 @@
+#!/bin/bash
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#
+# This script is used to create a release candidate. It will update the current
+# .parquetcppversion as well as creates a tag for the new release candidate and
+# publishes the source distribution and signatures to be voted on.
+#
+#   master~1 (0.5.0-snapshot) ----- master (0.6.0-snapshot)
+#                             \---- 0.5.0 (0.5.0)
+#
+# A email template will be generated after successfully generating a release
+# candidate which will need to be sent to the dev@ and private@ mailing lists.
+#
+set -o errexit
+set -o nounset
+
+rc_number=0
+override_version=''
+parquetcpp_git_web_url='https://git-wip-us.apache.org/repos/asf?p=parquet-cpp.git'
+parquet_svn_dist_url='https://dist.apache.org/repos/dist/dev/parquet'
+
+function print_help_and_exit {
+cat <<EOF
+Apache Parquet C++ release candidate tool.
+
+Usage: $0 [-h] [-l p|m|M] [-r #] [-p | publish]
+
+  -h  Print this help message and exit
+  -l  Increment level, must be one of:
+        p, patch (default)
+        m, minor
+        M, major
+  -v  Override the existing version in .parquetcppversion
+  -r  Release candidate number (default: 0)
+  -p  Publish the release candidate (default: dry-run, does not publish anything)
+EOF
+exit 0
+}
+
+publish=0
+increment_level="patch"
+rc_number=0
+while getopts ":hl:v:r:p" opt; do
+  case $opt in
+    l)
+      case ${OPTARG} in
+        'p' | 'patch') increment_level='patch' ;;
+        'm' | 'minor') increment_level='minor' ;;
+        'M' | 'major') increment_level='major' ;;
+         *) echo 'Unknown increment level'; exit 1 ;;
+      esac
+      ;;
+    r)
+      rc_number=${OPTARG}
+      ;;
+    p)
+      publish=1
+      ;;
+    h)
+      print_help_and_exit
+      ;;
+    v)
+      override_version=${OPTARG}
+      ;;
+    *  )
+      echo "Unknown option: -$OPTARG"
+      print_help_and_exit
+      ;;
+  esac
+done
+
+shift $(($OPTIND - 1))
+if [[ "${1:-dry-run}" == "publish" ]]; then
+  publish=1
+fi
+
+# Update local repository
+git fetch --all -q
+git fetch --tags -q
+
+# Verify that this is a clean repository
+if [[ -n "`git status --porcelain`" ]]; then
+  echo "ERROR: Please run from a clean git repository."
+  exit 1
+elif [[ "`git rev-parse --abbrev-ref HEAD`" != "master" ]]; then
+  echo "ERROR: This script must be run from master."
+  exit 1
+fi
+
+if [[ ! -f .parquetcppversion ]]; then
+  echo "Warrning: This script must be run from the root of the repository"
+  exit 1
+fi
+
+# Calculate the new version string
+current_version=$(cat .parquetcppversion | tr '[a-z]' '[A-Z]')
+if ! [[ $current_version =~ .*-SNAPSHOT ]]; then
+  echo "ERROR: .parquetcppversion is required to contain 'SNAPSHOT', it is ${current_version}"
+  exit 1
+else
+  if [[ $override_version != "" ]]; then
+    current_version=$override_version
+  fi
+
+  major=`echo $current_version | cut -d. -f1`
+  minor=`echo $current_version | cut -d. -f2`
+  patch=`echo $current_version | cut -d. -f3 | cut -d- -f1`
+
+  current_version="${major}.${minor}.${patch}"
+
+  if [[ $increment_level == "patch" ]]; then
+    new_master_version="${major}.${minor}.$((patch + 1))"
+  elif [[ $increment_level == "minor" ]]; then
+    new_master_version="${major}.$((minor + 1)).0"
+  elif [[ $increment_level == "major" ]]; then
+    new_master_version="$((major + 1)).0.0"
+  else
+    echo "Unknown release increment ${increment_level}"
+    exit 1
+  fi
+
+  new_snapshot_version="${new_master_version}-SNAPSHOT"
+fi
+
+# Add the rc tag to the current version
+rc_version="${current_version}-rc${rc_number}"
+rc_version_tag="rel/${rc_version}"
+
+echo
+echo "Generating release candidate ${rc_version}"
+echo
+
+# Make sure the tag does not exist
+if git rev-parse ${rc_version_tag} >/dev/null 2>&1; then
+  echo "ERROR: tag ${rc_version_tag} exists."
+  exit 1
+fi
+
+# Reset instructions
+current_git_rev=$(git rev-parse HEAD)
+function print_reset_instructions {
+cat <<EOF
+To roll back your local repo you will need to run:
+
+  git checkout master
+  git reset --hard ${current_git_rev}
+  git tag -D ${rc_version_tag}
+EOF
+}
+
+# If anything goes wrong from here then print roll back instructions before exiting.
+function print_rollback_instructions {
+  echo "ERROR: Looks like something has failed while creating the release candidate."
+  print_reset_instructions
+}
+trap print_rollback_instructions EXIT
+
+# All check are now complete, before we start alert if we are in dry-run
+if [[ $publish == 0 ]]; then
+  echo "Performing dry-run, run with '-p' when you are ready to run and publish a release
candidate"
+fi
+
+# This should be a clean repo we are working against. Run clean just to ensure it is.
+git clean -fdxq
+
+echo "Generating changelog"
+./dev/release/changelog $current_version
+git add CHANGELOG
+git commit -m "Updating CHANGELOG for ${current_version} release."
+
+echo "Committing updated .parquetcppversion on master"
+echo $new_snapshot_version > .parquetcppversion
+git add .parquetcppversion
+git commit -m "Incrementing snapshot version to ${new_snapshot_version}."
+
+echo "Creating ${rc_version} staging branch"
+git checkout -b "stage_${rc_version_tag}"
+
+echo "Updating .parquetcppversion on staging branch"
+# Increment the version and create a branch
+echo ${rc_version} > .parquetcppversion
+git add .parquetcppversion
+git commit -m "Updating .parquetcppversion to ${rc_version}."
+
+# Build the source distribution from the new branch
+echo "Building the source distribution"
+dist_dir=dist
+dist_name="apache-parquet-cpp-${rc_version}"
+
+mkdir -p ${dist_dir}
+git archive --prefix=${dist_name}/ -o ${dist_dir}/${dist_name}.tar.gz HEAD
+pushd ${dist_dir}
+  # Sign the tarball.
+  echo "Signing the distribution"
+  gpg --armor --output ${dist_name}.tar.gz.asc --detach-sig ${dist_name}.tar.gz
+
+  # Create the checksums
+  echo "Creating checksums"
+  gpg --print-md MD5 ${dist_name}.tar.gz > ${dist_name}.tar.gz.md5
+  shasum ${dist_name}.tar.gz > ${dist_name}.tar.gz.sha
+popd
+
+parquet_svn_rc_url="${parquet_svn_dist_url}/${rc_version}"
+
+# Publish release candidate to svn and commit and push the new git tag
+if [[ $publish == 1 ]]; then
+  echo "Publishing release candidate to ${parquet_svn_rc_url}"
+  svn mkdir ${parquet_svn_rc_url} -m "parquet-cpp-${current_version} release candidate ${rc_version_tag}"
+  svn co --depth=empty ${parquet_svn_rc_url} ${dist_dir}
+  pushd ${dist_dir}
+  svn add ${dist_name}*
+  svn ci -m "parquet-cpp-${current_version} release candidate ${rc_version_tag}"
+  popd
+
+  echo "Creating tag ${rc_version_tag}"
+  git tag -s ${rc_version_tag} \
+    -m "Apache Parquet C++ ${current_version} release candidate ${rc_version_tag}"
+  git push origin "${rc_version_tag}"
+
+  echo "Pushing updated .parquetcppversion to master"
+  git checkout master
+  git push origin master
+fi
+
+echo "Done creating the release candidate. The following draft email has been created"
+echo "to send to the dev@parquet.apache.org mailing list"
+echo
+
+# Create the email template for the release candidate to be sent to the mailing lists.
+if [[ $(uname) == Darwin ]]; then
+  vote_end=$(date -v+3d)
+else
+  vote_end=$(date -d+3days)
+fi
+
+MESSAGE=$(cat <<__EOF__
+To: dev@parquet.apache.org
+Subject: [VOTE] Release Apache Parquet C++ ${current_version} RC${rc_number}
+
+All,
+
+I propose that we accept the following release candidate as the official
+Apache Parquet C++ ${current_version} release.
+
+Parquet C++ ${rc_version} includes the following:
+---
+The RELEASE NOTES for the release are available at:
+${parquetcpp_git_web_url}&f=RELEASE-NOTES.md&hb=${rc_version_tag}
+
+The CHANGELOG for the release is available at:
+${parquetcpp_git_web_url}&f=CHANGELOG&hb=${rc_version_tag}
+
+The tag used to create the release candidate is:
+${parquetcpp_git_web_url};a=shortlog;h=refs/tags/${rc_version_tag}
+
+The release candidate is available at:
+${parquet_svn_rc_url}/${dist_name}.tar.gz
+
+The MD5 checksum of the release candidate can be found at:
+${parquet_svn_rc_url}/${dist_name}.tar.gz.md5
+
+The signature of the release candidate can be found at:
+${parquet_svn_rc_url}/${dist_name}.tar.gz.asc
+
+The GPG key used to sign the release are available at:
+${parquet_svn_dist_url}/KEYS
+
+Please download, verify, and test.
+
+The vote will close on ${vote_end}
+
+[ ] +1 Release this as Apache Parquet C++ ${current_version}
+[ ] +0
+[ ] -1 Do not release this as Apache Parquet C++ ${current_version} because...
+
+__EOF__
+)
+
+echo "--------------------------------------------------------------------------------"
+echo
+echo "${MESSAGE}"
+echo
+echo "--------------------------------------------------------------------------------"
+echo
+
+# Print reset instructions if this was a dry-run
+if [[ $publish == 0 ]]; then
+  echo
+  echo "This was a dry run, nothing has been published."
+  echo
+  print_reset_instructions
+fi
+
+# Unset error message handler and exit
+trap '' EXIT
+exit 0

http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/release/verify-release-candidate
----------------------------------------------------------------------
diff --git a/dev/release/verify-release-candidate b/dev/release/verify-release-candidate
new file mode 100755
index 0000000..efcc111
--- /dev/null
+++ b/dev/release/verify-release-candidate
@@ -0,0 +1,92 @@
+#!/usr/bin/env bash
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#
+# Downloads a release candidate and verifies that it passes binary
+# verification (signature and checksums) and test suites.
+#
+set -ex
+
+HERE=$(cd `dirname "${BASH_SOURCE[0]:-$0}"` && pwd)
+
+parquet_svn_dev_dist_url='https://dist.apache.org/repos/dist/dev/parquet'
+
+download_dist_file() {
+  curl -f -O ${parquet_svn_dev_dist_url}/$1
+}
+
+download_rc_file() {
+  download_dist_file ${verify_version}/$1
+}
+
+import_gpg_keys() {
+  download_dist_file KEYS
+  gpg --import KEYS
+}
+
+fetch_archive() {
+  local dist_name=$1
+  download_rc_file ${dist_name}.tar.gz
+  download_rc_file ${dist_name}.tar.gz.asc
+  download_rc_file ${dist_name}.tar.gz.md5
+  download_rc_file ${dist_name}.tar.gz.sha
+  gpg --verify ${dist_name}.tar.gz.asc ${dist_name}.tar.gz
+  gpg --print-md MD5 ${dist_name}.tar.gz | diff - ${dist_name}.tar.gz.md5
+  shasum ${dist_name}.tar.gz | diff - ${dist_name}.tar.gz.sha
+}
+
+run_tests() {
+  # Build
+  source ./setup_build_env.sh
+  mkdir -p build
+  cd build
+  cmake -DPARQUET_ARROW=ON ..
+  make -j5
+
+  # Test
+  export PARQUET_TEST_DATA=$(dirname $(pwd))/data
+  make unittest
+}
+
+setup_tempdir() {
+  cleanup() {
+    rm -fr "$TMPDIR"
+  }
+  trap cleanup EXIT
+  TMPDIR=$(mktemp -d -t "$1.XXXXX")
+}
+
+case $# in
+  1) verify_version="$1"
+     ;;
+
+  *) echo "Usage: $0 RC_VERSION"
+     exit 1
+     ;;
+esac
+
+setup_tempdir "parquet-cpp-$verify_version"
+echo "Working in sandbox $TMPDIR"
+cd $TMPDIR
+
+import_gpg_keys
+
+dist_name="apache-parquet-cpp-${verify_version}"
+fetch_archive $dist_name
+tar xvzf ${dist_name}.tar.gz
+cd ${dist_name}
+run_tests
+
+echo 'Release candidate looks good!'
+exit 0


Mime
View raw message