From commits-return-881-archive-asf-public=cust-asf.ponee.io@mnemonic.apache.org Mon Feb 15 22:37:22 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-he-de.apache.org (mxout1-he-de.apache.org [95.216.194.37]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id BD4FF18064D for ; Mon, 15 Feb 2021 23:37:22 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-he-de.apache.org (ASF Mail Server at mxout1-he-de.apache.org) with SMTP id 1B22F62A34 for ; Mon, 15 Feb 2021 22:37:22 +0000 (UTC) Received: (qmail 52286 invoked by uid 500); 15 Feb 2021 22:37:21 -0000 Mailing-List: contact commits-help@mnemonic.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@mnemonic.apache.org Delivered-To: mailing list commits@mnemonic.apache.org Received: (qmail 52277 invoked by uid 99); 15 Feb 2021 22:37:21 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Feb 2021 22:37:21 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id 33C79820FF; Mon, 15 Feb 2021 22:37:21 +0000 (UTC) Date: Mon, 15 Feb 2021 22:37:21 +0000 To: "commits@mnemonic.apache.org" Subject: [mnemonic-site] branch master updated: MNEMONIC-587: Update development workflow on project website MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <161342864114.30300.14877222537234434623@gitbox.apache.org> From: yzhao@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: mnemonic-site X-Git-Refname: refs/heads/master X-Git-Reftype: branch X-Git-Oldrev: 1d2e9c5159589957f0908b0a278cc4e116645b57 X-Git-Newrev: 306701c316ee8d9cdeb61da4058260d4da0c8cd6 X-Git-Rev: 306701c316ee8d9cdeb61da4058260d4da0c8cd6 X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. yzhao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/mnemonic-site.git The following commit(s) were added to refs/heads/master by this push: new 306701c MNEMONIC-587: Update development workflow on project website new c240fe0 Merge pull request #6 from rhythmwind/issue-587 306701c is described below commit 306701c316ee8d9cdeb61da4058260d4da0c8cd6 Author: Zane AuthorDate: Wed Feb 10 22:49:48 2021 -0800 MNEMONIC-587: Update development workflow on project website Signed-off-by: Zane --- src/_docs/build.md | 89 ++++++++++++++------------------ src/_docs/devsetup.md | 64 +++++++++++++++-------- src/_docs/submitchanges.md | 124 ++++++++++++++++++++++++--------------------- 3 files changed, 148 insertions(+), 129 deletions(-) diff --git a/src/_docs/build.md b/src/_docs/build.md index 1acc495..f8b6c27 100644 --- a/src/_docs/build.md +++ b/src/_docs/build.md @@ -4,10 +4,9 @@ title: Project Build permalink: /docs/build.html --- -### How to build it ? - -Please see the file LICENSE for information on how this library is licensed. +### How to build it? +Please see the file LICENSE for information on how this library is licensed. Below you will find descriptions for each module. * **mnemonic-core** -- the submodule project for core * **mnemonic-collections** -- the submodule project for generic collections @@ -22,69 +21,55 @@ Please see the file LICENSE for information on how this library is licensed. * **mnemonic-hadoop/mnemonic-hadoop-mapreduce** -- the submodule project for Apache Hadoop mapreduce computing * **mnemonic-spark/mnemonic-spark-core** -- the submodule project for Apache Spark durable computing -To build this library, you may need to install some required packages on the build system: +To build this library without Docker image, you will need to install some required packages on the build system. Otherwise, you can build the project with Docker image generated [previously.](devsetup.html){:target="_blank"} -* **Maven** -- the building tool v3.2.1 or above [Required] -* **NVML** -- the NVM library (Please compile this library that was revised with 630862e82f) (http://pmem.io) [Optional if mnemonic-nvml-vmem-service/mnemonic-nvml-pmem-service are excluded, e.g. on MacOSX] -* **JDK** -- the Java Develop Kit 7-8 (please properly configure JAVA_HOME) [Required] -* **PMFS** -- the PMFS should be properly installed and configured on Linux system if you want to simulate read latency [Optional] -* **PMalloc** -- a supported durable memory native library(Latest) at https://github.com/NonVolatileComputing/pmalloc.git [Optional if mnemonic-pmalloc-service is excluded] +* **Maven** -- the building tool v3.2.1 or above. Optional if you choose to use Gradle. +* **Gradle** -- the building tool v6.7.1 or above. Optional if you choose to use Maven. +* **NVML** -- the [NVM library.](http://pmem.io){:target="_blank"} Please compile this library that was revised with 630862e82f. +* **JDK** -- the Java Develop Kit 7-8. Please properly configure JAVA_HOME in environment variables. +* **PMFS** -- the PMFS should be properly installed and configured on Linux system if you want to simulate read latency. Optional if you choose to use it for simulation purpose. +* **PMalloc** -- a supported durable memory native library [here.](https://github.com/NonVolatileComputing/pmalloc.git){:target="_blank"} -Once the build system is setup, this Library is built using this command at the top level: -```bash - $ git clean -xdf # if pull from a git repo. - $ mvn clean package install -``` +*** +You can use following commmand lines to install build tools for this project. -To exclude a customized memory service for your platform e.g. OSX, note that if you excluded one or both memory services, some or all testcases/examples will fail since their dependent memory services are unavailable. ```bash - $ git clean -xdf # if pull from a git repo. - $ mvn -pl '!mnemonic-memory-services/mnemonic-nvml-vmem-service' clean package install + # ---- install build tools ---- + $ sudo apt-get -y update && \ + apt-get install -y default-jdk cmake check git pkg-config autoconf man build-essential gcc g++ uuid-dev pandoc devscripts flex doxygen maven + $ sudo apt-get install -y libndctl-dev libpmem-dev libvmem-dev libpmemobj-dev + $ sudo apt-get clean + # ---- update environment variables ---- + $ export JAVA_HOME=/usr/lib/jvm/default-java + $ export PATH=$JAVA_HOME/bin:$PATH + $ source ~/.profile + # ---- install pmalloc ---- + $ git clone https://github.com/NonVolatileComputing/pmalloc.git && \ + cd pmalloc && mkdir build && cd build && cmake .. && make && make install + # ---- clone Mnemonic ---- + $ git clone https://github.com/apache/mnemonic.git && \ + cd mnemonic && mvn clean package install ``` +*** -To install this package to local repository (required to run examples and testcases): -```bash - $ mvn clean install -``` - +Once the build system is setup, this Library is built using this command at the top level: -To run an example: ```bash - $ # requires 'vmem' memory service to run, please refer to the code of test cases for more examples. - $ mvn exec:exec -Pexample -pl mnemonic-examples + $ git clean -xdf # if pull from a git repo. + # ------ For Maven --------- + $ mvn clean install + # ------ For Gradle -------- + $ ./gradlew clean + $ ./gradlew build -x test ``` +To exclude a customized memory service for your platform e.g. OSX, note that if you excluded one or both memory services, some or all testcases/examples will fail since their dependent memory services are unavailable. -To run several test cases: ```bash - $ # a testcase for module "mnemonic-core" that requires 'pmalloc' memory service to pass - $ mvn -Dtest=DurablePersonNGTest test -pl mnemonic-core -DskipTests=false - - $ # a testcase for module "mnemonic-core" that requires 'pmalloc' memory service to pass - $ mvn -Dtest=NonVolatileMemAllocatorNGTest test -pl mnemonic-core -DskipTests=false - - $ # a testcase for module "mnemonic-core" that requires 'vmem' memory service to pass - $ mvn -Dtest=VolatileMemAllocatorNGTest test -pl mnemonic-core -DskipTests=false - - $ # a testcase for module "mnemonic-core" that requires 'vmem memory service to pass - $ mvn -Dtest=MemClusteringNGTest test -pl mnemonic-core -DskipTests=false - - $ # a testcase for module "mnemonic-collection" that requires 'pmalloc' memory service to pass - $ mvn -Dtest=DurableSinglyLinkedListNGTest test -pl mnemonic-collections -DskipTests=false - - $ # a testcase for module "mnemonic-collection" that requires 'pmalloc' memory service to pass - $ mvn -Dtest=DurablePersonNGTest test -pl mnemonic-collections -DskipTests=false - - $ # a testcase for module "mnemonic-computing-services/mnemonic-utilities-service" that requires 'pmalloc' memory service to pass - $ mvn -Dtest=DurableSinglyLinkedListNGPrintTest test -pl mnemonic-computing-services/mnemonic-utilities-service -DskipTests=false - - $ # a testcase for module "mnemonic-computing-services/mnemonic-utilities-service" that requires 'pmalloc' memory service to pass - $ mvn -Dtest=DurableSinglyLinkedListNGSortTest test -pl mnemonic-computing-services/mnemonic-utilities-service -DskipTests=false - - $ # a testcase for module "mnemonic-hadoop/mnemonic-hadoop-mapreduce" that requires 'pmalloc' memory service to pass - $ mvn -Dtest=MneMapreducePersonDataTest test -pl mnemonic-hadoop/mnemonic-hadoop-mapreduce -DskipTests=false + $ git clean -xdf # if pull from a git repo. + $ mvn -pl '!mnemonic-memory-services/mnemonic-nvml-vmem-service' clean package install ``` diff --git a/src/_docs/devsetup.md b/src/_docs/devsetup.md index 6b938d5..fcf98b2 100644 --- a/src/_docs/devsetup.md +++ b/src/_docs/devsetup.md @@ -4,38 +4,62 @@ title: Development Setup permalink: /docs/devsetup.html --- -This document provides a best practice for general development setup, -we suggest to use this practice on your first try for code contribution. +This document provides a best practice for general development setup and code distribution. Please practice on your computer before trying for code contribution. -It is based on Docker and simple for preparation, no root privilege needed except for Docker and Git installation. +*** + +Pre-requisites: + +* Install Git on your system - [Git-scm.](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git){:target="_blank"} + +* Register Github Account if you dont have one already. You can register your Github acocunt [here.](http://github.com){:target="_blank"} + +* Obtain Docker on your system. You can get docker from [here.](https://www.docker.com/){:target="_blank"} + +* (Optional) Install IDE on your operating system. e.g. [Visual Studio Code](https://code.visualstudio.com/download){:target="_blank"} or [IntelliJ](https://www.jetbrains.com/idea/){:target="_blank"} + +*** + +* Step 1 - Configure Git tool if you are behind the firewall. [Show Me How](https://gist.github.com/evantoli/f8c23a37eb3558ab8765){:target="_blank"} + +* Step 2 - Fork the following Mnemonic project to your own Github account. [Project Link](https://github.com/apache/mnemonic){:target="_blank"} + +* Step 3 - Clone the forked Mnemonic project from your own Github account by running the command below. [Show Me How](https://docs.github.com/en/free-pro-team@latest/github/creating-cloning-and-archiving-repositories/cloning-a-repository){:target="_blank"} -* Install Git on your system [Git-scm](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git){:target="_blank"} -* [First-Time Git Setup](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup){:target="_blank"} ```bash - $ git config --global user.name "" - $ git config --global user.email "" + $ git clone git@github.com:[your account id]/mnemonic.git ``` -* Install Docker on your system [Docker Official Website](https://www.docker.com/){:target="_blank"} -* Install IDE on your Linux/MacOS/Windows box [IntelliJ IDEA Official Website](https://www.jetbrains.com/idea/){:target="_blank"} -* Register an account on Github for yourself if not yet have one -* Log into Github and fork [Apache Mnemonic repository](https://github.com/apache/mnemonic){:target="_blank"}. - [Fork and Set up Git](https://help.github.com/articles/fork-a-repo/){:target="_blank"} -* Configure Apache Mnemonic repo. as upstream remote in your local repository + +* Step 4 - Configure a remote upstream from Github for upstream sync by running the command below. [Show Me How](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/configuring-a-remote-for-a-fork){:target="_blank"} + ```bash - $ cd $ git remote add upstream https://github.com/apache/mnemonic.git ``` -* Prepare Mnemonic Docker image on you system + +* Step 5 - Setting your commit email address and name for local Mnemonic repo by running the command below. [Show Me How](https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-your-github-user-account/setting-your-commit-email-address){:target="_blank"} + + +```bash + $ cd mnemonic + $ git config user.email “[your apache id]@ email@example.comapache.org” + $ git config user.name “[your name]” +``` + +* Step 6 - Prepare Mnemonic Docker image on your own system by running one of following command below. + ```bash + $ # --------- For CentOS --------- $ docker build -t mnedev https://github.com/apache/mnemonic.git#:docker/docker-CentOS - $ # --------- OR --------- + $ # --------- For Ubuntu --------- $ docker build -t mnedev https://github.com/apache/mnemonic.git#:docker/docker-Ubuntu ``` -* Mount your host repo. to Docker container, this following command would hide the original Mnemonic project folder in container + +* Step 7 - Mount your host repository to Docker container, this following command would hide the original Mnemonic project folder in container. + ```bash $ docker run -it -v :/ws/mnemonic mnedev ``` -* Open Mnemonic project in the IDE -After setup, you are able to edit project code on your host while compiling and test your code in isolated Docker environment. -Please leave enough space for docker based compiling and testing (>30G for test running) +Congratulation! You have successfully completed the development setup. You are now ready to edit the project code on your host while compiling and test in isolated Docker envinroment. Please make sure you have enough space for docker based compiling and testing (>30G for test running). + +Please refer to [Submit Changes](submitchanges.html){:target="_blank"} page if you would like to contribute to this project! \ No newline at end of file diff --git a/src/_docs/submitchanges.md b/src/_docs/submitchanges.md index 4722717..d9e7dcf 100644 --- a/src/_docs/submitchanges.md +++ b/src/_docs/submitchanges.md @@ -4,60 +4,70 @@ title: Submit Changes permalink: /docs/submitchanges.html --- -After [Development Setup](devsetup.html), you can follow the steps below to -submit your code/comments and document changes. - -* Make sure the changes are linked to your [Apache JIRA](https://issues.apache.org/jira/projects/MNEMONIC){:target="_blank"} ticket -* Review your changes according to the JIRA ticket in question -* Building project in the Docker container -```bash - ## for Docker container - # cd /ws/mnemonic - # mvn clean install -``` -* Resolve any problems found by building process -* Run your test cases specific to your changes, please refer to bin/test.conf -* Resolve any problems found by your test cases -* Add your test cases to *bin/test.conf* -* Commit the changes to your local repository in IDE
- for Intellij IDEA, please refer to [Committing Changes to a Local Git Repository](https://www.jetbrains.com/help/idea/commit-and-push-changes.html#commit){:target="_blank"}
- Please use JIRA id and title to fill up the comment of commit
- for example -```bash - ## for host - # git commit -m "MNEMONIC-301: Improve the documentation of architecture page" -``` -* Run all test cases if your changes could impact existing functionalities
-Note: Please make sure enough available space for test data generation (>30G) -```bash - ## for Docker container - # cd /ws/mnemonic - # bin/runall.sh -``` -* Resolve any problems found by test cases and then **amend** your previous commit in IDE
- for Intellij IDEA, please refer to [Committing Changes to a Local Git Repository](https://www.jetbrains.com/help/idea/commit-and-push-changes.html#commit){:target="_blank"} -```bash - ## for host - # git commit --amend -``` -* Make sure there is only 1 local commit for each JIRA ticket, otherwise please squash them into one -* Rebase your codebase before pushing local commits to your remote repository - 1. fetch update from remote [Fetching Changes](https://www.jetbrains.com/help/idea/sync-with-a-remote-repository.html#fetch){:target="_blank"} - ```bash - ## for host - # git fetch -a - ``` - 2. Rebase your changes onto upstream/master [Rebasing Branches](https://www.jetbrains.com/help/idea/apply-changes-from-one-branch-to-another.html#rebase-branch){:target="_blank"} - ```bash - ## for host - # git rebase upstream/master - ``` -* Push local commits to your remote repository [Pushing Changes](https://www.jetbrains.com/help/idea/commit-and-push-changes.html#push){:target="_blank"} -```bash - ## for host - # git push origin - ## NOTE: use force push if your amended commit has already been pushed -``` -* Create a PR in Github [Creating a pull request](https://help.github.com/articles/creating-a-pull-request/){:target="_blank"} -* Fix any issues come from community review -* Change the JIRA ticket status to RESOLVED if the button exists + +After [Development Setup](devsetup.html), you can follow the steps below to submit your code/comments and document changes. + +*** + +* Step 1 - Sync your forked Mnemonic project from upstream. [Show Me How](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/syncing-a-fork){:target="_blank"} + + ** Please keep the master branch untouched, otherwise we can't keep it exactly synced. ** + +```bash + $ git fetch upstream + $ git checkout master + $ git merge upstream/master + $ git push +``` + +* Step 2 - Take over an interested issue/task by clicking the “Start Progress” button in the Apache issue web portal. [Project Issue Portal](https://issues.apache.org/jira/projects/MNEMONIC/){:target="_blank"} + +* Step 3 - Create a new branch for your selected issue/task. + +```bash + $ git checkout master + $ git checkout -b issue-[issue id] +``` + +* Step 4 - Apply patch files if any. + +```bash + $ git apply [patch file] +``` + +* Step 5 - Fix/implement the selected issue/task on the specific branch.Please ensure you are working on the create branch, e.g. issue-548, using the following command to confirm it + +```bash + $ git status +``` + +* Step 6 - Stage your new code or code changes before stopping work on it. + +```bash + $ git add [your changed/new files] +``` + +* Step 7 - Rebase your local works from upstream if it takes a long time. Commit your bugfix or feature code once the issue/task got addressed. Stage all your changed/new files first, please refer to step 6 above. If any conflict occurred, please refer to the [following way to resolve it.](https://docs.github.com/en/free-pro-team@latest/github/using-git/resolving-merge-conflicts-after-a-git-rebase){:target="_blank"} + +```bash + $ git commit -s -m “MNEMONIC-[issue id]: [issue description]” + $ git rebase upstream/master +``` + +* Step 8 - Verify your code changes again + +```bash + $ git diff HEAD^… +``` + +* Step 9 - Push your new branch and commit to your own github repo. + +```bash + $ git push --set-upstream origin issue-XXX +``` + +* Step 10 - Submit a PR. Visit the Mnemonic repo. in your own Github account, you can see a message about the new/changed branch, you can submit the PR via click the link in the message. [Show Me How](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request){:target="_blank"} + +* Step 11 - Once PR got submitted, Please click “Resolve Issue” button in [Apache issue web portal.](https://confluence.atlassian.com/adminjiraserver/working-with-workflows-938847362.html){:target="_blank"}. + +* Step 12 - Normally, wait for 2-5 calendar days for reviewers to take comments on the PR. Address all comments as possible. Once the PR got verified by reviewers, Please click “Close Issue” button in Apache issue web portal. \ No newline at end of file