mnemonic-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yz...@apache.org
Subject [mnemonic-site] branch master updated: MNEMONIC-587: Update development workflow on project website
Date Mon, 15 Feb 2021 22:37:21 GMT
This is an automated email from the ASF dual-hosted git repository.

yzhao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mnemonic-site.git


The following commit(s) were added to refs/heads/master by this push:
     new 306701c  MNEMONIC-587: Update development workflow on project website
     new c240fe0  Merge pull request #6 from rhythmwind/issue-587
306701c is described below

commit 306701c316ee8d9cdeb61da4058260d4da0c8cd6
Author: Zane <rhythmwind@hotmail.com>
AuthorDate: Wed Feb 10 22:49:48 2021 -0800

    MNEMONIC-587: Update development workflow on project website
    
    Signed-off-by: Zane <rhythmwind@hotmail.com>
---
 src/_docs/build.md         |  89 ++++++++++++++------------------
 src/_docs/devsetup.md      |  64 +++++++++++++++--------
 src/_docs/submitchanges.md | 124 ++++++++++++++++++++++++---------------------
 3 files changed, 148 insertions(+), 129 deletions(-)

diff --git a/src/_docs/build.md b/src/_docs/build.md
index 1acc495..f8b6c27 100644
--- a/src/_docs/build.md
+++ b/src/_docs/build.md
@@ -4,10 +4,9 @@ title: Project Build
 permalink: /docs/build.html
 ---
 
-### How to build it ?
-
-Please see the file LICENSE for information on how this library is licensed.
+### How to build it?
 
+Please see the file LICENSE for information on how this library is licensed. Below you will
find descriptions for each module.
 
 * **mnemonic-core** -- the submodule project for core
 * **mnemonic-collections** -- the submodule project for generic collections
@@ -22,69 +21,55 @@ Please see the file LICENSE for information on how this library is licensed.
 * **mnemonic-hadoop/mnemonic-hadoop-mapreduce** -- the submodule project for Apache Hadoop
mapreduce computing
 * **mnemonic-spark/mnemonic-spark-core** -- the submodule project for Apache Spark durable
computing
 
-To build this library, you may need to install some required packages on the build system:
+To build this library without Docker image, you will need to install some required packages
on the build system. Otherwise, you can build the project with Docker image generated [previously.](devsetup.html){:target="_blank"}
 
-* **Maven** -- the building tool v3.2.1 or above [Required]
-* **NVML** -- the NVM library (Please compile this library that was revised with 630862e82f)
(http://pmem.io) [Optional if mnemonic-nvml-vmem-service/mnemonic-nvml-pmem-service are excluded,
e.g. on MacOSX]
-* **JDK** -- the Java Develop Kit 7-8 (please properly configure JAVA_HOME) [Required]
-* **PMFS** -- the PMFS should be properly installed and configured on Linux system if you
want to simulate read latency [Optional]
-* **PMalloc** -- a supported durable memory native library(Latest) at https://github.com/NonVolatileComputing/pmalloc.git
[Optional if mnemonic-pmalloc-service is excluded]
+* **Maven** -- the building tool v3.2.1 or above. <I>Optional if you choose to use
Gradle.</I>
+* **Gradle** --  the building tool v6.7.1 or above. <I>Optional if you choose to use
Maven.</I>
+* **NVML** -- the [NVM library.](http://pmem.io){:target="_blank"} Please compile this library
that was revised with 630862e82f. 
+* **JDK** -- the Java Develop Kit 7-8. Please properly configure JAVA_HOME in environment
variables.
+* **PMFS** -- the PMFS should be properly installed and configured on Linux system if you
want to simulate read latency. <I>Optional if you choose to use it for simulation purpose.</I>
+* **PMalloc** -- a supported durable memory native library [here.](https://github.com/NonVolatileComputing/pmalloc.git){:target="_blank"}

 
 
-Once the build system is setup, this Library is built using this command at the top level:
-```bash
-  $ git clean -xdf # if pull from a git repo.
-  $ mvn clean package install
-```
+***
 
+You can use following commmand lines to install build tools for this project.
 
-To exclude a customized memory service for your platform e.g. OSX, note that if you excluded
one or both memory services, some or all testcases/examples will fail since their dependent
memory services are unavailable.
 ```bash
-  $ git clean -xdf # if pull from a git repo.
-  $ mvn -pl '!mnemonic-memory-services/mnemonic-nvml-vmem-service' clean package install
+  # ---- install build tools ----
+  $ sudo apt-get -y update && \
+    apt-get install -y default-jdk cmake check git pkg-config autoconf man build-essential
gcc g++ uuid-dev pandoc devscripts flex doxygen maven
+  $ sudo apt-get install -y libndctl-dev libpmem-dev libvmem-dev libpmemobj-dev
+  $ sudo apt-get clean
+  # ---- update environment variables ----
+  $ export JAVA_HOME=/usr/lib/jvm/default-java
+  $ export PATH=$JAVA_HOME/bin:$PATH
+  $ source ~/.profile
+  # ---- install pmalloc ----
+  $ git clone https://github.com/NonVolatileComputing/pmalloc.git && \
+    cd pmalloc && mkdir build && cd build && cmake .. &&
make && make install
+  # ---- clone Mnemonic ----
+  $ git clone https://github.com/apache/mnemonic.git && \
+    cd mnemonic && mvn clean package install
 ```
 
+***
 
-To install this package to local repository (required to run examples and testcases):
-```bash
-  $ mvn clean install
-```
-
+Once the build system is setup, this Library is built using this command at the top level:
 
-To run an example:
 ```bash
-  $ # requires 'vmem' memory service to run, please refer to the code of test cases for more
examples.
-  $ mvn exec:exec -Pexample -pl mnemonic-examples
+  $ git clean -xdf # if pull from a git repo.
+  # ------ For Maven ---------
+  $ mvn clean install
+  # ------ For Gradle --------
+  $ ./gradlew clean
+  $ ./gradlew build -x test
 ```
 
+To exclude a customized memory service for your platform e.g. OSX, note that if you excluded
one or both memory services, some or all testcases/examples will fail since their dependent
memory services are unavailable.
 
-To run several test cases:
 ```bash
-  $ # a testcase for module "mnemonic-core" that requires 'pmalloc' memory service to pass
-  $ mvn -Dtest=DurablePersonNGTest test -pl mnemonic-core -DskipTests=false
-  
-  $ # a testcase for module "mnemonic-core" that requires 'pmalloc' memory service to pass
-  $ mvn -Dtest=NonVolatileMemAllocatorNGTest test -pl mnemonic-core -DskipTests=false
-  
-  $ # a testcase for module "mnemonic-core" that requires 'vmem' memory service to pass
-  $ mvn -Dtest=VolatileMemAllocatorNGTest test -pl mnemonic-core -DskipTests=false
-  
-  $ # a testcase for module "mnemonic-core" that requires 'vmem memory service to pass
-  $ mvn -Dtest=MemClusteringNGTest test -pl mnemonic-core -DskipTests=false
-  
-  $ # a testcase for module "mnemonic-collection" that requires 'pmalloc' memory service
to pass
-  $ mvn -Dtest=DurableSinglyLinkedListNGTest  test -pl mnemonic-collections -DskipTests=false
-  
-  $ # a testcase for module "mnemonic-collection" that requires 'pmalloc' memory service
to pass
-  $ mvn -Dtest=DurablePersonNGTest  test -pl mnemonic-collections -DskipTests=false
-  
-  $ # a testcase for module "mnemonic-computing-services/mnemonic-utilities-service" that
requires 'pmalloc' memory service to pass
-  $ mvn -Dtest=DurableSinglyLinkedListNGPrintTest test -pl mnemonic-computing-services/mnemonic-utilities-service
-DskipTests=false
-  
-  $ # a testcase for module "mnemonic-computing-services/mnemonic-utilities-service" that
requires 'pmalloc' memory service to pass
-  $ mvn -Dtest=DurableSinglyLinkedListNGSortTest test -pl mnemonic-computing-services/mnemonic-utilities-service
-DskipTests=false
-  
-  $ # a testcase for module "mnemonic-hadoop/mnemonic-hadoop-mapreduce" that requires 'pmalloc'
memory service to pass
-  $ mvn -Dtest=MneMapreducePersonDataTest test -pl mnemonic-hadoop/mnemonic-hadoop-mapreduce
-DskipTests=false
+  $ git clean -xdf # if pull from a git repo.
+  $ mvn -pl '!mnemonic-memory-services/mnemonic-nvml-vmem-service' clean package install
 ```
 
diff --git a/src/_docs/devsetup.md b/src/_docs/devsetup.md
index 6b938d5..fcf98b2 100644
--- a/src/_docs/devsetup.md
+++ b/src/_docs/devsetup.md
@@ -4,38 +4,62 @@ title: Development Setup
 permalink: /docs/devsetup.html
 ---
 
-This document provides a best practice for general development setup, 
-we suggest to use this practice on your first try for code contribution.
+This document provides a best practice for general development setup and code distribution.
Please practice on your computer before trying for code contribution.
 
-It is based on Docker and simple for preparation, no root privilege needed except for Docker
and Git installation. 
+***
+
+Pre-requisites: 
+
+* Install Git on your system - [Git-scm.](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git){:target="_blank"}
+
+* Register Github Account if you dont have one already. You can register your Github acocunt
[here.](http://github.com){:target="_blank"}
+
+* Obtain Docker on your system. You can get docker from [here.](https://www.docker.com/){:target="_blank"}
+
+* (Optional) Install IDE on your operating system. e.g. [Visual Studio Code](https://code.visualstudio.com/download){:target="_blank"}
or [IntelliJ](https://www.jetbrains.com/idea/){:target="_blank"}
+
+***
+
+* Step 1 - Configure Git tool if you are behind the firewall. <I>[Show Me How](https://gist.github.com/evantoli/f8c23a37eb3558ab8765){:target="_blank"}</I>
+
+* Step 2 - Fork the following Mnemonic project to your own Github account. <I>[Project
Link](https://github.com/apache/mnemonic){:target="_blank"}</I>
+
+* Step 3 - Clone the forked Mnemonic project from your own Github account by running the
command below. <I>[Show Me How](https://docs.github.com/en/free-pro-team@latest/github/creating-cloning-and-archiving-repositories/cloning-a-repository){:target="_blank"}</I>
 
-* Install Git on your system [Git-scm](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git){:target="_blank"}
-* [First-Time Git Setup](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup){:target="_blank"}
 ```bash
-  $ git config --global user.name "<Your Name>"
-  $ git config --global user.email "<Your Email Address>"
+  $ git clone git@github.com:[your account id]/mnemonic.git
 ```
-* Install Docker on your system [Docker Official Website](https://www.docker.com/){:target="_blank"}
-* Install IDE on your Linux/MacOS/Windows box [IntelliJ IDEA Official Website](https://www.jetbrains.com/idea/){:target="_blank"}
-* Register an account on Github for yourself if not yet have one
-* Log into Github and fork [Apache Mnemonic repository](https://github.com/apache/mnemonic){:target="_blank"}.
-  [Fork and Set up Git](https://help.github.com/articles/fork-a-repo/){:target="_blank"}
-* Configure Apache Mnemonic repo. as upstream remote in your local repository
+
+* Step 4 - Configure a remote upstream from Github for upstream sync by running the command
below. <I>[Show Me How](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/configuring-a-remote-for-a-fork){:target="_blank"}</I>
+
 ```bash
-  $ cd <your_project_folder>
   $ git remote add upstream https://github.com/apache/mnemonic.git
 ```
-* Prepare Mnemonic Docker image on you system
+
+* Step 5 - Setting your commit email address and name for local Mnemonic repo by running
the command below. <I>[Show Me How](https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-your-github-user-account/setting-your-commit-email-address){:target="_blank"}</I>
+
+
+```bash
+  $ cd mnemonic
+  $ git config user.email “[your apache id]@ email@example.comapache.org”
+  $ git config user.name “[your name]”
+```
+
+* Step 6 - Prepare Mnemonic Docker image on your own system by running one of following command
below.
+
 ```bash
+  $ # --------- For CentOS ---------
   $ docker build -t mnedev https://github.com/apache/mnemonic.git#:docker/docker-CentOS
-  $ # --------- OR --------- 
+  $ # --------- For Ubuntu --------- 
   $ docker build -t mnedev https://github.com/apache/mnemonic.git#:docker/docker-Ubuntu
 ```
-* Mount your host repo. to Docker container, this following command would hide the original
Mnemonic project folder in container 
+
+* Step 7 - Mount your host repository to Docker container, this following command would hide
the original Mnemonic project folder in container.
+
 ```bash
   $ docker run -it -v <mnemonic_project_folder>:/ws/mnemonic mnedev
 ```
-* Open Mnemonic project in the IDE 
 
-After setup, you are able to edit project code on your host while compiling and test your
code in isolated Docker environment.
-Please leave enough space for docker based compiling and testing (>30G for test running)
+<B>Congratulation!</B> You have successfully completed the development setup.
You are now ready to edit the project code on your host while compiling and test in isolated
Docker envinroment. Please make sure you have enough space for docker based compiling and
testing (>30G for test running). 
+
+Please refer to [Submit Changes](submitchanges.html){:target="_blank"} page if you would
like to contribute to this project!
\ No newline at end of file
diff --git a/src/_docs/submitchanges.md b/src/_docs/submitchanges.md
index 4722717..d9e7dcf 100644
--- a/src/_docs/submitchanges.md
+++ b/src/_docs/submitchanges.md
@@ -4,60 +4,70 @@ title: Submit Changes
 permalink: /docs/submitchanges.html
 ---
 
-After [Development Setup](devsetup.html), you can follow the steps below to
-submit your code/comments and document changes.
-
-* Make sure the changes are linked to your [Apache JIRA](https://issues.apache.org/jira/projects/MNEMONIC){:target="_blank"}
ticket
-* Review your changes according to the JIRA ticket in question
-* Building project in the Docker container
-```bash
-  ## for Docker container
-  # cd /ws/mnemonic
-  # mvn clean install
-```
-* Resolve any problems found by building process
-* Run your test cases specific to your changes, please refer to bin/test.conf
-* Resolve any problems found by your test cases
-* Add your test cases to *bin/test.conf*
-* Commit the changes to your local repository in IDE<br/>
-  for Intellij IDEA, please refer to [Committing Changes to a Local Git Repository](https://www.jetbrains.com/help/idea/commit-and-push-changes.html#commit){:target="_blank"}<br/>
-  Please use JIRA id and title to fill up the comment of commit<br/>
-  for example
-```bash
-  ## for host
-  # git commit -m "MNEMONIC-301: Improve the documentation of architecture page"
-```
-* Run all test cases if your changes could impact existing functionalities<br/>
-Note: Please make sure enough available space for test data generation (>30G)
-```bash
-  ## for Docker container
-  # cd /ws/mnemonic
-  # bin/runall.sh
-```
-* Resolve any problems found by test cases and then **amend** your previous commit in IDE<br/>
-  for Intellij IDEA, please refer to [Committing Changes to a Local Git Repository](https://www.jetbrains.com/help/idea/commit-and-push-changes.html#commit){:target="_blank"}
-```bash
-  ## for host
-  # git commit --amend
-```
-* Make sure there is only 1 local commit for each JIRA ticket, otherwise please squash them
into one
-* Rebase your codebase before pushing local commits to your remote repository
-  1. fetch update from remote [Fetching Changes](https://www.jetbrains.com/help/idea/sync-with-a-remote-repository.html#fetch){:target="_blank"}
-  ```bash
-    ## for host
-    # git fetch -a
-  ```
-  2. Rebase your changes onto upstream/master [Rebasing Branches](https://www.jetbrains.com/help/idea/apply-changes-from-one-branch-to-another.html#rebase-branch){:target="_blank"}
-  ```bash
-    ## for host
-    # git rebase upstream/master
-  ```
-* Push local commits to your remote repository [Pushing Changes](https://www.jetbrains.com/help/idea/commit-and-push-changes.html#push){:target="_blank"}
-```bash
-  ## for host
-  # git push origin
-  ## NOTE: use force push if your amended commit has already been pushed
-``` 
-* Create a PR in Github [Creating a pull request](https://help.github.com/articles/creating-a-pull-request/){:target="_blank"}
-* Fix any issues come from community review
-* Change the JIRA ticket status to RESOLVED if the button exists
+
+After [Development Setup](devsetup.html), you can follow the steps below to submit your code/comments
and document changes.
+
+***
+
+* Step 1 - Sync your forked Mnemonic project from upstream. <I>[Show Me How](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/syncing-a-fork){:target="_blank"}</I>
+
+  **<I> Please keep the master branch untouched, otherwise we can't keep it exactly
synced. </I>**
+
+```bash
+  $ git fetch upstream
+  $ git checkout master
+  $ git merge upstream/master
+  $ git push
+```
+
+* Step 2 - Take over an interested issue/task by clicking the “Start Progress” button
in the Apache issue web portal. <I>[Project Issue Portal](https://issues.apache.org/jira/projects/MNEMONIC/){:target="_blank"}</I>
+
+* Step 3 - Create a new branch for your selected issue/task.
+
+```bash
+  $ git checkout master
+  $ git checkout -b issue-[issue id]
+```
+
+* Step 4 - Apply patch files if any.
+
+```bash
+  $ git apply [patch file]
+```
+
+* Step 5 - Fix/implement the selected issue/task on the specific branch.Please ensure you
are working on the create branch, e.g. issue-548, using the following command to confirm it
+
+```bash
+  $ git status
+```
+
+* Step 6 - Stage your new code or code changes before stopping work on it.
+
+```bash
+  $ git add [your changed/new files]
+```
+
+* Step 7 - Rebase your local works from upstream if it takes a long time. Commit your bugfix
or feature code once the issue/task got addressed. Stage all your changed/new files first,
please refer to step 6 above. If any conflict occurred, please refer to the <I>[following
way to resolve it.](https://docs.github.com/en/free-pro-team@latest/github/using-git/resolving-merge-conflicts-after-a-git-rebase){:target="_blank"}</I>
+
+```bash
+  $ git commit -s -m “MNEMONIC-[issue id]: [issue description]”
+  $ git rebase upstream/master
+```
+
+* Step 8 - Verify your code changes again
+
+```bash
+  $ git diff HEAD^…
+```
+
+* Step 9 - Push your new branch and commit to your own github repo.
+
+```bash
+  $ git push --set-upstream origin issue-XXX
+```
+
+* Step 10 - Submit a PR. Visit the Mnemonic repo. in your own Github account, you can see
a message about the new/changed branch, you can submit the PR via click the link in the message.
<I>[Show Me How](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request){:target="_blank"}</I>

+
+* Step 11 - Once PR got submitted, Please click “Resolve Issue” button in <I>[Apache
issue web portal.](https://confluence.atlassian.com/adminjiraserver/working-with-workflows-938847362.html){:target="_blank"}</I>.
+
+* Step 12 - Normally, wait for 2-5 calendar days for reviewers to take comments on the PR.
Address all comments as possible. Once the PR got verified by reviewers, Please click “Close
Issue” button in Apache issue web portal.
\ No newline at end of file


Mime
View raw message