flink-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rmetz...@apache.org
Subject git commit: [Documentation] Move "How to contrib"; "coding guidelines" out of the code repo (part of FLINK-1012) Improve build instructions with respect to Hadoop 1 and Hadoop 2 dependencies. [ci skip]
Date Sat, 26 Jul 2014 17:07:49 GMT
Repository: incubator-flink
Updated Branches:
  refs/heads/master 2e50eac53 -> 25b4099d3

[Documentation] Move "How to contrib"; "coding guidelines" out of the code repo (part of FLINK-1012)
Improve build instructions with respect to Hadoop 1 and Hadoop 2 dependencies.
[ci skip]

Project: http://git-wip-us.apache.org/repos/asf/incubator-flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-flink/commit/25b4099d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-flink/tree/25b4099d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-flink/diff/25b4099d

Branch: refs/heads/master
Commit: 25b4099d3af85ae881aaa52ddae0d71f37e67b4c
Parents: 2e50eac
Author: Robert Metzger <rmetzger@apache.org>
Authored: Sat Jul 26 19:05:49 2014 +0200
Committer: Robert Metzger <rmetzger@apache.org>
Committed: Sat Jul 26 19:06:44 2014 +0200

 docs/building.md          | 32 +++++++++++++++-
 docs/coding_guidelines.md | 62 +-----------------------------
 docs/how_to_contribute.md | 85 +-----------------------------------------
 3 files changed, 33 insertions(+), 146 deletions(-)

diff --git a/docs/building.md b/docs/building.md
index 86810a4..a07b324 100644
--- a/docs/building.md
+++ b/docs/building.md
@@ -3,7 +3,7 @@ title:  "Build Flink"
-In order to build Flink, you need the source code. Either download the source of a release
or clone the git repository. In addition to that, you need Maven 3 and a JDK. Note that you
can not build Flink with Oracle JDK 6 due to a unresolved bug in the Java compiler. It works
well with OpenJDK 6 and all Java 7 and 8 compilers.
+In order to build Flink, you need the source code. Either download the source of a release
or clone the git repository. In addition to that, you need Maven 3 and a JDK (Java Development
Kit). Note that you can not build Flink with Oracle JDK 6 due to a unresolved bug in the Java
compiler. It works well with OpenJDK 6 and all Java 7 and 8 compilers.
 To clone from git, enter:
@@ -13,6 +13,7 @@ git clone {{ site.FLINK_GITHUB_URL }}
 The simplest way of building Flink is by running:
+cd incubator-flink
 mvn clean package -DskipTests
@@ -23,7 +24,8 @@ This instructs Maven (`mvn`) to first remove all existing builds (`clean`)
and t
 ## Build Flink for a specific Hadoop Version
 This section covers building Flink for a specific Hadoop version. Most users do not need
to do this manually.
-The problem is that Flink uses HDFS and YARN which are both from Apache Hadoop. There exist
many different builds of Hadoop (from both the upstream project and the different Hadoop distributions).
Typically errors arise with the RPC services. An error could look like this:
+The problem is that Flink uses HDFS and YARN which are both dependencies from Apache Hadoop.
There exist many different versions of Hadoop (from both the upstream project and the different
Hadoop distributions). If a user is using a wrong combination of versions, exceptions like
this one occur:
 ERROR: The job was not successfully submitted to the nephele job manager:
@@ -32,6 +34,32 @@ ERROR: The job was not successfully submitted to the nephele job manager:
     Protocol message contained an invalid tag (zero).; Host Details :
+There are two main versions of Hadoop that we need to differentiate:
+- Hadoop 1, with all versions starting with zero or one, like 0.20, 0.23 or 1.2.1.
+- Hadoop 2 with all versions including or higher than 2.2.0.
+The main differentiation between Hadoop 1 and Hadoop 2 is the availability of Hadoop YARN
(Hadoops cluster resource manager).
+**To build Flink for Hadoop 2**, issue the following command:
+mvn clean package -DskipTests -Dhadoop.profile=2
+The `-Dhadoop.profile=2` flag instructs Maven to build Flink with YARN support and the Hadoop
2 HDFS client.
+Usually, this flag is sufficient for full support of Flink for Hadoop 2-versions.
+However, you can also **specify a specific Hadoop version to build against**:
+mvn clean package -DskipTests -Dhadoop.profile=2 -Dhadoop.version=2.4.1
+**To build Flink against a vendor specific Hadoop version**, issue the following command:
+mvn clean package -DskipTests -Pvendor-repos -Dhadoop.version=2.2.0-cdh5.0.0-beta-2
+The `-Pvendor-repos` activates a Maven [build profile](http://maven.apache.org/guides/introduction/introduction-to-profiles.html)
that includes the repositories of popular Hadoop vendors such as Cloudera, Hortonworks, or
 ### Background
 The builds with Maven are controlled by [properties](http://maven.apache.org/pom.html#Properties)
and <a href="http://maven.apache.org/guides/introduction/introduction-to-profiles.html">build

diff --git a/docs/coding_guidelines.md b/docs/coding_guidelines.md
index 6ab5475..8d96b37 100644
--- a/docs/coding_guidelines.md
+++ b/docs/coding_guidelines.md
@@ -2,64 +2,4 @@
 title:  "Coding Guidelines"
-These are the coding and style guidelines that we follow in the Flink project.
-## Guidelines for pull requests and patches
-- A pull request should relate to a JIRA issue; create an issue if none exists for the change
you want to make. The latest commit message should reference that issue. An example commit
message would be "[FLINK-633] Fix NullPointerException for empty UDF parameters". That way,
the pull request automatically gives a description of what it does, for example what bug does
it fix in what way?
-- We consider pull requests as requests to merge the referenced code *as is* into the current
*stable* master branch. Therefore, a pull request should not be "work in progress". Open a
pull request if you are confident that it can be merged into the current master branch without
problems. If you rather want comments on your code, post a link to your working branch.
-- Please do not combine various unrelated changes in a single pull request. Rather, open
multiple individual pull requests. This ensures that pull requests are *topic related*, can
be merged more easily, and typically result in topic-specific merge conflicts only.
-- Any pull request where the tests do not pass or which does not compile will not undergo
any further review. We recommend to connect your private GitHub accounts with [Travis CI](http://travis-ci.org/)
(like the Flink GitHub repository). Travis will run tests for all tested environments whenever
you push something into *your* Github repository.
-- Please keep reformatting of source files to a minimum. Diffs become unreadable if you (or
your IDE automatically) remove or replace whitespaces, reformat code, or comments. Also, other
patches that affect the same files become un-mergeable. Please configure your IDE such that
code is not automatically reformatted. Pull requests with excessive or unnecessary code reformatting
might be rejected.
-- All new features need to be backed by tests, *strictly*. It is very easy that a later merge
accidentally throws out a feature or breaks it. This will not be caught if the feature is
not guarded by tests. Anything not covered by a test is considered cosmetic.
-- Before opening a pull request follow this checklist:
- - Rebase onto the latest version of the master branch
- - Clean up your commits, i.e., squash them in a reasonable way and give meaningful commit
- - Run *all* tests either locally with ```mvn clean verify``` or use Travis CI to check the
-- When you get comments on the pull request asking for changes, append commits for these
changes. *Do not rebase and squash them.* It allows people to review the cleanup work independently.
Otherwise reviewers have to go through the entire set of diffs again.
-- Public methods and classes that are part of the user-facing API need to have JavaDocs.
Please write meaningful docs. Good docs are concise and informative.
-- Give meaningful exception messages. Try to imagine why an exception could be thrown (what
a user did wrong) and give a message that will help a user to resolve the problem.
-- Follow the checkstyle rules (see below). The checkstyle plugin verifies these rules when
you build the code. If your code does not follow the checkstyle rules, Maven will not compile
it and consequently the build will fail.
-## Coding Style Guidelines
-- Make sure you have Apache License headers in your files. The RAT plugin is checking for
that when you build the code.
-- We are using tabs for indentation, not spaces. We are not religious there, it just happened
to be the way that we started with tabs, and it is important to not mix them (merge/diff conflicts).
-- All statements after `if`, `for`, `while`, `do`, ... must always be encapsulated in a block
with curly braces (even if the block contains one statement):
-for (...) {
- ...
-If you are wondering why, recall the famous [*goto bug*](https://www.imperialviolet.org/2014/02/22/applebug.html)
in Apple's SSL library.
--  Do not use wildcard imports in the core files. They can cause problems when adding to
the code and in some cases even during refactoring. Exceptions are the Tuple classes, Tuple-related
utilities, and Flink user programs, when importing operators/functions. Tests are a special
case of the user programs.
-- Remove all unused imports.
-- Do not use raw generic types, unless strictly necessary (sometime necessary for signature
matches, arrays).
-- Add annotations to suppress warnings, if they cannot be avoided (such as "unchecked", or
-- Do not add "non-javadoc" comments. 
-- Add comments to your code. What is it doing? Add JavaDocs or inherit them by not adding
any comments to the methods. Do not automatically generate comments and avoid unnecessary
comments like
-i++;  // increment by one
+The coding guidelines are now located [on the project website](http://flink.incubator.apache.org/coding_guidelines.html).
\ No newline at end of file

diff --git a/docs/how_to_contribute.md b/docs/how_to_contribute.md
index 4b532d9..9c28505 100644
--- a/docs/how_to_contribute.md
+++ b/docs/how_to_contribute.md
@@ -1,86 +1,5 @@
-title:  "Contribute"
+title:  "How to contribute"
-The Flink project welcomes all sorts contributions in the form of code (improvements, features,
bugfixes), documentation, tests, and community participation (discussions & questions).
-## Easy Issues for Starters
-We maintain all known issues and feature drafts in the [Flink project JIRA](https://issues.apache.org/jira/browse/FLINK-989?jql=project%20%3D%20FLINK).
-We also try to maintain a [list of simple "starter issues"](https://issues.apache.org/jira/browse/FLINK-933?jql=project%20%3D%20FLINK%20AND%20labels%20%3D%20starter)
that we believe are good tasks for new contributors. Those tasks are meant to allow people
to get into the project and become familiar with the process of contributing. Feel free to
ask questions about issues that you would be interested in working on.
-In addition, you can find a list of ideas for projects and improvements in the [projects
wiki page](http://the/wiki/url).
-## Contributing Code & Documentation
-This section gives you a brief introduction in how to contribute code and documentation to
Flink. We maintain both the code and the documentation in the same repository, so the process
is essentially the same for both. We use [git](http://git-scm.com/) for the code and documentation
version control.
-The Flink project accepts code contributions though the [GitHub Mirror](https://github.com/apache/incubator-flink),
in the form of [Pull Requests](https://help.github.com/articles/using-pull-requests). Pull
requests are basically a simpler way of offering a patch, by providing a pointer to a code
branch that contains the change.
-### Setting up the Infrastructure and Creating a Pull Request
-1. The first step is to create yourself a copy of the Flink code base. We suggest to fork
the [Flink GitHub Mirror Repository](https://github.com/apache/incubator-flink) into your
own [GitHub](https://github.com) account. You need to register on GitHub for that, if you
have no account so far.
-2. Next, clone your repository fork to your local machine.
-git clone https://github.com/<your-user-name>/incubator-flink.git
-3. It is typically helpful to switch to a *topic branch* for the changes. To create a dedicated
branch based on the current master, use the following command:
-git checkout -b myBranch master
-4. Now you can create your changes, compile the code, and validate the changes. Here are
some pointers on how to [set up the Eclipse IDE for development](https://github.com/apache/incubator-flink/#eclipse-setup-and-debugging),
and how to [build the code](https://github.com/apache/incubator-flink/#build-flink).
-5. After you have finalized your contribution, verify the compliance with the contribution
guidelines (see below), and commit them. To make the changes easily mergeable, please rebase
them to the latest version of the main repositories master branch. Assuming you created a
topic branch (step 3), you can follow this sequence of commands to do that:
-Switch to the master branch, update it to the latest revision, switch back to your topic
branch, and rebase it on top of the master branch.
-git checkout master
-git pull https://github.com/apache/incubator-flink.git master
-git checkout myBranch
-git rebase master
-Have a look [here](https://help.github.com/articles/using-git-rebase) for more information
about rebasing commits.
-6. Push the contribution it back into your fork of the Flink repository.
-git push origin myBranch
-Go the website of your repository fork (`https://github.com/<your-user-name>/incubator-flink`)
and use the "Create Pull Request" button to start creating a pull request. Make sure that
the base fork is `apache/incubator-flink master` and the head fork selects the branch with
your changes. Give the pull request a meaningful description and send it.
-### Verifying the Compliance of your Code
-Before sending a patch or pull request, please verify that it complies with the guidelines
of the project. While we try to keep the set of guidelines small and easy, it is important
to follow those rules in order to guarantee good code quality, to allow efficient reviews,
and to allow committers to easily merge your changes.
-Please have a look at the [coding guidelines](coding_guidelines.html) for a guide to the
format of code and pull requests.
-Most important of all, verify that your changes are correct and do not break existing functionality.
Run the existing tests by calling `mvn verify` in the root directory of the repository, and
make sure that the tests succeed. We encourage every contributor to use a *continuous integration*
service that will automatically test the code in your repository whenever you push a change.
Flink is pre-configured for [Travis CI](http://docs.travis-ci.com/), which can be easily enabled
for your private repository fork (it uses GitHub for authentication, so you so not need an
additional account). Simply add the *Travis CI* hook to your repository (*settings -->
webhooks & services --> add service*) and enable tests for the "incubator-flink" repository
on [Travis](https://travis-ci.org/profile).
-When contributing documentation, please review the rendered HTML versions of the documents
you changed. You can look at the HTML pages by using the rendering script in preview mode.

-cd docs
-./build_docs.sh -p
-Now, open your browser at `http://localhost:4000` and check out the pages you changed.
-## How to become a committer
-There is no strict protocol for becoming a committer. Candidates for new committers are typically
people that are active contributors and community members.
-Being an active community member means participating on mailing list discussions, helping
to answer questions, being respectful towards others, and following the meritocratic principles
of community management. Since the "Apache Way" has a strong focus on the project community,
this part is very important.
-Of course, contributing code to the project is important as well. A good way to start is
contributing improvements, new features, or bugfixes. You need to show that you take responsibility
for the code that you contribute, add tests/documentation, and help maintaining it. 
-Finally, candidates for new committers are suggested by current committers, mentors, or PMC
members, and voted upon by the PMC.
+The "How to contribute"-guide is now located [on the project website](http://flink.incubator.apache.org/how-to-contribute.html).
\ No newline at end of file

View raw message