Return-Path: X-Original-To: apmail-gora-commits-archive@www.apache.org Delivered-To: apmail-gora-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5892210251 for ; Mon, 2 Mar 2015 19:23:57 +0000 (UTC) Received: (qmail 58444 invoked by uid 500); 2 Mar 2015 19:23:57 -0000 Delivered-To: apmail-gora-commits-archive@gora.apache.org Received: (qmail 58409 invoked by uid 500); 2 Mar 2015 19:23:57 -0000 Mailing-List: contact commits-help@gora.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@gora.apache.org Delivered-To: mailing list commits@gora.apache.org Received: (qmail 58400 invoked by uid 99); 2 Mar 2015 19:23:57 -0000 Received: from eris.apache.org (HELO hades.apache.org) (140.211.11.105) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2015 19:23:57 +0000 Received: from hades.apache.org (localhost [127.0.0.1]) by hades.apache.org (ASF Mail Server at hades.apache.org) with ESMTP id C29B4AC0041 for ; Mon, 2 Mar 2015 19:23:56 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1663401 - /gora/site/trunk/content/current/gora-shims.md Date: Mon, 02 Mar 2015 19:23:56 -0000 To: commits@gora.apache.org From: lewismc@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20150302192356.C29B4AC0041@hades.apache.org> Author: lewismc Date: Mon Mar 2 19:23:56 2015 New Revision: 1663401 URL: http://svn.apache.org/r1663401 Log: GORA-384 Provide documentation on Gora Shims layer Modified: gora/site/trunk/content/current/gora-shims.md Modified: gora/site/trunk/content/current/gora-shims.md URL: http://svn.apache.org/viewvc/gora/site/trunk/content/current/gora-shims.md?rev=1663401&r1=1663400&r2=1663401&view=diff ============================================================================== --- gora/site/trunk/content/current/gora-shims.md (original) +++ gora/site/trunk/content/current/gora-shims.md Mon Mar 2 19:23:56 2015 @@ -40,34 +40,116 @@ should be used as below.** [Click Here](http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-distribution|0.6|bundle) -##Direct Dependency - #gora-shims-hadoop ##Description +This module contains functional Java code for enabling dynamic selection of the Hadoop environment based upon the +presence of **org.apache.hadoop.util.VersionInfo**. This is a two-part process where we first obtain the [Hadoop +Major version](https://github.com/apache/gora/blob/master/gora-shims-hadoop/src/main/java/org/apache/gora/shims/hadoop/HadoopShimFactory.java#L79-L94). +We then [select the appropriate Gora Shim](https://github.com/apache/gora/blob/master/gora-shims-hadoop/src/main/java/org/apache/gora/shims/hadoop/HadoopShimFactory.java#L56-L77) +based on detecting the Hadoop Major version. This code looks the following: + + + /** + * Get the Hadoop major version number. + * + * @return The major version number of Hadoop. + */ + public String getMajorVersion() { + String vers = VersionInfo.getVersion(); + String[] parts = vers.split("\\."); + if (parts.length < 2) { + throw new RuntimeException("Unable to parse Hadoop version: " + + vers + " (expected X.Y.* format)"); + } + return parts[0]; + } + + /** + * Get the Hadoop shim for the Hadoop version on the class path. In case it + * fails to obtain an appropriate shim (i.e. unsupported Hadoop version), it + * throws a {@link RuntimeException}. + * + * Note that this method is potentially costly. + * + * @return A newly created instance of a {@link HadoopShim}. + */ + public HadoopShim getHadoopShim() { + String version = getMajorVersion(); + String className = HADOOP_VERSION_TO_IMPL_MAP.get(version); + try { + Class class1 = Class.forName(className); + return HadoopShim.class.cast(class1.newInstance()); + } catch (Exception e) { + throw new RuntimeException( + "Could not load Hadoop shim for version " + version + + ", className=" + className, e); + } + } -##Dependency - +##Dependency Definition -##Direct Dependency +[Click Here](http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-hadoop|0.6|bundle) #gora-shims-hadoop1 ##Description +This module provides all functionality for Hadoop 1.X support in Gora. The Hadoop version is specified within +the project [pom.xml](https://github.com/apache/gora/blob/master/pom.xml) within the **${hadoop1-version}** variable. + +The actual [code implementation](https://github.com/apache/gora/blob/master/gora-shims-hadoop1/src/main/java/org/apache/gora/shims/hadoop1/HadoopShim1.java) +is very simple. Essentially it consists of two Java methods + + /** + * {@inheritDoc} + */ + public Job createJob(Configuration configuration) throws IOException { + return new Job(configuration); + } + /** + * {@inheritDoc} + */ + public JobContext createJobContext(Configuration configuration) { + return new JobContext(configuration, null); + } -##Dependency +##Dependency Definition + +[Click Here](http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-hadoop1|0.6|bundle) -##Direct Dependency #gora-shims-hadoop2 ##Description -##Dependency +This module provides all functionality for Hadoop 2.X support in Gora. The Hadoop version is specified within +the project [pom.xml](https://github.com/apache/gora/blob/master/pom.xml) within the **${hadoop2-version}** variable. + +The actual [code implementation](https://github.com/apache/gora/blob/master/gora-shims-hadoop1/src/main/java/org/apache/gora/shims/hadoop1/HadoopShim1.java) +is very simple. Essentially it consists of two Java methods + /** + * {@inheritDoc} + * + * Use the Hadoop 2.x way of creating a {@link Job} object. + */ + public Job createJob(Configuration configuration) throws IOException { + Job instance = Job.getInstance(configuration); + return instance; + } + /** + * {@inheritDoc} + * + * Use the Hadoop 2.x way of creating a {@link JobContext} object. + */ + public JobContext createJobContext(Configuration configuration) { + return new JobContextImpl(configuration, null); + } + +##Dependency Definition +[Click Here](http://search.maven.org/#artifactdetails|org.apache.gora|gora-shims-hadoop2|0.6|bundle) -##Direct Dependency