Return-Path: X-Original-To: apmail-giraph-dev-archive@www.apache.org Delivered-To: apmail-giraph-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 17898D803 for ; Mon, 19 Nov 2012 21:34:07 +0000 (UTC) Received: (qmail 66962 invoked by uid 500); 19 Nov 2012 21:34:06 -0000 Delivered-To: apmail-giraph-dev-archive@giraph.apache.org Received: (qmail 66928 invoked by uid 500); 19 Nov 2012 21:34:06 -0000 Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@giraph.apache.org Delivered-To: mailing list dev@giraph.apache.org Delivered-To: moderator for dev@giraph.apache.org Received: (qmail 59224 invoked by uid 99); 19 Nov 2012 21:30:53 -0000 Content-Type: multipart/alternative; boundary="===============8045591828627672148==" MIME-Version: 1.0 Subject: Re: Review Request: Edge-based input from HCatalog From: "Nitay Joffe" To: "giraph" , "Nitay Joffe" , "Alessandro Presta" Date: Mon, 19 Nov 2012 21:30:49 -0000 Message-ID: <20121119213049.3393.16097@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Nitay Joffe" X-ReviewGroup: giraph X-ReviewRequest-URL: https://reviews.apache.org/r/8034/ X-Sender: "Nitay Joffe" References: <20121113034614.10558.85377@reviews.apache.org> In-Reply-To: <20121113034614.10558.85377@reviews.apache.org> Reply-To: "Nitay Joffe" --===============8045591828627672148== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/8034/#review13587 ----------------------------------------------------------- /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/G= iraphHCatInputFormat.java yeah it's really unfortunate we have to do this - but hopefully sooner = it will be cleaner as things progress /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/G= iraphHCatInputFormat.java why do you need this and the method above, just have the Configuration = one alone. /trunk/giraph-formats-contrib/src/main/java/org/apache/hcatalog/mapreduce/H= CatUtils.java Can we make all the parts of this diff where you copy code into TODOs a= nd file JIRAs with HCatalog to make the appropriate changes? We should trac= k that these are copied over and work towards remove them as opposed to let= ting them pollute our code. /trunk/pom.xml I think I do this in one of my diffs, in case not - can you make some v= ariable like hive.version to use here everywhere? - Nitay Joffe On Nov. 13, 2012, 3:46 a.m., Alessandro Presta wrote: > = > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/8034/ > ----------------------------------------------------------- > = > (Updated Nov. 13, 2012, 3:46 a.m.) > = > = > Review request for giraph. > = > = > Description > ------- > = > Implemented Edge/VertexValue input formats for HCatalog. > Unfortunately, I had to pretty much copy some functionality from HCatalog= because of privacy restrictions, and add our own GiraphHCatInputFormat (mu= ch like with GiraphFileInputFormat). > Also, I had to make HiveGiraphRunner a little less type-safe, because HCa= talogVertexValueInputFormat is not a subclass of HCatalogVertexInputFormat.= If this is an issue, I can add an empty interface to address it. > = > = > This addresses bug GIRAPH-405. > https://issues.apache.org/jira/browse/GIRAPH-405 > = > = > Diffs > ----- > = > /trunk/giraph-formats-contrib/pom.xml 1406239 = > /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatal= og/GiraphHCatInputFormat.java PRE-CREATION = > /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatal= og/HCatalogEdgeInputFormat.java PRE-CREATION = > /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatal= og/HCatalogVertexInputFormat.java 1406239 = > /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatal= og/HCatalogVertexOutputFormat.java 1406239 = > /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatal= og/HCatalogVertexValueInputFormat.java PRE-CREATION = > /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatal= og/HiveGiraphRunner.java 1406239 = > /trunk/giraph-formats-contrib/src/main/java/org/apache/hcatalog/mapredu= ce/HCatUtils.java PRE-CREATION = > /trunk/giraph-formats-contrib/src/main/java/org/apache/hcatalog/mapredu= ce/package-info.java PRE-CREATION = > /trunk/pom.xml 1406239 = > = > Diff: https://reviews.apache.org/r/8034/diff/ > = > = > Testing > ------- > = > - mvn verify > - tested on a real application that runs on top of Hive > = > = > Thanks, > = > Alessandro Presta > = > --===============8045591828627672148==--