giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman" <initialcont...@gmail.com>
Subject Re: Review Request: Refactor CLI arg parsing out of GiraphRunner
Date Tue, 12 Feb 2013 20:20:58 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9350/
-----------------------------------------------------------

(Updated Feb. 12, 2013, 8:20 p.m.)


Review request for giraph.


Changes
-------

Just a rebase, this passes mvn verify and completes cluster job run successfully. Did move
the Giraph IO format path parsing back into the "general CLI parsing" and out of the "hadoop
MR specific parsing" I had it in in the last iteration. This is due to what Alessandro told
me (we can use these IO formats, basically as-is, even on non-Hadoop MR platform)

This should put us in a strong position for refactoring the Runners (Hive and Giraph) into
one class, or alternately into a base and several very simple subclasses, either way. At this
point, I plan to start the YARN-specific code in YarnGiraphJob and attempt to avoid having
a specific YarnGiraphRunner class. JIRAs on this to be posted soon...


Description
-------

Refactor CLI parsing of args and the assembly of the GiraphConfiguration into its own class
so that other future Runner classes besides GiraphRunner can use it w/o duplication. You get
the idea.


Diffs (updated)
-----

  giraph-core/src/main/java/org/apache/giraph/GiraphRunner.java 1516fff 
  giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java 8797c0e 
  giraph-core/src/main/java/org/apache/giraph/io/formats/GiraphFileInputFormat.java 9c160fe

  giraph-core/src/main/java/org/apache/giraph/job/GiraphConfigurationValidator.java PRE-CREATION

  giraph-core/src/main/java/org/apache/giraph/job/GiraphJob.java 9f711da 
  giraph-core/src/main/java/org/apache/giraph/job/GiraphTypeValidator.java de2a66d 
  giraph-core/src/main/java/org/apache/giraph/utils/ConfigurationUtils.java PRE-CREATION 
  giraph-core/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java ae8ebf3 
  giraph-core/src/test/java/org/apache/giraph/io/TestJsonBase64Format.java e2f6fd3 
  giraph-examples/src/test/java/org/apache/giraph/vertex/TestVertexTypes.java 80187ef 

Diff: https://reviews.apache.org/r/9350/diff/


Testing
-------


Thanks,

Eli Reisman


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message