giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Avery Ching (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-214) GiraphJob should have configuration split out of it to be cleaner (GiraphConf)
Date Tue, 14 Aug 2012 08:15:38 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433987#comment-13433987
] 

Avery Ching commented on GIRAPH-214:
------------------------------------

If I'm not mistaken, isn't ReflectionsUtils.newInstance() the only issue here?  That being
said, the only time we have to use it is in the RPC I think.  Otherwise once we get the Configuration,
we can convert it to a GiraphConf and only pass that one around.  Are there any other places
that we need to use Configuration?  I think I prefer Option #1 due to having the ability to
cast around.

Once we add GIRAPH-211, we can remove Hadoop RPC and get rid of that section.

I can understand the arguments about separating the config definitions, but I'd personally
prefer methods on GiraphConf to get all the configuration for simplicity.  If separating config
definitions is a big deal, one could consider a class hierarchy that was something like

Configuration->NetworkConf->WorkerConf->->MasterConf->GiraphConf

But I think it would be fairly simple to have a single GiraphConf and then group the methods
by their area of the code.
                
> GiraphJob should have configuration split out of it to be cleaner (GiraphConf)
> ------------------------------------------------------------------------------
>
>                 Key: GIRAPH-214
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-214
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Avery Ching
>            Assignee: Eli Reisman
>            Priority: Minor
>         Attachments: GIRAPH-214-1.patch, GIRAPH-214-2.patch, GIRAPH-214-3.patch, GIRAPH-214-4.patch,
GIRAPH-214-5-option1.patch, GIRAPH-214-6-option1.patch
>
>
> Currently all the configuration for Giraph is part of GiraphJob, making things messy
for GiraphJob.
> It would be better if we added a GiraphConf (similar to Hive) that is responsible for
handling configuration of the Job.
> i.e.
> public class GiraphJob extends Configuration....
> To simplify config, we should make get/set methods for as many of the parameters as possible.
> We are targeting configuration such as
>   /**
>    * Set the vertex class (required)
>    *
>    * @param vertexClass Runs vertex computation
>    */
>   public final void setVertexClass(Class<?> vertexClass) {
>     getConfiguration().setClass(VERTEX_CLASS, vertexClass, BasicVertex.class);
>   }
>   /**
>    * Set the vertex input format class (required)
>    *
>    * @param vertexInputFormatClass Determines how graph is input
>    */
>   public final void setVertexInputFormatClass(
>       Class<?> vertexInputFormatClass) {
>     getConfiguration().setClass(VERTEX_INPUT_FORMAT_CLASS,
>         vertexInputFormatClass,
>         VertexInputFormat.class);
>   }
>   /**
>    * Set the vertex output format class (optional)
>    *
>    * @param vertexOutputFormatClass Determines how graph is output
>    */
>   public final void setVertexOutputFormatClass(
>       Class<?> vertexOutputFormatClass) {
>     getConfiguration().setClass(VERTEX_OUTPUT_FORMAT_CLASS,
>         vertexOutputFormatClass,
>         VertexOutputFormat.class);
>   }
>   /**
>    * Set the vertex combiner class (optional)
>    *
>    * @param vertexCombinerClass Determines how vertex messages are combined
>    */
>   public final void setVertexCombinerClass(Class<?> vertexCombinerClass) {
>     getConfiguration().setClass(VERTEX_COMBINER_CLASS,
>         vertexCombinerClass,
>         VertexCombiner.class);
>   }
>   /**
>    * Set the graph partitioner class (optional)
>    *
>    * @param graphPartitionerFactoryClass Determines how the graph is partitioned
>    */
>   public final void setGraphPartitionerFactoryClass(
>       Class<?> graphPartitionerFactoryClass) {
>     getConfiguration().setClass(GRAPH_PARTITIONER_FACTORY_CLASS,
>         graphPartitionerFactoryClass,
>         GraphPartitionerFactory.class);
>   }
>   /**
>    * Set the vertex resolver class (optional)
>    *
>    * @param vertexResolverClass Determines how vertex mutations are resolved
>    */
>   public final void setVertexResolverClass(Class<?> vertexResolverClass) {
>     getConfiguration().setClass(VERTEX_RESOLVER_CLASS,
>         vertexResolverClass,
>         VertexResolver.class);
>   }
>   /**
>    * Set the worker context class (optional)
>    *
>    * @param workerContextClass Determines what code is executed on a each
>    *        worker before and after each superstep and computation
>    */
>   public final void setWorkerContextClass(Class<?> workerContextClass) {
>     getConfiguration().setClass(WORKER_CONTEXT_CLASS,
>         workerContextClass,
>         WorkerContext.class);
>   }
> ...etc. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message