giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Akila Wajirasena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-745) add support for EdgeOutputFormat also for giraph-hive
Date Sun, 18 May 2014 06:33:15 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14001025#comment-14001025
] 

Akila Wajirasena commented on GIRAPH-745:
-----------------------------------------

While I was implementing this I found that BspServiceWorker saveEdges has some duplicate code
in line 1150 & 1156

{code}

    ProgressableUtils.getResultsWithNCallables(callableFactory, numThreads,
        "save-vertices-%d", getContext());

    LoggerUtils.setStatusAndLog(getContext(), LOG, Level.INFO,
      "saveEdges: Done saving edges.");
    // YARN: must complete the commit the "task" output, Hadoop isn't there.
    if (conf.isPureYarnJob() &&
      conf.getVertexOutputFormatClass() != null) {
      try {
        OutputCommitter outputCommitter =
          edgeOutputFormat.getOutputCommitter(getContext());
        if (outputCommitter.needsTaskCommit(getContext())) {

{code}

I think  {{"save-vertices-%d"}} and {{conf.getVertexOutputFormatClass()}} != null should be
changed appropriately.
Please comment.


> add support for EdgeOutputFormat also for giraph-hive
> -----------------------------------------------------
>
>                 Key: GIRAPH-745
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-745
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Claudio Martella
>            Assignee: Akila Wajirasena
>
> should be able to write edges through EdgeOutputFormat to Hive tables.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message