giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Akila Wajirasena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-745) add support for EdgeOutputFormat also for giraph-hive
Date Sun, 18 May 2014 15:40:38 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14001099#comment-14001099
] 

Akila Wajirasena commented on GIRAPH-745:
-----------------------------------------

In addition we cannot use both {{EdgeOutputFormat}} and {{VertexOutputFormat}} in the same
time. Because according to this method {{BspOutputFormat}} will return only {{VertexOutputCommiter}}
even if both output formats use.

{code}
  public OutputCommitter getOutputCommitter(TaskAttemptContext context)
    throws IOException, InterruptedException {
    ImmutableClassesGiraphConfiguration conf =
        new ImmutableClassesGiraphConfiguration(context.getConfiguration());
    if (!conf.hasVertexOutputFormat() && !conf.hasEdgeOutputFormat()) {
      LOG.warn("getOutputCommitter: Returning " +
          "ImmutableOutputCommiter (does nothing).");
      return new ImmutableOutputCommitter();
    }

    if (conf.hasVertexOutputFormat()) {
      return conf.createWrappedVertexOutputFormat().getOutputCommitter(context);
    } else {
      return conf.createWrappedEdgeOutputFormat().getOutputCommitter(context);
    }
  }
{code}

I tested this with hive output formats and edge output will be written correctly only if we
don't have a vertex output format.

> add support for EdgeOutputFormat also for giraph-hive
> -----------------------------------------------------
>
>                 Key: GIRAPH-745
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-745
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Claudio Martella
>            Assignee: Akila Wajirasena
>
> should be able to write edges through EdgeOutputFormat to Hive tables.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message