hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suraj Menon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-561) Support consuming partitioned files
Date Sun, 13 Jan 2013 21:10:12 GMT

    [ https://issues.apache.org/jira/browse/HAMA-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13552310#comment-13552310
] 

Suraj Menon commented on HAMA-561:
----------------------------------

After the patch for HAMA-700, I think this should work if we disable the Runtime partitioning.
However, if record conversion is needed, we would need to run PartitionRunner with number
of tasks equal to the number of splits and the RecordConverter should return the peer index
in the function getPartitionId .
                
> Support consuming partitioned files
> -----------------------------------
>
>                 Key: HAMA-561
>                 URL: https://issues.apache.org/jira/browse/HAMA-561
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp core
>            Reporter: praveen sripati
>            Priority: Critical
>             Fix For: 0.7.0
>
>
> Current the input partitioning is done when the job is submitted and the partitioner
has been specified. There might be a scenario where the input data has already been partiononed
or there might be a better way of partioning of the input data. So, Hama should be made aware
that the files are already partitioned files and the messages should only be sent to the appropriate
bsp task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message