incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HAMA-576) Improve sendMessages in Vertex
Date Fri, 18 May 2012 17:55:12 GMT

     [ https://issues.apache.org/jira/browse/HAMA-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas Jungblut updated HAMA-576:
---------------------------------

    Attachment: HAMA-576_1.patch

Urgent fix for trunk, I have made the class add for the generics wrong, so ID types are overridden
by the value types.

So this patch will fix this, also I added a first testcase with pagerank (most features used
in graph package and most volatile). 
It will run a HamaTestCluster for checking serialization and stuff.

I have now made a stateful message that implies maps for meta messaging, IDs for repair functions
and vertex messages. 
The whole thing should now have less overhead, at least 4 bytes (for the size of the map)
and the string bytes for the classnames.
The classes are no instantiated by the user given classes in the job.
And I fixed some code smells like missing override annotations and stuff, my eclipse was wrongly
configured so I have introduced many of them.

Testcases are working fine, I have no running benchmark mapreduce job to check it with more
scale but I assume it works.

One note for our build guys here, I have added a test jar on the core project because the
graph project needs that hama cluster testcase.
We should include it in the dist as well for possible downstream projects.
                
> Improve sendMessages in Vertex
> ------------------------------
>
>                 Key: HAMA-576
>                 URL: https://issues.apache.org/jira/browse/HAMA-576
>             Project: Hama
>          Issue Type: Sub-task
>          Components: bsp core, graph
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
>            Priority: Blocker
>             Fix For: 0.5.0
>
>         Attachments: HAMA-576_1.patch
>
>
> Currently it is pretty inefficient solved by a MapWritable per single vertex value.
> We should move out the map from a single vertex and instead put it into a SINGLE map.
This will save us lots of space and time while messaging.
> I'm considering this as a blocker, since this is just a huge drawback in comparision
to giraph.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message