hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject Re: where is OffHeapVerticesInfo?
Date Fri, 22 May 2015 12:27:34 GMT
ok, thanks Edward for the clarification.

I would have liked an heads up here on dev@ to see if I could help there,
as systems low on memory would still benefit from that feature.
I will investigate if we can revamp OffHeapVerticesInfo to play well with
multi threading and let you know.

Regards,
Tommaso

2015-05-22 13:52 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org>:

> Hi,
>
> To improve the graph job performance, I used multi-threading for
> computing vertices[1]. Because of this, the VerticesInfo would surely
> need to be a thread-safe collection. MapVerticesInfo uses
> ConcurrentHashMap and putIfAbsent to guarantee concurrency. I also
> tried to keep OffHeapVerticesInfo, but DirectMemory buffers seems not
> thread safe (If I wrong Please let me know).
>
> If there's way to solve the concurrency, I think we can add again. :)
>
> 1.
> https://issues.apache.org/jira/browse/HAMA-946?focusedCommentId=14489195&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14489195
>
> On Fri, May 22, 2015 at 8:17 PM, Tommaso Teofili
> <tommaso.teofili@gmail.com> wrote:
> > ok, I found it out myself by quickly going through commits at [1].
> >
> > My question then is how does removal of that class relate to improving
> > graph package ?
> > Changing things in a non backward compatible way may still be acceptable
> > since we are 0.x but I think we (Edward, in this case) really need to
> make
> > it clear if we intend to remove features (removing that class means
> > removing the feature of storing vertices off heap) before doing that.
> >
> > Thanks and regards,
> > Tommaso
> >
> > [1] : http://svn.apache.org/viewvc?view=revision&revision=1673079
> >
> > 2015-05-22 13:13 GMT+02:00 Tommaso Teofili <tommaso.teofili@gmail.com>:
> >
> >> Hi all,
> >>
> >> I was having a look at Hama code again after a while and I cannot find
> >> anymore the stuff for off heap storage of graph vertices, was it
> removed ?
> >> If so why and what's the Jira issue for that ?
> >>
> >> Thanks and regards,
> >> Tommaso
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message