hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject Re: loading vertices into RAM
Date Thu, 16 Jan 2014 10:09:59 GMT
of course you would also need to add DirectMemory [1][2] (core and kryo
modules) to the classpath (e.g. if you're triggering jobs by scripts).

Tommaso

[1] :
http://repo1.maven.org/maven2/org/apache/directmemory/directmemory-cache/0.2/directmemory-cache-0.2.jar
[2] :
http://repo1.maven.org/maven2/org/apache/directmemory/directmemory-kryo/0.2/directmemory-kryo-0.2.jar


2014/1/16 Tommaso Teofili <tommaso.teofili@gmail.com>

> it's in hama-graph 0.6.3 [1].
>
> Tommaso
>
>
> [1] :
> http://grepcode.com/file/repo1.maven.org$maven2@org.apache.hama$hama-graph@0.6.3@org$apache$hama$graph$OffHeapVerticesInfo.java
>
>
> 2014/1/15 Edward J. Yoon <edwardyoon@apache.org>
>
>> Today I tried to use off-heap vertices info but it throws Class not
>> found exceptions. :/ Which jar need to be included to use it?
>>
>> On Wed, Jan 15, 2014 at 3:59 PM, Tommaso Teofili
>> <tommaso.teofili@gmail.com> wrote:
>> > and OffHeapVerticesInfo for loading vertices off heap, which is
>> available
>> > with 0.6.3 as well if I recall correctly.
>> > Tommaso
>> >
>> >
>> > 2014/1/15 Edward J. Yoon <edwardyoon@apache.org>
>> >
>> >> There are few implementations.
>> >>
>> >>  - ListVerticesinfo: loads vertices into array list.
>> >>  - MapVerticesinfo: loads vertices into tree map.
>> >>  - DiskVerticesInfo: loads vertices into a local file.
>> >>
>> >> You can choose one of them by setting the "hama.graph.vertices.info"
>> >> in job configuration.
>> >>
>> >>   > conf.setClass("hama.graph.vertices.info",
>> >> "org.apache.hama.graph.ListVerticesInfo".
>> >>
>> >> With the latest 0.6.3 version, you can use only ListVerticesInfo.
>> >> Please use the TRUNK.
>> >>
>> >>
>> >> On Tue, Jan 14, 2014 at 11:18 PM, Ammar Sahib <ammar.sahib@yahoo.com>
>> >> wrote:
>> >> > Hi
>> >> >
>> >> > According to the BSP model, the data is processed in the RAM and
>> that is
>> >> the reason why Pregel model is faster than the MapReduce (MapReduce
>> >> writedown to disk). Can someone explains to me how to be sure that all
>> the
>> >> graph vertices are actually been loaded in RAM?
>> >> >
>> >> >
>> >> > How would HAMA behave if the vertices values are so big such that the
>> >> available RAM memory is not enough to contains all of the vertices?
>> >> >
>> >> > Regards
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message