giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Avery Ching" <avery.ch...@gmail.com>
Subject Re: Review Request: GIRAPH-528
Date Wed, 06 Mar 2013 19:59:28 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9732/#review17487
-----------------------------------------------------------



giraph-core/src/main/java/org/apache/giraph/job/GiraphConfigurationValidator.java
<https://reviews.apache.org/r/9732/#comment37110>

    Actually let's pull the warning.  It makes the user feel like something is wrong when
this is perfectly acceptable.


- Avery Ching


On March 6, 2013, 5:45 a.m., Alessandro Presta wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9732/
> -----------------------------------------------------------
> 
> (Updated March 6, 2013, 5:45 a.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> -------
> 
> Summary of changes:
> - Vertex: only one abstract Vertex class. All vertices are mutable. Removed some unused
methods that unnecessarily exposed internals. It delegates edge storage to VertexEdges. When
an Iterable is passed to setEdges(), it checks whether it's an instance of VertexEdges (just
copy the reference) or not (create and initialize VertexEdges with the Iterable). The former
is the case with edge input (i.e. EdgeStore), the latter often happens with vertex input.
> - VertexEdges: interface for the out-edges data structure. Removed unused return values
for add()/remove(): we don't need them because PartitionStats are updated after each compute(),
so it's irrational to require them and then discard them.
> - EdgeStore: incoming edges are aggregated in the VertexEdges implementation of choice.
Moving them to the owner vertex involves simply passing a reference.
> - ImmutableClassesGiraphConfiguration: added factory methods for VertexEdges. Depending
on the needs, one can only instantiate the class, or also initialize it with either an initial
capacity or an Iterable. For example, in a VertexInputFormat, one may already have a list
of edges (call with iterable) or read the number of edges and then stream over the input (call
with capacity, then add each edge).
> - Edge: removed Comparable implementation. It was only used by a test. We shouldn't expose
more functionality to the user than is needed to write Giraph applications.
> - ByteArrayEdges: this is the default VertexEdges implementation, as it's generic and
reasonably efficient. I moved the representative edge inside ByteArrayEdgeIterator for better
encapsulation. The "only one iterator at a time" constraint differs from other implementations,
and the gain is negligible (we still have to instantiate an ExtendedDataOutput for each iterator
creation).
> - ArrayListEdges, HashMapEdges, LongDoubleArrayEdges, LongNullArrayEdges, LongDoubleHashMapEdges,
LongNullHashSetEdges: these are the other provided implementations. the first two are generic,
the second two are backed by dynamic arrays of primitives, the last two are backed by primitive
open hash maps. The iterators they return are read-only. ArrayListEdges can recognize if it's
being initialized with an ArrayList, and in that case just copies the reference (optimized
for some VertexInputFormats).
> - general cleanup: removed unneeded or buggy classes like BspUtils; fixed some tests
that could only work with Java collection-backed edges (introduced the EdgeIterables helper
class to correctly compare two edge iterables); removed the plethora of Vertex implementations,
most of which were either unused or only used by a test.
> 
> 
> This addresses bug GIRAPH-528.
>     https://issues.apache.org/jira/browse/GIRAPH-528
> 
> 
> Diffs
> -----
> 
>   giraph-accumulo/src/test/java/org/apache/giraph/io/accumulo/TestAccumuloVertexFormat.java
8894199503a972072fc69a49ad1faf5419aa4551 
>   giraph-accumulo/src/test/java/org/apache/giraph/io/accumulo/edgemarker/AccumuloEdgeInputFormat.java
30d40f621c028a5d01d64b0c035fa3adf374b73a 
>   giraph-accumulo/src/test/java/org/apache/giraph/io/accumulo/edgemarker/AccumuloEdgeOutputFormat.java
70288f68dcb3131834703f6bd0b500d238bc32bf 
>   giraph-core/pom.xml 5ce3eaa126f7f4dece5cec98f648c7da84c2646a 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/AggregatorsBenchmark.java 4e4704262bf7ce70c4584c8172a60e1f7a0aeb28

>   giraph-core/src/main/java/org/apache/giraph/benchmark/ByteArrayVertexPageRankBenchmark.java
7e51c261c5157ad0935e1388f8a9a7a9f0f38acc 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/EdgeListVertexPageRankBenchmark.java
fe1e346b39bec2eb2ee29bd68c813e5a21e35b2e 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/HashMapVertexPageRankBenchmark.java
ef3754078a373947117acd18ee75cd6ef8cf3d92 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/HashMapVertexShortestPathsBenchmark.java
a9d6deb8007f107b4c1e29b0f0eaa2d400ffe20d 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/MultiGraphByteArrayVertexPageRankBenchmark.java
9144641f3c359c28588ad489993cdb2657041b14 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/MultiGraphEdgeListVertexPageRankBenchmark.java
712ca99f9076729270619e66cb0da83fc78e0a36 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/MultiGraphRepresentativeVertexPageRankBenchmark.java
96288323e6028e779113d2520ea9edad497bb0e1 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java 06ee80cff101e3b759e19703cd2915209629d895

>   giraph-core/src/main/java/org/apache/giraph/benchmark/PageRankComputation.java 57cc201e5a01e9a9c4307a82867ceb1322697646

>   giraph-core/src/main/java/org/apache/giraph/benchmark/PageRankVertex.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java c8e33dde71019ac249ffec43e0bfdebca0920637

>   giraph-core/src/main/java/org/apache/giraph/benchmark/RepresentativeVertexPageRankBenchmark.java
331ae41a2c0df6b124cbf33944b05f080b49ce94 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/ShortestPathsBenchmark.java 1843da92119cb31025a000f02b0e84a5e47dcc20

>   giraph-core/src/main/java/org/apache/giraph/benchmark/ShortestPathsComputation.java
19f557509c38dd9a2f74fbdaad06a89d30b29ff7 
>   giraph-core/src/main/java/org/apache/giraph/benchmark/ShortestPathsVertex.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/bsp/BspOutputFormat.java 65c3ebc7974574f9814ab821d1c63237c8a49037

>   giraph-core/src/main/java/org/apache/giraph/bsp/BspUtils.java 335047d506f15c6047c546df60a15481bfaa0cae

>   giraph-core/src/main/java/org/apache/giraph/comm/MsgList.java 140b6e836dad8eeb099e14c994c2d40e0269acac

>   giraph-core/src/main/java/org/apache/giraph/comm/SendEdgeCache.java f239c1a6a47620c49d4dc3e71066f7629fa5b15d

>   giraph-core/src/main/java/org/apache/giraph/comm/SendMutationsCache.java 9ea968a84abab1081be052692445ebb3db145dcf

>   giraph-core/src/main/java/org/apache/giraph/comm/SendPartitionCache.java 911b6ba42a2f338a71aa247fc70f0f85541d9c50

>   giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java 7b4baa1a7ffc74343cb959e1df23aa9ab24f7376

>   giraph-core/src/main/java/org/apache/giraph/comm/WorkerClientRequestProcessor.java
43311f4e4fe7d61fd77fa91ae23280c02a4940a7 
>   giraph-core/src/main/java/org/apache/giraph/comm/messages/DiskBackedMessageStoreByPartition.java
6e6cb9b2382c702a6040691634f7f623effae9ba 
>   giraph-core/src/main/java/org/apache/giraph/comm/messages/SequentialFileMessageStore.java
c2c6141cc3efc243ee6b1a6c4aea570b542d541f 
>   giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java
0fc1858f16c0302c60064c912f3549981eb756b7 
>   giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java 697b6ce54afb1339c5fd52ab7add229a8afd19d0

>   giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerEdgesRequest.java
f301bbfd89397c716f47a0885b9fa085eedca742 
>   giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java 5c2a01a226202eb1ceba5857f140eef4ed31ed48

>   giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java 3ea8d3be522afd5b58710654fda1e8fceeb9d207

>   giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java fcdd57bdf7a44cf222b5d8b1e597216d92b80551

>   giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java
e6c4cc640515fddf15cff024fa783cf7eaff890b 
>   giraph-core/src/main/java/org/apache/giraph/edge/ArrayListEdges.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/ByteArrayEdges.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/ConfigurableVertexEdges.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/edge/DefaultEdge.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/Edge.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/EdgeFactory.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/EdgeNoValue.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/HashMapEdges.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/HashMultimapEdges.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/edge/LongDoubleArrayEdges.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/edge/LongDoubleHashMapEdges.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/edge/LongNullArrayEdges.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/edge/LongNullHashSetEdges.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/edge/MultiRandomAccessVertexEdges.java
PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/MutableEdge.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/ReuseObjectsVertexEdges.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/edge/StrictRandomAccessVertexEdges.java
PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/VertexEdges.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/edge/package-info.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/graph/ComputeCallable.java c7aff7c220214ce0220000ab009a9bf8407486cc

>   giraph-core/src/main/java/org/apache/giraph/graph/DefaultEdge.java 039f0d7f876800d186780d7296ed0e86601abf07

>   giraph-core/src/main/java/org/apache/giraph/graph/DefaultVertexResolver.java c88b2b9cedf1c29bad31e4ffb821a4c6a953ea95

>   giraph-core/src/main/java/org/apache/giraph/graph/Edge.java 185e3c357a8cc163d85c1b235e57bb9e46e74054

>   giraph-core/src/main/java/org/apache/giraph/graph/EdgeFactory.java a3e6efb311091446d7baaa21f46845460815c770

>   giraph-core/src/main/java/org/apache/giraph/graph/EdgeNoValue.java 4ac6759724a4ae6542071584c8638fb6c067ba59

>   giraph-core/src/main/java/org/apache/giraph/graph/EdgeStore.java 621036787e95cf340e3a873e5a5be33d02ab447f

>   giraph-core/src/main/java/org/apache/giraph/graph/GiraphTransferRegulator.java 726c21e35a04fb86e7177cdb5b2bdae6ed46a104

>   giraph-core/src/main/java/org/apache/giraph/graph/GraphTaskManager.java 20fa5c50ed38ba2b70b5471354093a01e5440b56

>   giraph-core/src/main/java/org/apache/giraph/graph/MutableEdge.java 52e4c47f12d710edba6cb011ef20ee8119d0aa46

>   giraph-core/src/main/java/org/apache/giraph/graph/ReverseEdgeDuplicator.java 4415cc219d8b977210f04cfd4d82cdc2015029e9

>   giraph-core/src/main/java/org/apache/giraph/graph/Vertex.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/graph/VertexChanges.java ef61dbb47639b573eff93cca6da1c76f9facf1e2

>   giraph-core/src/main/java/org/apache/giraph/graph/VertexMutations.java fa333412eed62793d87665d1630a8ce6f6faa1d6

>   giraph-core/src/main/java/org/apache/giraph/graph/VertexResolver.java 4a36706f39077f0d64505138d1c677ae20b84a6a

>   giraph-core/src/main/java/org/apache/giraph/io/EdgeReader.java ed6fad1619f1e71316f062bc073a0fe0bbd83404

>   giraph-core/src/main/java/org/apache/giraph/io/ReverseEdgeDuplicator.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/io/VertexReader.java 1b1c896eb7b0578d1e8110650e1528120f865935

>   giraph-core/src/main/java/org/apache/giraph/io/VertexValueReader.java 923ca5c55bf6ac8d12b8662a36dab26e8d128a8f

>   giraph-core/src/main/java/org/apache/giraph/io/VertexWriter.java 82a19bb7f834327913f2db711a2c5c2c570329e3

>   giraph-core/src/main/java/org/apache/giraph/io/formats/AdjacencyListTextVertexInputFormat.java
5092352d78520d5e27b36c3ca93cd0e0bed2a8e6 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/AdjacencyListTextVertexOutputFormat.java
934663e4cfe83225d194872aeb1c72a2bf02f414 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/IdWithValueTextOutputFormat.java
352f0540cd80b9f6d34ebc78a5203a71ed4e4244 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/IntIntNullIntTextInputFormat.java
b00e495a2deccb756353dae7b8323714b9f54ae1 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/IntNullNullNullTextInputFormat.java
dda3f2f3399728d3cf51dda6412d03aa10a744a0 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/IntNullReverseTextEdgeInputFormat.java
1e3b64342adf8be4be85e0c2be5d236233c5b7cf 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/JsonBase64VertexInputFormat.java
21ca427a6ad3b4afb2971f2e7c434fc383a39be8 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/JsonBase64VertexOutputFormat.java
0599742ee94672f7ab06c655a630bd7fcae9bc5e 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/JsonLongDoubleFloatDoubleVertexInputFormat.java
2df20f105f510c9577bcc26dabb2c86b6537b770 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/JsonLongDoubleFloatDoubleVertexOutputFormat.java
9a751ae5d2a16871386a13e0fd83e65f6d050015 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/LongDoubleDoubleAdjacencyListVertexInputFormat.java
4e352014b2cb54d921a80a5427b503b70b945d0c 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/PseudoRandomEdgeInputFormat.java
202486380d53b9654a10bf9ea2bc9c093616418a 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/PseudoRandomVertexInputFormat.java
4da8f9dcd9aa401acb9d45bb55d3acd86dae461b 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/SequenceFileVertexInputFormat.java
6a5813b2ca54e6714298bb14a3ba991038dcc6e9 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/SequenceFileVertexOutputFormat.java
0538db972fabe2ad35c8893062d672a139af60ce 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/TextDoubleDoubleAdjacencyListVertexInputFormat.java
36d00dbafb7fc0e245545e050d248cb6e2f2fa2c 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/TextEdgeInputFormat.java c9f5df1c622243f555ccaec19e3891d8369fe9fb

>   giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexInputFormat.java e359f66d211f65e27793f93e2a08848d52824383

>   giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexOutputFormat.java
9f1fe1f353530da2b3b6cc3456b2c10836b88ce6 
>   giraph-core/src/main/java/org/apache/giraph/job/GiraphConfigurationValidator.java 16b0d4829659b53f1c2b0c11fa9987a7bff2fd1e

>   giraph-core/src/main/java/org/apache/giraph/partition/ByteArrayPartition.java 2260837309b8c767af7ad310a52d0e05caad65f7

>   giraph-core/src/main/java/org/apache/giraph/partition/DiskBackedPartitionStore.java
585ab851780676a532bdfe0dce4c0e2fa5d1d317 
>   giraph-core/src/main/java/org/apache/giraph/partition/Partition.java 657c054de5b45f8e3a976e64aaee7842b2013452

>   giraph-core/src/main/java/org/apache/giraph/partition/SimplePartition.java cbf6bc33e06b62e2a8d3d255046f619e16676ed8

>   giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayEdges.java 1d8fc26b895a4ed6cb338cb5163ed6cd1b2e3a34

>   giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdData.java bd464d54c18edfd575c0df06e88720dc138157c1

>   giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdEdges.java 1cfd21e447d6a04d2eb5f1cc50fd4acad403dd4f

>   giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdMessages.java fd06783c8133c5e25d5d996a49ca3de0cf3e1be2

>   giraph-core/src/main/java/org/apache/giraph/utils/ComparisonUtils.java bb940eab1be82e4e69acf45727d3858ea59bfe44

>   giraph-core/src/main/java/org/apache/giraph/utils/ConfigurationUtils.java bd2a9c46eb17e7a42d59d55a5906385bc9a6d5b5

>   giraph-core/src/main/java/org/apache/giraph/utils/EdgeComparator.java PRE-CREATION

>   giraph-core/src/main/java/org/apache/giraph/utils/EdgeIterables.java ab288fb1ccebbc1c5446149212d3d6236699fac0

>   giraph-core/src/main/java/org/apache/giraph/utils/EmptyIterable.java 382c39c35b127a865c1d103b371f851c851155f7

>   giraph-core/src/main/java/org/apache/giraph/utils/ExtendedByteArrayDataOutput.java
247130be4501f0008cdad22a199f20eda9646d64 
>   giraph-core/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 889798fb8813fa2db005376591d07890783a5105

>   giraph-core/src/main/java/org/apache/giraph/utils/ReflectionUtils.java ae2c556c865e46abb3ce049aebd709ce73d5826b

>   giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableDoubleArrayIterator.java
cdf662ecd9bf54374f24d4caaf01d640024d566b 
>   giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableIntArrayIterator.java
c9ba0efa3bcf757477d59cd2958b87e6fd0fbce9 
>   giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableLongArrayIterator.java
c580f9c55b44a05d4125d30669e74a166c89bd6b 
>   giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableLongFloatEdgeArrayIterable.java
b7670580939a8e51a8e2eaafc9ba72592875149b 
>   giraph-core/src/main/java/org/apache/giraph/utils/UnmodifiableLongNullEdgeArrayIterable.java
18f280ae2bb15750b32bb79e6b2d0c7e8813418b 
>   giraph-core/src/main/java/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java
da6d6cd9d42baf38b180e80af0cf66212926b171 
>   giraph-core/src/main/java/org/apache/giraph/utils/WritableUtils.java 129923d917269cafa9f1c1f468488d4674a23622

>   giraph-core/src/main/java/org/apache/giraph/vertex/ByteArrayVertex.java 1e56b208a114a4dd2d987488df0ef32cf0211537

>   giraph-core/src/main/java/org/apache/giraph/vertex/ByteArrayVertexBase.java 26c3f62faccbb08f95ae62b46b9920805dbf59f9

>   giraph-core/src/main/java/org/apache/giraph/vertex/EdgeListVertex.java 882bbb8a8277d8fd8a2433d1af72e1ca559d6897

>   giraph-core/src/main/java/org/apache/giraph/vertex/EdgeListVertexBase.java ec04569669829f8df64c72de0465f9b6ed7e4628

>   giraph-core/src/main/java/org/apache/giraph/vertex/HashMapVertex.java 2160c3be096271a9c60a27b4113d607b4baa78ee

>   giraph-core/src/main/java/org/apache/giraph/vertex/IntIntNullIntVertex.java a2090e8e12bdc17fce367c47d2567b44c40898f9

>   giraph-core/src/main/java/org/apache/giraph/vertex/IntNullNullNullVertex.java f36f6dbde928c8e63cba015d26b917490657ff61

>   giraph-core/src/main/java/org/apache/giraph/vertex/LongDoubleFloatDoubleEdgeListVertex.java
9d4f6e7f513a0bf0b0322405e3e4edae689d0c10 
>   giraph-core/src/main/java/org/apache/giraph/vertex/LongDoubleFloatDoubleVertex.java
befadea477f40b1c74c6f87a461921ed8e78d78b 
>   giraph-core/src/main/java/org/apache/giraph/vertex/LongDoubleNullDoubleVertex.java
63f1d28363e1bebaeec8ac8723491ca2af085131 
>   giraph-core/src/main/java/org/apache/giraph/vertex/MultiGraphByteArrayVertex.java a50f48dbb43c5aba82450dd32911e61e33adca6b

>   giraph-core/src/main/java/org/apache/giraph/vertex/MultiGraphEdgeListVertex.java b60d8953ab3191e588c02e1503fed4bb753bd66b

>   giraph-core/src/main/java/org/apache/giraph/vertex/MultiGraphRepresentativeVertex.java
4733e2a6011ec8e1cc4eef1d2eb61abe777ec310 
>   giraph-core/src/main/java/org/apache/giraph/vertex/MutableVertex.java a6d7ce523b1b99dc9de28d3762fd3027da01506e

>   giraph-core/src/main/java/org/apache/giraph/vertex/RepresentativeVertex.java f805007b8bb8f89e9388cf89c2e81f92328b2b1c

>   giraph-core/src/main/java/org/apache/giraph/vertex/RepresentativeVertexBase.java 4de6ed85b499e74b04e93c3780324a6b9e9f2b83

>   giraph-core/src/main/java/org/apache/giraph/vertex/SimpleMutableVertex.java e12f783f407edc1bfdf4c0dcc7f2ffec526e2a78

>   giraph-core/src/main/java/org/apache/giraph/vertex/SimpleVertex.java ecea3fba7d721f0d51e003ffdc629fc4563b60f7

>   giraph-core/src/main/java/org/apache/giraph/vertex/Vertex.java a959ef8ec476fd063d88f42598aa3751d603a140

>   giraph-core/src/main/java/org/apache/giraph/vertex/package-info.java 5d983e240c715b401619a942a90fffbaac2b08a0

>   giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java 3b510b2e1b615df88010918ff222fcd5b3431077

>   giraph-core/src/main/java/org/apache/giraph/worker/DefaultWorkerContext.java 0ffdc0f129cadbb6a69836c46a9b9e13af94bd64

>   giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallable.java c577cbbedd09356db25dae3f90a7af8450e6f816

>   giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallable.java de1ae599af993f1fc2ffd1b1c151b1c287a1dad2

>   giraph-core/src/test/java/org/apache/giraph/BspCase.java 4867a5006539e6d0744cd3b401f133cbee027709

>   giraph-core/src/test/java/org/apache/giraph/bsp/BspUtilsTest.java 5c77d124d40b4938ca04907f1708f78f98f9ea0e

>   giraph-core/src/test/java/org/apache/giraph/comm/ConnectionTest.java 381ca7c3521979d59415af3992beb8ac4160c1fe

>   giraph-core/src/test/java/org/apache/giraph/comm/RequestFailureTest.java c42754fb5cc3488e3f9e8859f4bb3b425d358cb2

>   giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java 7468c477992ac5e1ca95a8997b86d230e70bc8ac

>   giraph-core/src/test/java/org/apache/giraph/comm/SaslConnectionTest.java aa89b857dd963cb98818a9060ba6e660948cd3d8

>   giraph-core/src/test/java/org/apache/giraph/comm/TestMessageStores.java 5ce7a89354639e81068360a978a8db387667550b

>   giraph-core/src/test/java/org/apache/giraph/conf/TestObjectCreation.java PRE-CREATION

>   giraph-core/src/test/java/org/apache/giraph/edge/TestMultiGraphEdges.java PRE-CREATION

>   giraph-core/src/test/java/org/apache/giraph/edge/TestMultiRandomAccessEdges.java PRE-CREATION

>   giraph-core/src/test/java/org/apache/giraph/edge/TestNullValueEdges.java PRE-CREATION

>   giraph-core/src/test/java/org/apache/giraph/edge/TestStrictGraphEdges.java PRE-CREATION

>   giraph-core/src/test/java/org/apache/giraph/edge/TestStrictRandomAccessEdges.java PRE-CREATION

>   giraph-core/src/test/java/org/apache/giraph/graph/TestVertexAndEdges.java PRE-CREATION

>   giraph-core/src/test/java/org/apache/giraph/io/TestAdjacencyListTextVertexOutputFormat.java
036924fb87c92f46f7773be74b2083313a497445 
>   giraph-core/src/test/java/org/apache/giraph/io/TestEdgeInput.java c0806221388e290ffa4e028fba679b66c57f2f5d

>   giraph-core/src/test/java/org/apache/giraph/io/TestIdWithValueTextOutputFormat.java
e77033874ed65bdab4597269e785005ff870716c 
>   giraph-core/src/test/java/org/apache/giraph/io/TestJsonBase64Format.java b3c63f6b2dfb559343d8c6a2dabe67cd3d95e672

>   giraph-core/src/test/java/org/apache/giraph/io/TestLongDoubleDoubleAdjacencyListVertexInputFormat.java
7ef8eae152abf885ada059fac61e3dbc8e13b7bc 
>   giraph-core/src/test/java/org/apache/giraph/io/TestTextDoubleDoubleAdjacencyListVertexInputFormat.java
d1e267e988b1ff3f90e56cd07b0020fe4a769b57 
>   giraph-core/src/test/java/org/apache/giraph/master/TestMasterObserver.java f67a6e0ac15e00528e95df43e54c2e1e107e4b6e

>   giraph-core/src/test/java/org/apache/giraph/partition/TestGiraphTransferRegulator.java
2671708bc024d5029fe04265ef4cff51e661a209 
>   giraph-core/src/test/java/org/apache/giraph/partition/TestPartitionStores.java b4cddf6c6f2d858b1a2ab85433dc5f4b93ebe82c

>   giraph-core/src/test/java/org/apache/giraph/utils/ComparisonUtilsTest.java 789c9eeb3b73198e31d5ac23c3d5e5603d92e479

>   giraph-core/src/test/java/org/apache/giraph/utils/MockUtils.java 04c7a3c028f91029d47448a1341256924d15830c

>   giraph-core/src/test/java/org/apache/giraph/vertex/TestIntIntNullIntVertex.java 37e87683c63562ccd76ac5b6edb5680104d63e81

>   giraph-core/src/test/java/org/apache/giraph/vertex/TestMultiGraphVertex.java 173c453ddfe5118f02ce393642e3f89901e78f2c

>   giraph-core/src/test/java/org/apache/giraph/vertex/TestMutableVertex.java 2b347c6804fd9c488406b1ebba3e91675f4626d8

>   giraph-examples/pom.xml a6255113387bbf74d33bbc2c00f0a622edbc0787 
>   giraph-examples/src/main/java/org/apache/giraph/examples/AggregatorsTestVertex.java
c00fda0cc5f6b9e113ed08c59492f7dc169c350e 
>   giraph-examples/src/main/java/org/apache/giraph/examples/ConnectedComponentsVertex.java
b5c90e5b233ace68cd90ecd0ae222c5e90aeda92 
>   giraph-examples/src/main/java/org/apache/giraph/examples/IdentityVertex.java 70000356121581d3c2ba65235401801f6dcc3a6e

>   giraph-examples/src/main/java/org/apache/giraph/examples/LongDoubleDoubleDoubleTextInputFormat.java
PRE-CREATION 
>   giraph-examples/src/main/java/org/apache/giraph/examples/LongDoubleFloatDoubleTextInputFormat.java
b1305cc74ce90fadfe740b80547aff69447fba94 
>   giraph-examples/src/main/java/org/apache/giraph/examples/NormalizingLongDoubleDoubleDoubleTextInputFormat.java
PRE-CREATION 
>   giraph-examples/src/main/java/org/apache/giraph/examples/NormalizingLongDoubleFloatDoubleTextInputFormat.java
1d41a943dab833db46b31aee09151bb471a08b59 
>   giraph-examples/src/main/java/org/apache/giraph/examples/PartitionContextTestVertex.java
f86c32397a6854ec2c2c4046adbc4f03be7b9562 
>   giraph-examples/src/main/java/org/apache/giraph/examples/RandomWalkVertex.java 5669cf27cec180599aa9da1266926a20e559d82f

>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleCheckpointVertex.java
09c7f5616931fa078fecb6816c387b806a4bc85b 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleCombinerVertex.java
be0f16cb4a1780e213238cd9510340cdbd812cdf 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleFailVertex.java 521ed97121d1f57b519f6f0fa67556e14b4b23d2

>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleInDegreeCountVertex.java
e05da3ecaced37b228cf18741e399b1f4bca4e59 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleMasterComputeVertex.java
e481ff7570d8e08006190bc560a24ee8bc7156bc 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleMsgVertex.java 8016490a60fc99dc89d19b2f5280dc9670554ba8

>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleMutateGraphVertex.java
6032e88d52ed979c356a68c2367ad0019a48477a 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleOutDegreeCountVertex.java
1e010a101babc872369061d7fdce5c0971e399ae 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java
f36f1acb3c49fa95188020ce75f1e2131f9e4040 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleShortestPathsVertex.java
1bec7ebad125bec73469a7fe0489b5b63c5ee128 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleSuperstepVertex.java
2afe15dde832249c2af6a19e1a8fe4fdc6664af8 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleTextVertexOutputFormat.java
a57c6d24297ee52157fd0651dba9c74202e8e42c 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleTriangleClosingVertex.java
469de85254e1dfd10c392f7fa615619632c1827a 
>   giraph-examples/src/main/java/org/apache/giraph/examples/SimpleVertexWithWorkerContext.java
f6488d5d0857758bd624576fbcc0ce72b0b62a8c 
>   giraph-examples/src/main/java/org/apache/giraph/examples/VerifyMessage.java 994b59dc5174b3ff657805f896a41c4229ea243a

>   giraph-examples/src/main/java/org/apache/giraph/examples/VertexWithDoubleValueDoubleEdgeTextOutputFormat.java
PRE-CREATION 
>   giraph-examples/src/main/java/org/apache/giraph/examples/VertexWithDoubleValueFloatEdgeTextOutputFormat.java
ef58bb8367e09434179d268f2a51ca748855f510 
>   giraph-examples/src/test/java/org/apache/giraph/TestBspBasic.java 0d6d1d0126744c3bec4c78b613f87cd669fabbf8

>   giraph-examples/src/test/java/org/apache/giraph/examples/ConnectedComponentsVertexTest.java
49a2f33ac26e37133fe9a518fdb7abd1adac06db 
>   giraph-examples/src/test/java/org/apache/giraph/examples/RandomWalkWithRestartVertexTest.java
4052fe1b8b5e575a79f4a07f0f95ea6e30a1f3a5 
>   giraph-examples/src/test/java/org/apache/giraph/examples/SimpleShortestPathsVertexTest.java
a372a2ddc4f5fde1e73b935d1c0aebab36326a95 
>   giraph-examples/src/test/java/org/apache/giraph/examples/SimpleTriangleClosingVertexTest.java
969c1f41e5f4a3d6f0d7085abe2d95492f92da1d 
>   giraph-examples/src/test/java/org/apache/giraph/examples/TryMultiIpcBindingPortsTest.java
2f9704d01d81a978e8c6c32b256260c97288141b 
>   giraph-examples/src/test/java/org/apache/giraph/vertex/TestVertexTypes.java bcfbd39c01825e33300b9e5ce6deff3110e0eaf7

>   giraph-hbase/src/test/java/org/apache/giraph/io/hbase/TestHBaseRootMarkerVertextFormat.java
fe5b72e4413cb640f099a6ebb0aa0aa93a499d3e 
>   giraph-hbase/src/test/java/org/apache/giraph/io/hbase/edgemarker/TableEdgeInputFormat.java
f45f07ae2340b9a0536e5bd944c8a4ff53ade089 
>   giraph-hbase/src/test/java/org/apache/giraph/io/hbase/edgemarker/TableEdgeOutputFormat.java
a1313570099e9d3075dc12f89be9a6c3091faae5 
>   giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatGiraphRunner.java 1bd0235f997ab93c2029e11c4b71230837a04cff

>   giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatalogEdgeInputFormat.java
d8987da2c7dfbcc47e379e5a61e965bd97c7b1d5 
>   giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexInputFormat.java
9d52b64751dffc63baa4e39331337892d8f668d2 
>   giraph-hcatalog/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexOutputFormat.java
4bab7dd09248c4ed5cfc0be51ee5b7f75e0cb3a6 
>   giraph-hive/src/main/java/org/apache/giraph/hive/HiveGiraphRunner.java cef96b530f65cde8ee5cc8ef30dd8a242aea6f76

>   giraph-hive/src/main/java/org/apache/giraph/hive/input/edge/HiveEdgeReader.java 34dc4df1f654ef8cc1888802124041cbf9378ab2

>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveToVertex.java 31f0e64b2b15d45be8e6630a4e8a383784564420

>   giraph-hive/src/main/java/org/apache/giraph/hive/input/vertex/HiveVertexReader.java
c5974de8a0e457b54b273086cd3ab05b5b27042b 
>   giraph-hive/src/main/java/org/apache/giraph/hive/output/HiveVertexWriter.java ea24fc526de052ef8a1d53680a1c9fadda32a0a0

>   giraph-hive/src/main/java/org/apache/giraph/hive/output/VertexToHive.java a72cf0b8ec6a136326870049759b92c5b064d6f7

>   pom.xml 901e31fcc0bc868b8742401cd20e01b06fc60bcd 
> 
> Diff: https://reviews.apache.org/r/9732/diff/
> 
> 
> Testing
> -------
> 
> 1) mvn verify
> 2) benchmarks (https://gist.github.com/apresta/7b8ff52951da76ee8a47)
> In short: no significant differences when using byte arrays. Using primitive arrays is
faster, and becomes feasible when using edge input too (no copies are performed when moving
edges to their source vertices).
> 3) check results on real dataset (planned)
> 
> 
> Thanks,
> 
> Alessandro Presta
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message