tinkerpop-commits mailing list archives: March 2015

Site index · List index
Message list1 · 2 · 3 · 4 · 5 · 6 · 7 · Next »Thread · Author · Date
ok...@apache.org incubator-tinkerpop git commit: first push on the spark adaptor....crazy dep hell. Mon, 02 Mar, 16:24
ok...@apache.org incubator-tinkerpop git commit: finally made it through the dependency maze. Spark and Giraph both cleanly work in hadoop-gremlin. Need to go deeper into the SparkGraphComputer implementation to make sure its perfect, but I think we have the pom.xml righ Mon, 02 Mar, 17:57
ok...@apache.org incubator-tinkerpop git commit: We now have a basic message passing engine implemented in Spark. Mon, 02 Mar, 21:56
ok...@apache.org incubator-tinkerpop git commit: Fix profiling of global nested steps, with test coverage. Mon, 02 Mar, 21:56
ok...@apache.org incubator-tinkerpop git commit: Have PageRankVertexProgram working properly over SparkGraphComputer -- hard coded style in a public static main(). Lots of things to do tomorrow to generalizae it and clean it all up. Tue, 03 Mar, 00:27
ok...@apache.org incubator-tinkerpop git commit: Added SparkVertex which lazily generates a serialized Vertex using transient and byte[]... this would be good in core, but for now, only used by SparkGraphComputer. Tue, 03 Mar, 00:43
ok...@apache.org incubator-tinkerpop git commit: Some really cool lazy optimizations to SparkGraphComputer. I 'get it' now. Easy peasy lemon squeezy. Tue, 03 Mar, 02:23
ok...@apache.org [1/2] incubator-tinkerpop git commit: Fix profiling of global nested steps, with test coverage. Tue, 03 Mar, 02:23
ok...@apache.org [2/2] incubator-tinkerpop git commit: Merge branch 'master' into spark Tue, 03 Mar, 02:23
spmalle...@apache.org [1/3] incubator-tinkerpop git commit: added tests for group-sideEffects Tue, 03 Mar, 11:56
spmalle...@apache.org [2/3] incubator-tinkerpop git commit: added tree tests (reducer) Tue, 03 Mar, 11:56
spmalle...@apache.org [3/3] incubator-tinkerpop git commit: Merge branch 'master' into sideeffect-reduce Tue, 03 Mar, 11:56
spmalle...@apache.org incubator-tinkerpop git commit: Add test to validate that bindings clear between scripengine evaluations. Tue, 03 Mar, 13:56
ok...@apache.org incubator-tinkerpop git commit: SparkGraphComputer is prim and proper. No longer using public static void main(). The GraphComputer API is legitamately implemented. All that is left --- MapReduce engine (easy) and GraphComputer Memory (hard). Tue, 03 Mar, 16:28
ok...@apache.org [1/4] incubator-tinkerpop git commit: added tests for group-sideEffects Tue, 03 Mar, 16:31
ok...@apache.org [2/4] incubator-tinkerpop git commit: added tree tests (reducer) Tue, 03 Mar, 16:31
ok...@apache.org [3/4] incubator-tinkerpop git commit: Merge branch 'master' into sideeffect-reduce Tue, 03 Mar, 16:31
ok...@apache.org [4/4] incubator-tinkerpop git commit: Merge branch 'master' into spark Tue, 03 Mar, 16:31
ok...@apache.org incubator-tinkerpop git commit: Spark jar cache supported for sending HADOOP_GREMLIN_LIBS jars to the cluster. Tue, 03 Mar, 16:46
spmalle...@apache.org [1/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 17:23
spmalle...@apache.org [2/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 17:23
spmalle...@apache.org [3/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 17:23
spmalle...@apache.org [4/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 17:23
spmalle...@apache.org [5/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 17:23
spmalle...@apache.org [6/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 17:23
ok...@apache.org incubator-tinkerpop git commit: Spark MapReduce engine built. It was more complex than I suspected. Will definately need to break up SparkGraphComputer in various XXXHelper classes with static methods... its pretty beefy right now. All that is left is Me Tue, 03 Mar, 18:05
ok...@apache.org [1/8] incubator-tinkerpop git commit: Add test to validate that bindings clear between scripengine evaluations. Tue, 03 Mar, 18:10
ok...@apache.org [2/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 18:10
ok...@apache.org [3/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 18:10
ok...@apache.org [4/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 18:10
ok...@apache.org [5/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 18:10
ok...@apache.org [6/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 18:10
ok...@apache.org [7/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. Tue, 03 Mar, 18:10
ok...@apache.org [8/8] incubator-tinkerpop git commit: fixed up spark/ branch now that is is Gryo and .kryo. @spmallette Tue, 03 Mar, 18:10
ok...@apache.org incubator-tinkerpop git commit: SparkGraphComputer implemented. There is still lots of cleanup and some optimizations that can be added, but the semantics are correct and 90 percent of the test cases are passing. Having some weird serialization issues on Tue, 03 Mar, 20:50
spmalle...@apache.org [1/3] incubator-tinkerpop git commit: Delete unused method. Tue, 03 Mar, 22:07
spmalle...@apache.org [2/3] incubator-tinkerpop git commit: Refactor OpProcessor to enable simple overrides. Tue, 03 Mar, 22:07
spmalle...@apache.org [3/3] incubator-tinkerpop git commit: Merge branch 'refactorForDashboardFix' of https://github.com/rjbriody/incubator-tinkerpop Tue, 03 Mar, 22:07
ok...@apache.org incubator-tinkerpop git commit: lots of clean up and organization. SparkGraphComputer is now really clean with all the dirty work being done by SparkHelper. Tue, 03 Mar, 22:31
ok...@apache.org [1/4] incubator-tinkerpop git commit: Delete unused method. Tue, 03 Mar, 22:34
ok...@apache.org [2/4] incubator-tinkerpop git commit: Refactor OpProcessor to enable simple overrides. Tue, 03 Mar, 22:34
ok...@apache.org [3/4] incubator-tinkerpop git commit: Merge branch 'refactorForDashboardFix' of https://github.com/rjbriody/incubator-tinkerpop Tue, 03 Mar, 22:34
ok...@apache.org [4/4] incubator-tinkerpop git commit: Merge branch 'master' into spark Tue, 03 Mar, 22:34
ok...@apache.org incubator-tinkerpop git commit: OMG -- Apache Configuration is the wooooorst --- the auto , deliminator ... total hole for the last hour. Tue, 03 Mar, 23:25
ok...@apache.org incubator-tinkerpop git commit: lots of cleanups and optimizations. a few bugs in Spark around the nitty gritty semantics of GraphComputer (incr, and, or...). Will fix up tomorrow. Wed, 04 Mar, 00:01
spmalle...@apache.org incubator-tinkerpop git commit: Make the handleIterator method throw a checked exception. Wed, 04 Mar, 12:17
spmalle...@apache.org incubator-tinkerpop git commit: Simplify the method signature of handleIterator. Wed, 04 Mar, 12:24
spmalle...@apache.org incubator-tinkerpop git commit: Moved stopwatch functionality into handleIterator. Wed, 04 Mar, 12:55
ok...@apache.org incubator-tinkerpop git commit: fixed an iteration offset bug in SparkGraphComputer. Wed, 04 Mar, 14:24
spmalle...@apache.org incubator-tinkerpop git commit: Update javadoc a bit. Wed, 04 Mar, 14:26
spmalle...@apache.org [1/2] incubator-tinkerpop git commit: Make SessionOpProcessor evalOp protected. Wed, 04 Mar, 14:32
spmalle...@apache.org [2/2] incubator-tinkerpop git commit: Merge branch 'master' of https://github.com/rjbriody/incubator-tinkerpop Wed, 04 Mar, 14:32
ok...@apache.org [1/8] incubator-tinkerpop git commit: Make the handleIterator method throw a checked exception. Wed, 04 Mar, 14:41
ok...@apache.org [2/8] incubator-tinkerpop git commit: Simplify the method signature of handleIterator. Wed, 04 Mar, 14:41
ok...@apache.org [3/8] incubator-tinkerpop git commit: Moved stopwatch functionality into handleIterator. Wed, 04 Mar, 14:41
ok...@apache.org [4/8] incubator-tinkerpop git commit: Update javadoc a bit. Wed, 04 Mar, 14:41
ok...@apache.org [5/8] incubator-tinkerpop git commit: Make SessionOpProcessor evalOp protected. Wed, 04 Mar, 14:41
ok...@apache.org [6/8] incubator-tinkerpop git commit: Merge branch 'master' of https://github.com/rjbriody/incubator-tinkerpop Wed, 04 Mar, 14:41
ok...@apache.org [7/8] incubator-tinkerpop git commit: random tweaks. about to merge master. will work from there now. Wed, 04 Mar, 14:41
ok...@apache.org [8/8] incubator-tinkerpop git commit: Merge branch 'master' into spark Wed, 04 Mar, 14:41
ok...@apache.org [01/20] incubator-tinkerpop git commit: first push on the spark adaptor....crazy dep hell. Wed, 04 Mar, 14:41
ok...@apache.org [02/20] incubator-tinkerpop git commit: finally made it through the dependency maze. Spark and Giraph both cleanly work in hadoop-gremlin. Need to go deeper into the SparkGraphComputer implementation to make sure its perfect, but I think we have the pom. Wed, 04 Mar, 14:41
ok...@apache.org [03/20] incubator-tinkerpop git commit: We now have a basic message passing engine implemented in Spark. Wed, 04 Mar, 14:41
ok...@apache.org [04/20] incubator-tinkerpop git commit: Have PageRankVertexProgram working properly over SparkGraphComputer -- hard coded style in a public static main(). Lots of things to do tomorrow to generalizae it and clean it all up. Wed, 04 Mar, 14:41
ok...@apache.org [05/20] incubator-tinkerpop git commit: Added SparkVertex which lazily generates a serialized Vertex using transient and byte[]... this would be good in core, but for now, only used by SparkGraphComputer. Wed, 04 Mar, 14:42
ok...@apache.org [06/20] incubator-tinkerpop git commit: Some really cool lazy optimizations to SparkGraphComputer. I 'get it' now. Easy peasy lemon squeezy. Wed, 04 Mar, 14:42
ok...@apache.org [07/20] incubator-tinkerpop git commit: Merge branch 'master' into spark Wed, 04 Mar, 14:42
ok...@apache.org [08/20] incubator-tinkerpop git commit: SparkGraphComputer is prim and proper. No longer using public static void main(). The GraphComputer API is legitamately implemented. All that is left --- MapReduce engine (easy) and GraphComputer Memory (hard). Wed, 04 Mar, 14:42
ok...@apache.org [09/20] incubator-tinkerpop git commit: Merge branch 'master' into spark Wed, 04 Mar, 14:42
ok...@apache.org [10/20] incubator-tinkerpop git commit: Spark jar cache supported for sending HADOOP_GREMLIN_LIBS jars to the cluster. Wed, 04 Mar, 14:42
ok...@apache.org [11/20] incubator-tinkerpop git commit: Spark MapReduce engine built. It was more complex than I suspected. Will definately need to break up SparkGraphComputer in various XXXHelper classes with static methods... its pretty beefy right now. All that is le Wed, 04 Mar, 14:42
ok...@apache.org [12/20] incubator-tinkerpop git commit: fixed up spark/ branch now that is is Gryo and .kryo. @spmallette Wed, 04 Mar, 14:42
ok...@apache.org [13/20] incubator-tinkerpop git commit: SparkGraphComputer implemented. There is still lots of cleanup and some optimizations that can be added, but the semantics are correct and 90 percent of the test cases are passing. Having some weird serialization i Wed, 04 Mar, 14:42
ok...@apache.org [14/20] incubator-tinkerpop git commit: lots of clean up and organization. SparkGraphComputer is now really clean with all the dirty work being done by SparkHelper. Wed, 04 Mar, 14:42
ok...@apache.org [15/20] incubator-tinkerpop git commit: Merge branch 'master' into spark Wed, 04 Mar, 14:42
ok...@apache.org [16/20] incubator-tinkerpop git commit: OMG -- Apache Configuration is the wooooorst --- the auto , deliminator ... total hole for the last hour. Wed, 04 Mar, 14:42
ok...@apache.org [17/20] incubator-tinkerpop git commit: lots of cleanups and optimizations. a few bugs in Spark around the nitty gritty semantics of GraphComputer (incr, and, or...). Will fix up tomorrow. Wed, 04 Mar, 14:42
ok...@apache.org [18/20] incubator-tinkerpop git commit: fixed an iteration offset bug in SparkGraphComputer. Wed, 04 Mar, 14:42
ok...@apache.org [19/20] incubator-tinkerpop git commit: random tweaks. about to merge master. will work from there now. Wed, 04 Mar, 14:42
ok...@apache.org [20/20] incubator-tinkerpop git commit: Merge branch 'master' into spark Wed, 04 Mar, 14:42
ok...@apache.org incubator-tinkerpop git commit: fixed a bug in GraphComputerTest and ElementHelperTest has two commented out tests because null does not equal Vertex. Wed, 04 Mar, 15:03
ok...@apache.org incubator-tinkerpop git commit: fixing up HadoopGremlinPlugin to work for both Spark and Giraph. A few dangling oddities that I will fix up in a few. Added SparkGraphComputer to the documentation. Wed, 04 Mar, 16:58
ok...@apache.org incubator-tinkerpop git commit: got some more lazy out of SparkGraphComputer message passer. Wed, 04 Mar, 19:47
ok...@apache.org incubator-tinkerpop git commit: lots of optimization goodies here. learning more and more about Spark as I go and getting clever about it. My powers are increasing. Soon I will crush you all with my army of machines. Wed, 04 Mar, 20:35
ok...@apache.org incubator-tinkerpop git commit: SparkGraphComputer is becoming a well oiled machine... so many heres and theres that are making it more and more memory efficient and lazy. Wed, 04 Mar, 21:12
ok...@apache.org incubator-tinkerpop git commit: killin it over here. another massive reduction in object creation. Wed, 04 Mar, 21:21
ok...@apache.org incubator-tinkerpop git commit: about to dive into something big. Wed, 04 Mar, 22:27
ok...@apache.org incubator-tinkerpop git commit: introduced the concept of a Payload where there are VertexPayloads and MessagesPayloads. This greatly reduces the binary footprint of a message in SparkGraphComputer. Wed, 04 Mar, 23:26
ok...@apache.org incubator-tinkerpop git commit: more minor realizations about Spark. Wed, 04 Mar, 23:52
spmalle...@apache.org incubator-tinkerpop git commit: Remove "cluster" stuff from gremlin-core. Thu, 05 Mar, 12:53
spmalle...@apache.org incubator-tinkerpop git commit: Refactored TinkerWorkerPool to allow reuse of it for mapreduce or vertexprograms. Thu, 05 Mar, 13:24
spmalle...@apache.org incubator-tinkerpop git commit: Add test to validate that it is possible to compile scripts without bindings. Thu, 05 Mar, 15:42
spmalle...@apache.org [1/2] incubator-tinkerpop git commit: Fix units on metrics duration for GraphSON Thu, 05 Mar, 16:17
spmalle...@apache.org [2/2] incubator-tinkerpop git commit: Merge branch 'fixMetricsDurationUnitsInGraphSON' of https://github.com/rjbriody/incubator-tinkerpop Thu, 05 Mar, 16:17
ok...@apache.org incubator-tinkerpop git commit: smart about caching RDDs in SparkGraphComputer and now you don't need to read the RDD off of disk for each MapReduce job. Tests run significantly faster. Thu, 05 Mar, 17:25
spmalle...@apache.org incubator-tinkerpop git commit: Added compile() method to GremlinExecutor. Thu, 05 Mar, 19:34
spmalle...@apache.org incubator-tinkerpop git commit: Change API for GremlinExecutor to take null for language rather than Optional<String> Thu, 05 Mar, 19:49
ok...@apache.org [1/3] incubator-tinkerpop git commit: fixed a obnoxious bug around GraphMemory in SparkGraphComputer --- had to use a combination of BroadcastVariables and Accumulators. What a time pit that was. Thu, 05 Mar, 22:47
ok...@apache.org [2/3] incubator-tinkerpop git commit: GraphComputer Memory no longer returns the mutated value on worker set -- its too difficult to get right in distributed systems. Thu, 05 Mar, 22:47
ok...@apache.org [3/3] incubator-tinkerpop git commit: Minor tweak to CHANGELOG. Thu, 05 Mar, 22:47
Message list1 · 2 · 3 · 4 · 5 · 6 · 7 · Next »Thread · Author · Date
Box list
Feb 2019189
Jan 2019487
Dec 2018433
Nov 2018459
Oct 2018943
Sep 2018516
Aug 2018960
Jul 2018605
Jun 2018845
May 2018866
Apr 20181008
Mar 20181185
Feb 2018379
Jan 2018723
Dec 2017333
Nov 2017779
Oct 2017473
Sep 2017884
Aug 2017494
Jul 2017992
Jun 20171082
May 2017660
Apr 2017337
Mar 2017729
Feb 20171052
Jan 20171626
Dec 2016869
Nov 20161023
Oct 20161010
Sep 2016910
Aug 2016598
Jul 2016537
Jun 2016787
May 2016660
Apr 2016422
Mar 20161169
Feb 2016852
Jan 2016316
Dec 2015523
Nov 2015485
Oct 2015838
Sep 2015535
Aug 2015322
Jul 2015480
Jun 2015758
May 2015909
Apr 2015954
Mar 2015613
Feb 2015351
Jan 20159