| ok...@apache.org |
incubator-tinkerpop git commit: first push on the spark adaptor....crazy dep hell. |
Mon, 02 Mar, 16:24 |
| ok...@apache.org |
incubator-tinkerpop git commit: finally made it through the dependency maze. Spark and Giraph both cleanly work in hadoop-gremlin. Need to go deeper into the SparkGraphComputer implementation to make sure its perfect, but I think we have the pom.xml righ |
Mon, 02 Mar, 17:57 |
| ok...@apache.org |
incubator-tinkerpop git commit: We now have a basic message passing engine implemented in Spark. |
Mon, 02 Mar, 21:56 |
| ok...@apache.org |
incubator-tinkerpop git commit: Fix profiling of global nested steps, with test coverage. |
Mon, 02 Mar, 21:56 |
| ok...@apache.org |
incubator-tinkerpop git commit: Have PageRankVertexProgram working properly over SparkGraphComputer -- hard coded style in a public static main(). Lots of things to do tomorrow to generalizae it and clean it all up. |
Tue, 03 Mar, 00:27 |
| ok...@apache.org |
incubator-tinkerpop git commit: Added SparkVertex which lazily generates a serialized Vertex using transient and byte[]... this would be good in core, but for now, only used by SparkGraphComputer. |
Tue, 03 Mar, 00:43 |
| ok...@apache.org |
incubator-tinkerpop git commit: Some really cool lazy optimizations to SparkGraphComputer. I 'get it' now. Easy peasy lemon squeezy. |
Tue, 03 Mar, 02:23 |
| ok...@apache.org |
[1/2] incubator-tinkerpop git commit: Fix profiling of global nested steps, with test coverage. |
Tue, 03 Mar, 02:23 |
| ok...@apache.org |
[2/2] incubator-tinkerpop git commit: Merge branch 'master' into spark |
Tue, 03 Mar, 02:23 |
| spmalle...@apache.org |
[1/3] incubator-tinkerpop git commit: added tests for group-sideEffects |
Tue, 03 Mar, 11:56 |
| spmalle...@apache.org |
[2/3] incubator-tinkerpop git commit: added tree tests (reducer) |
Tue, 03 Mar, 11:56 |
| spmalle...@apache.org |
[3/3] incubator-tinkerpop git commit: Merge branch 'master' into sideeffect-reduce |
Tue, 03 Mar, 11:56 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Add test to validate that bindings clear between scripengine evaluations. |
Tue, 03 Mar, 13:56 |
| ok...@apache.org |
incubator-tinkerpop git commit: SparkGraphComputer is prim and proper. No longer using public static void main(). The GraphComputer API is legitamately implemented. All that is left --- MapReduce engine (easy) and GraphComputer Memory (hard). |
Tue, 03 Mar, 16:28 |
| ok...@apache.org |
[1/4] incubator-tinkerpop git commit: added tests for group-sideEffects |
Tue, 03 Mar, 16:31 |
| ok...@apache.org |
[2/4] incubator-tinkerpop git commit: added tree tests (reducer) |
Tue, 03 Mar, 16:31 |
| ok...@apache.org |
[3/4] incubator-tinkerpop git commit: Merge branch 'master' into sideeffect-reduce |
Tue, 03 Mar, 16:31 |
| ok...@apache.org |
[4/4] incubator-tinkerpop git commit: Merge branch 'master' into spark |
Tue, 03 Mar, 16:31 |
| ok...@apache.org |
incubator-tinkerpop git commit: Spark jar cache supported for sending HADOOP_GREMLIN_LIBS jars to the cluster. |
Tue, 03 Mar, 16:46 |
| spmalle...@apache.org |
[1/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 17:23 |
| spmalle...@apache.org |
[2/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 17:23 |
| spmalle...@apache.org |
[3/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 17:23 |
| spmalle...@apache.org |
[4/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 17:23 |
| spmalle...@apache.org |
[5/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 17:23 |
| spmalle...@apache.org |
[6/6] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 17:23 |
| ok...@apache.org |
incubator-tinkerpop git commit: Spark MapReduce engine built. It was more complex than I suspected. Will definately need to break up SparkGraphComputer in various XXXHelper classes with static methods... its pretty beefy right now. All that is left is Me |
Tue, 03 Mar, 18:05 |
| ok...@apache.org |
[1/8] incubator-tinkerpop git commit: Add test to validate that bindings clear between scripengine evaluations. |
Tue, 03 Mar, 18:10 |
| ok...@apache.org |
[2/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 18:10 |
| ok...@apache.org |
[3/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 18:10 |
| ok...@apache.org |
[4/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 18:10 |
| ok...@apache.org |
[5/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 18:10 |
| ok...@apache.org |
[6/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 18:10 |
| ok...@apache.org |
[7/8] incubator-tinkerpop git commit: Renamed Gremlin Kryo to simply Gryo. |
Tue, 03 Mar, 18:10 |
| ok...@apache.org |
[8/8] incubator-tinkerpop git commit: fixed up spark/ branch now that is is Gryo and .kryo. @spmallette |
Tue, 03 Mar, 18:10 |
| ok...@apache.org |
incubator-tinkerpop git commit: SparkGraphComputer implemented. There is still lots of cleanup and some optimizations that can be added, but the semantics are correct and 90 percent of the test cases are passing. Having some weird serialization issues on |
Tue, 03 Mar, 20:50 |
| spmalle...@apache.org |
[1/3] incubator-tinkerpop git commit: Delete unused method. |
Tue, 03 Mar, 22:07 |
| spmalle...@apache.org |
[2/3] incubator-tinkerpop git commit: Refactor OpProcessor to enable simple overrides. |
Tue, 03 Mar, 22:07 |
| spmalle...@apache.org |
[3/3] incubator-tinkerpop git commit: Merge branch 'refactorForDashboardFix' of https://github.com/rjbriody/incubator-tinkerpop |
Tue, 03 Mar, 22:07 |
| ok...@apache.org |
incubator-tinkerpop git commit: lots of clean up and organization. SparkGraphComputer is now really clean with all the dirty work being done by SparkHelper. |
Tue, 03 Mar, 22:31 |
| ok...@apache.org |
[1/4] incubator-tinkerpop git commit: Delete unused method. |
Tue, 03 Mar, 22:34 |
| ok...@apache.org |
[2/4] incubator-tinkerpop git commit: Refactor OpProcessor to enable simple overrides. |
Tue, 03 Mar, 22:34 |
| ok...@apache.org |
[3/4] incubator-tinkerpop git commit: Merge branch 'refactorForDashboardFix' of https://github.com/rjbriody/incubator-tinkerpop |
Tue, 03 Mar, 22:34 |
| ok...@apache.org |
[4/4] incubator-tinkerpop git commit: Merge branch 'master' into spark |
Tue, 03 Mar, 22:34 |
| ok...@apache.org |
incubator-tinkerpop git commit: OMG -- Apache Configuration is the wooooorst --- the auto , deliminator ... total hole for the last hour. |
Tue, 03 Mar, 23:25 |
| ok...@apache.org |
incubator-tinkerpop git commit: lots of cleanups and optimizations. a few bugs in Spark around the nitty gritty semantics of GraphComputer (incr, and, or...). Will fix up tomorrow. |
Wed, 04 Mar, 00:01 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Make the handleIterator method throw a checked exception. |
Wed, 04 Mar, 12:17 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Simplify the method signature of handleIterator. |
Wed, 04 Mar, 12:24 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Moved stopwatch functionality into handleIterator. |
Wed, 04 Mar, 12:55 |
| ok...@apache.org |
incubator-tinkerpop git commit: fixed an iteration offset bug in SparkGraphComputer. |
Wed, 04 Mar, 14:24 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Update javadoc a bit. |
Wed, 04 Mar, 14:26 |
| spmalle...@apache.org |
[1/2] incubator-tinkerpop git commit: Make SessionOpProcessor evalOp protected. |
Wed, 04 Mar, 14:32 |
| spmalle...@apache.org |
[2/2] incubator-tinkerpop git commit: Merge branch 'master' of https://github.com/rjbriody/incubator-tinkerpop |
Wed, 04 Mar, 14:32 |
| ok...@apache.org |
[1/8] incubator-tinkerpop git commit: Make the handleIterator method throw a checked exception. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[2/8] incubator-tinkerpop git commit: Simplify the method signature of handleIterator. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[3/8] incubator-tinkerpop git commit: Moved stopwatch functionality into handleIterator. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[4/8] incubator-tinkerpop git commit: Update javadoc a bit. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[5/8] incubator-tinkerpop git commit: Make SessionOpProcessor evalOp protected. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[6/8] incubator-tinkerpop git commit: Merge branch 'master' of https://github.com/rjbriody/incubator-tinkerpop |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[7/8] incubator-tinkerpop git commit: random tweaks. about to merge master. will work from there now. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[8/8] incubator-tinkerpop git commit: Merge branch 'master' into spark |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[01/20] incubator-tinkerpop git commit: first push on the spark adaptor....crazy dep hell. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[02/20] incubator-tinkerpop git commit: finally made it through the dependency maze. Spark and Giraph both cleanly work in hadoop-gremlin. Need to go deeper into the SparkGraphComputer implementation to make sure its perfect, but I think we have the pom. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[03/20] incubator-tinkerpop git commit: We now have a basic message passing engine implemented in Spark. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[04/20] incubator-tinkerpop git commit: Have PageRankVertexProgram working properly over SparkGraphComputer -- hard coded style in a public static main(). Lots of things to do tomorrow to generalizae it and clean it all up. |
Wed, 04 Mar, 14:41 |
| ok...@apache.org |
[05/20] incubator-tinkerpop git commit: Added SparkVertex which lazily generates a serialized Vertex using transient and byte[]... this would be good in core, but for now, only used by SparkGraphComputer. |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[06/20] incubator-tinkerpop git commit: Some really cool lazy optimizations to SparkGraphComputer. I 'get it' now. Easy peasy lemon squeezy. |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[07/20] incubator-tinkerpop git commit: Merge branch 'master' into spark |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[08/20] incubator-tinkerpop git commit: SparkGraphComputer is prim and proper. No longer using public static void main(). The GraphComputer API is legitamately implemented. All that is left --- MapReduce engine (easy) and GraphComputer Memory (hard). |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[09/20] incubator-tinkerpop git commit: Merge branch 'master' into spark |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[10/20] incubator-tinkerpop git commit: Spark jar cache supported for sending HADOOP_GREMLIN_LIBS jars to the cluster. |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[11/20] incubator-tinkerpop git commit: Spark MapReduce engine built. It was more complex than I suspected. Will definately need to break up SparkGraphComputer in various XXXHelper classes with static methods... its pretty beefy right now. All that is le |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[12/20] incubator-tinkerpop git commit: fixed up spark/ branch now that is is Gryo and .kryo. @spmallette |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[13/20] incubator-tinkerpop git commit: SparkGraphComputer implemented. There is still lots of cleanup and some optimizations that can be added, but the semantics are correct and 90 percent of the test cases are passing. Having some weird serialization i |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[14/20] incubator-tinkerpop git commit: lots of clean up and organization. SparkGraphComputer is now really clean with all the dirty work being done by SparkHelper. |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[15/20] incubator-tinkerpop git commit: Merge branch 'master' into spark |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[16/20] incubator-tinkerpop git commit: OMG -- Apache Configuration is the wooooorst --- the auto , deliminator ... total hole for the last hour. |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[17/20] incubator-tinkerpop git commit: lots of cleanups and optimizations. a few bugs in Spark around the nitty gritty semantics of GraphComputer (incr, and, or...). Will fix up tomorrow. |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[18/20] incubator-tinkerpop git commit: fixed an iteration offset bug in SparkGraphComputer. |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[19/20] incubator-tinkerpop git commit: random tweaks. about to merge master. will work from there now. |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
[20/20] incubator-tinkerpop git commit: Merge branch 'master' into spark |
Wed, 04 Mar, 14:42 |
| ok...@apache.org |
incubator-tinkerpop git commit: fixed a bug in GraphComputerTest and ElementHelperTest has two commented out tests because null does not equal Vertex. |
Wed, 04 Mar, 15:03 |
| ok...@apache.org |
incubator-tinkerpop git commit: fixing up HadoopGremlinPlugin to work for both Spark and Giraph. A few dangling oddities that I will fix up in a few. Added SparkGraphComputer to the documentation. |
Wed, 04 Mar, 16:58 |
| ok...@apache.org |
incubator-tinkerpop git commit: got some more lazy out of SparkGraphComputer message passer. |
Wed, 04 Mar, 19:47 |
| ok...@apache.org |
incubator-tinkerpop git commit: lots of optimization goodies here. learning more and more about Spark as I go and getting clever about it. My powers are increasing. Soon I will crush you all with my army of machines. |
Wed, 04 Mar, 20:35 |
| ok...@apache.org |
incubator-tinkerpop git commit: SparkGraphComputer is becoming a well oiled machine... so many heres and theres that are making it more and more memory efficient and lazy. |
Wed, 04 Mar, 21:12 |
| ok...@apache.org |
incubator-tinkerpop git commit: killin it over here. another massive reduction in object creation. |
Wed, 04 Mar, 21:21 |
| ok...@apache.org |
incubator-tinkerpop git commit: about to dive into something big. |
Wed, 04 Mar, 22:27 |
| ok...@apache.org |
incubator-tinkerpop git commit: introduced the concept of a Payload where there are VertexPayloads and MessagesPayloads. This greatly reduces the binary footprint of a message in SparkGraphComputer. |
Wed, 04 Mar, 23:26 |
| ok...@apache.org |
incubator-tinkerpop git commit: more minor realizations about Spark. |
Wed, 04 Mar, 23:52 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Remove "cluster" stuff from gremlin-core. |
Thu, 05 Mar, 12:53 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Refactored TinkerWorkerPool to allow reuse of it for mapreduce or vertexprograms. |
Thu, 05 Mar, 13:24 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Add test to validate that it is possible to compile scripts without bindings. |
Thu, 05 Mar, 15:42 |
| spmalle...@apache.org |
[1/2] incubator-tinkerpop git commit: Fix units on metrics duration for GraphSON |
Thu, 05 Mar, 16:17 |
| spmalle...@apache.org |
[2/2] incubator-tinkerpop git commit: Merge branch 'fixMetricsDurationUnitsInGraphSON' of https://github.com/rjbriody/incubator-tinkerpop |
Thu, 05 Mar, 16:17 |
| ok...@apache.org |
incubator-tinkerpop git commit: smart about caching RDDs in SparkGraphComputer and now you don't need to read the RDD off of disk for each MapReduce job. Tests run significantly faster. |
Thu, 05 Mar, 17:25 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Added compile() method to GremlinExecutor. |
Thu, 05 Mar, 19:34 |
| spmalle...@apache.org |
incubator-tinkerpop git commit: Change API for GremlinExecutor to take null for language rather than Optional<String> |
Thu, 05 Mar, 19:49 |
| ok...@apache.org |
[1/3] incubator-tinkerpop git commit: fixed a obnoxious bug around GraphMemory in SparkGraphComputer --- had to use a combination of BroadcastVariables and Accumulators. What a time pit that was. |
Thu, 05 Mar, 22:47 |
| ok...@apache.org |
[2/3] incubator-tinkerpop git commit: GraphComputer Memory no longer returns the mutated value on worker set -- its too difficult to get right in distributed systems. |
Thu, 05 Mar, 22:47 |
| ok...@apache.org |
[3/3] incubator-tinkerpop git commit: Minor tweak to CHANGELOG. |
Thu, 05 Mar, 22:47 |