datafu-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Hayes" <matthew.terence.ha...@gmail.com>
Subject Review Request 20830: DATAFU-44 Migrate Hourglass to Gradle
Date Tue, 29 Apr 2014 05:31:44 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20830/
-----------------------------------------------------------

Review request for DataFu.


Repository: datafu


Description
-------

This change looks big, but only because I moved a lot of files.  The only file that is really
new is the build.gradle file under datafu-hourglass.

I also:
1) Updated NOTICE with those from Hourglass that were missing
2) Fixed all the license headers for the Hourglass Java files using the Gradle tasks


Diffs
-----

  NOTICE c986a58 
  contrib/hourglass/.classpath 6864b5b 
  contrib/hourglass/.gitignore 07c4345 
  contrib/hourglass/.project 6ff2142 
  contrib/hourglass/.settings/org.eclipse.jdt.core.prefs 93e7466 
  contrib/hourglass/LICENSE 57bc88a 
  contrib/hourglass/NOTICE 26bf0e3 
  contrib/hourglass/README.md 31d2361 
  contrib/hourglass/build.xml 9739276 
  contrib/hourglass/changes.md 28c78a0 
  contrib/hourglass/ivy.xml 38d9d79 
  contrib/hourglass/ivysettings.xml af022dd 
  contrib/hourglass/overview.html f976ab9 
  contrib/hourglass/pom-template.xml 4890059 
  contrib/hourglass/src/java/datafu/hourglass/avro/AvroDateRangeMetadata.java f22a4b6 
  contrib/hourglass/src/java/datafu/hourglass/avro/AvroKeyValueWithMetadataOutputFormat.java
4574c69 
  contrib/hourglass/src/java/datafu/hourglass/avro/AvroKeyValueWithMetadataRecordWriter.java
4511d24 
  contrib/hourglass/src/java/datafu/hourglass/avro/AvroKeyWithMetadataOutputFormat.java b37c559

  contrib/hourglass/src/java/datafu/hourglass/avro/AvroKeyWithMetadataRecordWriter.java 0e61d87

  contrib/hourglass/src/java/datafu/hourglass/avro/AvroMultipleInputsKeyInputFormat.java 2686c32

  contrib/hourglass/src/java/datafu/hourglass/avro/AvroMultipleInputsUtil.java b15041a 
  contrib/hourglass/src/java/datafu/hourglass/avro/CombinedAvroKeyInputFormat.java d2b171d

  contrib/hourglass/src/java/datafu/hourglass/avro/package-info.java 54a428f 
  contrib/hourglass/src/java/datafu/hourglass/fs/DatePath.java 0ffba6e 
  contrib/hourglass/src/java/datafu/hourglass/fs/DateRange.java 485dd82 
  contrib/hourglass/src/java/datafu/hourglass/fs/PathUtils.java 4dfd5d1 
  contrib/hourglass/src/java/datafu/hourglass/fs/package-info.java 9d81006 
  contrib/hourglass/src/java/datafu/hourglass/jobs/AbstractJob.java 3b998a9 
  contrib/hourglass/src/java/datafu/hourglass/jobs/AbstractNonIncrementalJob.java 1c09cb4

  contrib/hourglass/src/java/datafu/hourglass/jobs/AbstractPartitionCollapsingIncrementalJob.java
b01eaad 
  contrib/hourglass/src/java/datafu/hourglass/jobs/AbstractPartitionPreservingIncrementalJob.java
74a8bc2 
  contrib/hourglass/src/java/datafu/hourglass/jobs/DateRangeConfigurable.java 5a9211b 
  contrib/hourglass/src/java/datafu/hourglass/jobs/DateRangePlanner.java 651a27f 
  contrib/hourglass/src/java/datafu/hourglass/jobs/ExecutionPlanner.java e075770 
  contrib/hourglass/src/java/datafu/hourglass/jobs/FileCleaner.java 1dfc24c 
  contrib/hourglass/src/java/datafu/hourglass/jobs/IncrementalJob.java 0a25cba 
  contrib/hourglass/src/java/datafu/hourglass/jobs/MaxInputDataExceededException.java f991752

  contrib/hourglass/src/java/datafu/hourglass/jobs/PartitionCollapsingExecutionPlanner.java
35e9294 
  contrib/hourglass/src/java/datafu/hourglass/jobs/PartitionCollapsingIncrementalJob.java
f92b76b 
  contrib/hourglass/src/java/datafu/hourglass/jobs/PartitionPreservingExecutionPlanner.java
a3fc77d 
  contrib/hourglass/src/java/datafu/hourglass/jobs/PartitionPreservingIncrementalJob.java
488fe7a 
  contrib/hourglass/src/java/datafu/hourglass/jobs/ReduceEstimator.java 9f2f068 
  contrib/hourglass/src/java/datafu/hourglass/jobs/Setup.java 96ee07d 
  contrib/hourglass/src/java/datafu/hourglass/jobs/StagedOutputJob.java 0445322 
  contrib/hourglass/src/java/datafu/hourglass/jobs/TimeBasedJob.java f8708c8 
  contrib/hourglass/src/java/datafu/hourglass/jobs/TimePartitioner.java 3a9e917 
  contrib/hourglass/src/java/datafu/hourglass/jobs/package-info.java 010bfd0 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/AvroKeyValueIdentityMapper.java b8571a0

  contrib/hourglass/src/java/datafu/hourglass/mapreduce/CollapsingCombiner.java aa8d64c 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/CollapsingMapper.java 805ed1c 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/CollapsingReducer.java 5424754 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/DelegatingCombiner.java 03d36f3 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/DelegatingMapper.java d170cf5 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/DelegatingReducer.java 4ddf918 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/DistributedCacheHelper.java bdb626a

  contrib/hourglass/src/java/datafu/hourglass/mapreduce/ObjectMapper.java ea1ca2e 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/ObjectProcessor.java f33c790 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/ObjectReducer.java 9340019 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/Parameters.java 65d3fd9 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/PartitioningCombiner.java 53d6a90

  contrib/hourglass/src/java/datafu/hourglass/mapreduce/PartitioningMapper.java 43b7130 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/PartitioningReducer.java d6726ac 
  contrib/hourglass/src/java/datafu/hourglass/mapreduce/package-info.java 5ee265b 
  contrib/hourglass/src/java/datafu/hourglass/model/Accumulator.java 4f7ac7e 
  contrib/hourglass/src/java/datafu/hourglass/model/KeyValueCollector.java 215dd8d 
  contrib/hourglass/src/java/datafu/hourglass/model/Mapper.java 7640ae5 
  contrib/hourglass/src/java/datafu/hourglass/model/Merger.java 0c27865 
  contrib/hourglass/src/java/datafu/hourglass/model/package-info.java 984c1fb 
  contrib/hourglass/src/java/datafu/hourglass/schemas/PartitionCollapsingSchemas.java 8d8246c

  contrib/hourglass/src/java/datafu/hourglass/schemas/PartitionPreservingSchemas.java d4f3bd2

  contrib/hourglass/src/java/datafu/hourglass/schemas/TaskSchemas.java 829755e 
  contrib/hourglass/src/java/datafu/hourglass/schemas/package-info.java 3a6d2b8 
  contrib/hourglass/test.sh f017e58 
  contrib/hourglass/test/java/datafu/hourglass/demo/CountById.java 166a5db 
  contrib/hourglass/test/java/datafu/hourglass/demo/EstimateCardinality.java eeaf8cc 
  contrib/hourglass/test/java/datafu/hourglass/demo/Examples.java 8df7539 
  contrib/hourglass/test/java/datafu/hourglass/demo/GenerateIds.java 2b794a8 
  contrib/hourglass/test/java/datafu/hourglass/demo/Main.java e406656 
  contrib/hourglass/test/java/datafu/hourglass/demo/NamedTool.java bf7d669 
  contrib/hourglass/test/java/datafu/hourglass/test/PartitionCollapsingExecutionPlannerTests.java
5249bc2 
  contrib/hourglass/test/java/datafu/hourglass/test/PartitionCollapsingJoinTest.java 4e8b4c7

  contrib/hourglass/test/java/datafu/hourglass/test/PartitionCollapsingTests.java 5950195

  contrib/hourglass/test/java/datafu/hourglass/test/PartitionPreservingCollapsingIntegrationTests.java
79835b4 
  contrib/hourglass/test/java/datafu/hourglass/test/PartitionPreservingJoinTests.java e7fa573

  contrib/hourglass/test/java/datafu/hourglass/test/PartitionPreservingTests.java 481c98d

  contrib/hourglass/test/java/datafu/hourglass/test/Schemas.java 2f3c67d 
  contrib/hourglass/test/java/datafu/hourglass/test/TestAvroJob.java 3f018f2 
  contrib/hourglass/test/java/datafu/hourglass/test/TestBase.java c63a6a3 
  contrib/hourglass/test/java/datafu/hourglass/test/jobs/ImpressionClickPartitionCollapsingJob.java
87ef074 
  contrib/hourglass/test/java/datafu/hourglass/test/jobs/ImpressionClickPartitionPreservingJob.java
d314b71 
  contrib/hourglass/test/java/datafu/hourglass/test/jobs/SimpleAvroJob.java 368ceae 
  contrib/hourglass/test/java/datafu/hourglass/test/jobs/SimplePartitionCollapsingCountJob.java
43d71fa 
  contrib/hourglass/test/java/datafu/hourglass/test/jobs/SimplePartitionCollapsingCountJob2.java
6b8a711 
  contrib/hourglass/test/java/datafu/hourglass/test/jobs/SimplePartitionPreservingCountJob.java
31e3558 
  contrib/hourglass/test/java/datafu/hourglass/test/jobs/counting/CountAccumulator.java ee21f94

  contrib/hourglass/test/java/datafu/hourglass/test/jobs/counting/CountWriter.java d781365

  contrib/hourglass/test/java/datafu/hourglass/test/jobs/counting/PartitionCollapsingIncrementalCountJob.java
a88b5f6 
  contrib/hourglass/test/java/datafu/hourglass/test/jobs/counting/PartitionPreservingIncrementalCountJob.java
73215d1 
  contrib/hourglass/test/java/datafu/hourglass/test/util/DailyTrackingWriter.java 34c05be

  contrib/hourglass/test/java/datafu/hourglass/test/util/TimestampDataWriter.java c9ea8f9

  contrib/hourglass/test_in_background.sh a89b531 
  datafu-hourglass/.gitignore PRE-CREATION 
  datafu-hourglass/README.md PRE-CREATION 
  datafu-hourglass/build.gradle PRE-CREATION 
  datafu-hourglass/changes.md PRE-CREATION 
  datafu-hourglass/overview.html PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroDateRangeMetadata.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroKeyValueWithMetadataOutputFormat.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroKeyValueWithMetadataRecordWriter.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroKeyWithMetadataOutputFormat.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroKeyWithMetadataRecordWriter.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroMultipleInputsKeyInputFormat.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/avro/AvroMultipleInputsUtil.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/avro/CombinedAvroKeyInputFormat.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/avro/package-info.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/fs/DatePath.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/fs/DateRange.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/fs/PathUtils.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/fs/package-info.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/AbstractJob.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/AbstractNonIncrementalJob.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/jobs/AbstractPartitionCollapsingIncrementalJob.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/AbstractPartitionPreservingIncrementalJob.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/DateRangeConfigurable.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/jobs/DateRangePlanner.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/jobs/ExecutionPlanner.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/jobs/FileCleaner.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/IncrementalJob.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/MaxInputDataExceededException.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/PartitionCollapsingExecutionPlanner.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/PartitionCollapsingIncrementalJob.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/PartitionPreservingExecutionPlanner.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/PartitionPreservingIncrementalJob.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/ReduceEstimator.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/Setup.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/StagedOutputJob.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/TimeBasedJob.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/TimePartitioner.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/jobs/package-info.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/AvroKeyValueIdentityMapper.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/CollapsingCombiner.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/CollapsingMapper.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/CollapsingReducer.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/DelegatingCombiner.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/DelegatingMapper.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/DelegatingReducer.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/DistributedCacheHelper.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/ObjectMapper.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/ObjectProcessor.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/ObjectReducer.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/Parameters.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/PartitioningCombiner.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/PartitioningMapper.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/PartitioningReducer.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/mapreduce/package-info.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/model/Accumulator.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/model/KeyValueCollector.java PRE-CREATION

  datafu-hourglass/src/main/java/datafu/hourglass/model/Mapper.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/model/Merger.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/model/package-info.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/schemas/PartitionCollapsingSchemas.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/schemas/PartitionPreservingSchemas.java
PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/schemas/TaskSchemas.java PRE-CREATION 
  datafu-hourglass/src/main/java/datafu/hourglass/schemas/package-info.java PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/demo/CountById.java PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/demo/EstimateCardinality.java PRE-CREATION

  datafu-hourglass/src/test/java/datafu/hourglass/demo/Examples.java PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/demo/GenerateIds.java PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/demo/Main.java PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/demo/NamedTool.java PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/PartitionCollapsingExecutionPlannerTests.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/PartitionCollapsingJoinTest.java PRE-CREATION

  datafu-hourglass/src/test/java/datafu/hourglass/test/PartitionCollapsingTests.java PRE-CREATION

  datafu-hourglass/src/test/java/datafu/hourglass/test/PartitionPreservingCollapsingIntegrationTests.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/PartitionPreservingJoinTests.java PRE-CREATION

  datafu-hourglass/src/test/java/datafu/hourglass/test/PartitionPreservingTests.java PRE-CREATION

  datafu-hourglass/src/test/java/datafu/hourglass/test/Schemas.java PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/TestAvroJob.java PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/TestBase.java PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/ImpressionClickPartitionCollapsingJob.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/ImpressionClickPartitionPreservingJob.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/SimpleAvroJob.java PRE-CREATION

  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/SimplePartitionCollapsingCountJob.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/SimplePartitionCollapsingCountJob2.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/SimplePartitionPreservingCountJob.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/counting/CountAccumulator.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/counting/CountWriter.java PRE-CREATION

  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/counting/PartitionCollapsingIncrementalCountJob.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/jobs/counting/PartitionPreservingIncrementalCountJob.java
PRE-CREATION 
  datafu-hourglass/src/test/java/datafu/hourglass/test/util/DailyTrackingWriter.java PRE-CREATION

  datafu-hourglass/src/test/java/datafu/hourglass/test/util/TimestampDataWriter.java PRE-CREATION

  gradle/dependency-versions.gradle 29d3e36 
  settings.gradle 62b1899 

Diff: https://reviews.apache.org/r/20830/diff/


Testing
-------

Built the jar:

gradle jar

Ran the unit tests:

gradle test


Thanks,

Matthew Hayes


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message