-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18139/
-----------------------------------------------------------
(Updated March 3, 2014, 9:09 p.m.)
Review request for DataFu.
Changes
-------
Added v4 patch after merging with master
Repository: datafu
Description
-------
This migrates the build system from Ant to Gradle. The JAR that is generated should be exactly
the same as that generated by Ant.
I've moved all the Pig UDF source code to datafu-pig. So it may look like a lot of changes,
but these are just file moves.
This should give us a good foundation to start adding new projects, like hive, crunch, etc.
Here are the things you want to pay attention to:
1) gradlew is a wrapper script that runs gradle. I just copied this from Samza.
2) in the root directory, build.gradle, settings.gradle, and gradle.properties provide basic
config and setup
3) all code under src and test was moved to datafu-pig
4) the gradle directory has some common configuration
5) in the datafu-pig directory, the build.gradle has the equivalent of the original build.xml
I've left the old build.xml file for reference, but I want to remove this and any other remnants
of the old system.
Diffs (updated)
-----
.gitignore 81e9ae8
.project 022f4a7
HEADER PRE-CREATION
README.md 8b3726c
build-plugin/.gitignore PRE-CREATION
build-plugin/build.gradle PRE-CREATION
build-plugin/src/main/java/org/adrianwalker/multilinestring/EcjMultilineProcessor.java PRE-CREATION
build-plugin/src/main/java/org/adrianwalker/multilinestring/JavacMultilineProcessor.java
PRE-CREATION
build-plugin/src/main/java/org/adrianwalker/multilinestring/Multiline.java PRE-CREATION
build-plugin/src/main/java/org/adrianwalker/multilinestring/MultilineProcessor.java PRE-CREATION
build-plugin/src/main/resources/META-INF/services/javax.annotation.processing.Processor
PRE-CREATION
build.gradle PRE-CREATION
datafu-pig/.gitignore PRE-CREATION
datafu-pig/build.gradle PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/AppendToBag.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/BagConcat.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/BagGroup.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/BagLeftOuterJoin.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/BagSplit.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/CountEach.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/DistinctBy.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/EmptyBagToNull.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/EmptyBagToNullFields.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/Enumerate.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/FirstTupleFromBag.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/NullToEmptyBag.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/PrependToBag.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/ReverseEnumerate.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/UnorderedPairs.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/bags/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/geo/HaversineDistInMiles.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/geo/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/hash/MD5.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/hash/SHA.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/hash/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/linkanalysis/PageRank.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/linkanalysis/PageRankImpl.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/linkanalysis/ProgressIndicator.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/linkanalysis/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/random/RandInt.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/random/RandomUUID.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/random/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/Reservoir.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/ReservoirSample.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/SampleByKey.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/ScoredTuple.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/SimpleRandomSample.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/SimpleRandomSampleWithReplacementElect.java
PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/SimpleRandomSampleWithReplacementVote.java
PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/WeightedReservoirSample.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/WeightedSample.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sampling/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sessions/SessionCount.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sessions/Sessionize.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sessions/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sets/SetDifference.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sets/SetIntersect.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sets/SetOperationsBase.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sets/SetUnion.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/sets/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/DoubleVAR.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/FloatVAR.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/HyperLogLogPlusPlus.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/IntVAR.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/LongVAR.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/MarkovPairs.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/Median.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/Quantile.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/QuantileUtil.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/StreamingMedian.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/StreamingQuantile.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/VAR.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/WilsonBinConf.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/entropy/ChaoShenEntropyEstimator.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/entropy/CondEntropy.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/entropy/EmpiricalCountEntropy.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/entropy/EmpiricalEntropyEstimator.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/entropy/Entropy.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/entropy/EntropyEstimator.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/entropy/EntropyUtil.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/stats/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/text/opennlp/CachedFile.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/text/opennlp/POSTag.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/text/opennlp/SentenceDetect.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/text/opennlp/TokenizeME.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/text/opennlp/TokenizeSimple.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/text/opennlp/TokenizeWhitespace.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/urls/UserAgentClassify.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/urls/package-info.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/AliasableEvalFunc.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/Assert.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/AssertUDF.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/BoolToInt.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/Coalesce.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/ContextualEvalFunc.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/DataFuException.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/FieldNotFound.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/In.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/InUDF.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/IntToBool.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/SimpleEvalFunc.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/TransposeTupleToBag.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/util/package-info.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/PigTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/bags/BagTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/geo/GeoTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/hash/HashTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/linkanalysis/PageRankImplTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/linkanalysis/PageRankTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/random/NumberTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/random/UUIDTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/sampling/SamplingTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/sampling/SimpleRandomSampleTest.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/sampling/SimpleRandomSampleTestOld.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/sampling/SimpleRandomSampleWithReplacementTest.java
PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/sampling/WeightedReservoirSamplingTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/sessions/SessionTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/sets/SetTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/EstimationTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/MarkovPairTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/QuantileTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/VARTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/WilsonBinConfTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/entropy/AbstractEntropyTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/entropy/ChaoShenEntropyTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/entropy/CondEntropyTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/entropy/EmpiricalCountEntropyTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/stats/entropy/EntropyTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/text/NLPTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/urls/UserAgentTest.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/util/AliasEvalFuncTest.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/util/AssertTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/util/CoalesceTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/util/InTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/util/IntBoolConversionPigTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/util/TransposeTest.java PRE-CREATION
gradle.properties PRE-CREATION
gradle/buildscript.gradle PRE-CREATION
gradle/dependency-versions.gradle PRE-CREATION
gradle/wrapper/gradle-wrapper.jar PRE-CREATION
gradle/wrapper/gradle-wrapper.properties PRE-CREATION
gradlew PRE-CREATION
plugin/java/org/adrianwalker/multilinestring/EcjMultilineProcessor.java 7dbfe9a
plugin/java/org/adrianwalker/multilinestring/JavacMultilineProcessor.java 39aa24e
plugin/java/org/adrianwalker/multilinestring/Multiline.java c29d3ef
plugin/java/org/adrianwalker/multilinestring/MultilineProcessor.java 9abdba5
settings.gradle PRE-CREATION
src/java/datafu/pig/bags/AppendToBag.java 55c9e76
src/java/datafu/pig/bags/BagConcat.java 290f44b
src/java/datafu/pig/bags/BagGroup.java 7b6c2e2
src/java/datafu/pig/bags/BagLeftOuterJoin.java ba6bc11
src/java/datafu/pig/bags/BagSplit.java 74d39d0
src/java/datafu/pig/bags/CountEach.java cfb0152
src/java/datafu/pig/bags/DistinctBy.java a79e4de
src/java/datafu/pig/bags/EmptyBagToNull.java 5524ad5
src/java/datafu/pig/bags/EmptyBagToNullFields.java 6933c28
src/java/datafu/pig/bags/Enumerate.java 8a0d072
src/java/datafu/pig/bags/FirstTupleFromBag.java 1f24984
src/java/datafu/pig/bags/NullToEmptyBag.java 09fffb3
src/java/datafu/pig/bags/PrependToBag.java 9292871
src/java/datafu/pig/bags/ReverseEnumerate.java c86ffcf
src/java/datafu/pig/bags/UnorderedPairs.java a1d149e
src/java/datafu/pig/bags/package-info.java 214b837
src/java/datafu/pig/geo/HaversineDistInMiles.java d1e3988
src/java/datafu/pig/geo/package-info.java 12b27b0
src/java/datafu/pig/hash/MD5.java b7b51da
src/java/datafu/pig/hash/SHA.java ff859e5
src/java/datafu/pig/hash/package-info.java 320a029
src/java/datafu/pig/linkanalysis/PageRank.java 80ff567
src/java/datafu/pig/linkanalysis/PageRankImpl.java 5d0b932
src/java/datafu/pig/linkanalysis/ProgressIndicator.java 3dc54c5
src/java/datafu/pig/linkanalysis/package-info.java 2c6c078
src/java/datafu/pig/random/RandInt.java de89c4a
src/java/datafu/pig/random/RandomUUID.java d63f4cf
src/java/datafu/pig/random/package-info.java 8c7750d
src/java/datafu/pig/sampling/Reservoir.java 09b4c95
src/java/datafu/pig/sampling/ReservoirSample.java 48deaad
src/java/datafu/pig/sampling/SampleByKey.java 90ea576
src/java/datafu/pig/sampling/ScoredTuple.java c793584
src/java/datafu/pig/sampling/SimpleRandomSample.java 8e8debf
src/java/datafu/pig/sampling/SimpleRandomSampleWithReplacementElect.java a59816a
src/java/datafu/pig/sampling/SimpleRandomSampleWithReplacementVote.java 598e58c
src/java/datafu/pig/sampling/WeightedReservoirSample.java 92af6a3
src/java/datafu/pig/sampling/WeightedSample.java a77164d
src/java/datafu/pig/sampling/package-info.java 0c7d8bd
src/java/datafu/pig/sessions/SessionCount.java fe68888
src/java/datafu/pig/sessions/Sessionize.java 52d159b
src/java/datafu/pig/sessions/package-info.java 686fbf0
src/java/datafu/pig/sets/SetDifference.java b6469e9
src/java/datafu/pig/sets/SetIntersect.java 0ba586e
src/java/datafu/pig/sets/SetOperationsBase.java c9997f8
src/java/datafu/pig/sets/SetUnion.java 49fd99c
src/java/datafu/pig/sets/package-info.java 2c5b087
src/java/datafu/pig/stats/DoubleVAR.java ab8eed2
src/java/datafu/pig/stats/FloatVAR.java 8a697e0
src/java/datafu/pig/stats/HyperLogLogPlusPlus.java 0ebd94f
src/java/datafu/pig/stats/IntVAR.java 0f34d8e
src/java/datafu/pig/stats/LongVAR.java 561f15a
src/java/datafu/pig/stats/MarkovPairs.java d3d7632
src/java/datafu/pig/stats/Median.java e33a84e
src/java/datafu/pig/stats/Quantile.java 6fd42d3
src/java/datafu/pig/stats/QuantileUtil.java c6fd36a
src/java/datafu/pig/stats/StreamingMedian.java c4c3be4
src/java/datafu/pig/stats/StreamingQuantile.java e4a65b4
src/java/datafu/pig/stats/VAR.java 6f22f25
src/java/datafu/pig/stats/WilsonBinConf.java 1448611
src/java/datafu/pig/stats/entropy/ChaoShenEntropyEstimator.java bfb7398
src/java/datafu/pig/stats/entropy/CondEntropy.java 26b743e
src/java/datafu/pig/stats/entropy/EmpiricalCountEntropy.java 388b80f
src/java/datafu/pig/stats/entropy/EmpiricalEntropyEstimator.java 3f0acab
src/java/datafu/pig/stats/entropy/Entropy.java 9dfff1a
src/java/datafu/pig/stats/entropy/EntropyEstimator.java 336ef86
src/java/datafu/pig/stats/entropy/EntropyUtil.java 685f3e2
src/java/datafu/pig/stats/package-info.java 4c100e7
src/java/datafu/pig/text/opennlp/CachedFile.java 5832c81
src/java/datafu/pig/text/opennlp/POSTag.java fb17c63
src/java/datafu/pig/text/opennlp/SentenceDetect.java 50537fd
src/java/datafu/pig/text/opennlp/TokenizeME.java f1f4257
src/java/datafu/pig/text/opennlp/TokenizeSimple.java cea48b4
src/java/datafu/pig/text/opennlp/TokenizeWhitespace.java 8efafb0
src/java/datafu/pig/urls/UserAgentClassify.java 4040fea
src/java/datafu/pig/urls/package-info.java 332beda
src/java/datafu/pig/util/AliasableEvalFunc.java ee2c3f3
src/java/datafu/pig/util/Assert.java a258915
src/java/datafu/pig/util/AssertUDF.java 16f9247
src/java/datafu/pig/util/BoolToInt.java 5ec8d3a
src/java/datafu/pig/util/Coalesce.java f8e25f4
src/java/datafu/pig/util/ContextualEvalFunc.java c534b77
src/java/datafu/pig/util/DataFuException.java 0066aa8
src/java/datafu/pig/util/FieldNotFound.java d624007
src/java/datafu/pig/util/In.java 0667914
src/java/datafu/pig/util/InUDF.java 5057285
src/java/datafu/pig/util/IntToBool.java d00e297
src/java/datafu/pig/util/SimpleEvalFunc.java 2d262b4
src/java/datafu/pig/util/TransposeTupleToBag.java f8a39df
src/java/datafu/pig/util/package-info.java cce46a0
test.sh f017e58
test/pig/datafu/test/pig/PigTests.java 9b4eddd
test/pig/datafu/test/pig/bags/BagTests.java 80bb0cc
test/pig/datafu/test/pig/geo/GeoTests.java 39c7b61
test/pig/datafu/test/pig/hash/HashTests.java ba19344
test/pig/datafu/test/pig/linkanalysis/PageRankImplTests.java 01d540f
test/pig/datafu/test/pig/linkanalysis/PageRankTests.java 371d2cc
test/pig/datafu/test/pig/random/NumberTests.java 867c27c
test/pig/datafu/test/pig/random/UUIDTests.java e199760
test/pig/datafu/test/pig/sampling/SamplingTests.java 9209133
test/pig/datafu/test/pig/sampling/SimpleRandomSampleTest.java bd34881
test/pig/datafu/test/pig/sampling/SimpleRandomSampleTestOld.java 15e1fd6
test/pig/datafu/test/pig/sampling/SimpleRandomSampleWithReplacementTest.java 338b3e7
test/pig/datafu/test/pig/sampling/WeightedReservoirSamplingTests.java 9bab8a0
test/pig/datafu/test/pig/sessions/SessionTests.java 084f970
test/pig/datafu/test/pig/sets/SetTests.java de6aa51
test/pig/datafu/test/pig/stats/EstimationTests.java 258f320
test/pig/datafu/test/pig/stats/MarkovPairTests.java b2f618e
test/pig/datafu/test/pig/stats/QuantileTests.java 1814016
test/pig/datafu/test/pig/stats/VARTests.java 776ed9e
test/pig/datafu/test/pig/stats/WilsonBinConfTests.java 34e0a48
test/pig/datafu/test/pig/stats/entropy/AbstractEntropyTests.java 6e512d4
test/pig/datafu/test/pig/stats/entropy/ChaoShenEntropyTests.java 0030bf2
test/pig/datafu/test/pig/stats/entropy/CondEntropyTests.java c6d62df
test/pig/datafu/test/pig/stats/entropy/EmpiricalCountEntropyTests.java 52fdcce
test/pig/datafu/test/pig/stats/entropy/EntropyTests.java e1611a7
test/pig/datafu/test/pig/text/NLPTests.java 372b17d
test/pig/datafu/test/pig/urls/UserAgentTest.java 82515ad
test/pig/datafu/test/pig/util/AliasEvalFuncTest.java 3b23d35
test/pig/datafu/test/pig/util/AssertTests.java bf558a7
test/pig/datafu/test/pig/util/CoalesceTests.java 0dc0f4b
test/pig/datafu/test/pig/util/InTests.java a21a0b2
test/pig/datafu/test/pig/util/IntBoolConversionPigTests.java 4017867
test/pig/datafu/test/pig/util/TransposeTest.java f131cd0
Diff: https://reviews.apache.org/r/18139/diff/
Testing
-------
You can generate the datafu-pig JAR with:
./gradlew assemble
All the tests should pass:
./gradlew test
You can generate the eclipse files with:
./gradlew eclipse
The tests should pass in eclipse too.
Thanks,
Matthew Hayes
|