pig-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Pig Wiki] Update of "PigTypesBranchIntegration" by AlanGates
Date Fri, 13 Jun 2008 19:22:12 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.

The following page has been changed by AlanGates:
http://wiki.apache.org/pig/PigTypesBranchIntegration

New page:
= Pig Types Branch Integration =

This page tracks the integration of the types branch back into the main branch.  

== Step 1: Merge patches from main into branch ==
A merge from trunk to the branch has not been done since February 27, 2008.  We need to begin
merging patches into the branch.  This will be particularly
challenging in many cases because large sections of the code have been rewritten.  This means
that the patches cannot be merged as is, but rather the functional
changes will need to be determined and the same changes made in the branch.

As new patches are committed to the trunk we need to track them here to see which category
they fall into.

=== Issues that we believe will be fixed in rework and just need tested ===

|| '''JIRA''' || '''Comments''' || '''Done'''||
|| [http://issues.apache.org/jira/browse/PIG-237 PIG-237] || pig allows to overwrite existing
files addressed in the type branch according to Pi need to verify. || No ||
|| [http://issues.apache.org/jira/browse/PIG-202 PIG-202] || comparator function not used
in local mode; is this already fixed? verify || No ||
|| [http://issues.apache.org/jira/browse/PIG-183 PIG-183] || pig could not resolve UDF class;
maybe already fixed; verify || No ||
|| [http://issues.apache.org/jira/browse/PIG-153 PIG-153] || Incorrect results when there
is a dump in between statements; might be fixed || No ||
|| [http://issues.apache.org/jira/browse/PIG-125 PIG-125] || better error handling; might
already have it; need to verify || No ||
|| [http://issues.apache.org/jira/browse/PIG-213 PIG-213] || non-static logger used; || No
||
|| [http://issues.apache.org/jira/browse/PIG-179 PIG-179] || On hadoop 0.16, some jobs using
combiner fail with an NPE || No ||
|| [http://issues.apache.org/jira/browse/PIG-178 PIG-178] || Use of schema on a secondary
output of SPLIT throws IndexOutOfBoundsException || No ||

=== Issues that do not touch rewritten code and should be easy ===

|| '''JIRA''' || '''Comments''' || '''Done'''||
|| [http://issues.apache.org/jira/browse/PIG-149 PIG-149] || fix for doc target can be committed
now || No ||
|| [http://issues.apache.org/jira/browse/PIG-150 PIG-150] || similar to 149; can be committed
now || No ||
|| [http://issues.apache.org/jira/browse/PIG-68 PIG-68] || build.xml improvements; might already
be in; otherwise trivial to fix || No ||
|| [http://issues.apache.org/jira/browse/PIG-124 PIG-124] || running 1 test at a time; might
already be there; trivial otherwise || No ||
|| [http://issues.apache.org/jira/browse/PIG-127 PIG-127] || target descriptions; might already
be in; otherwise a trivial change that can be done now || No ||
|| [http://issues.apache.org/jira/browse/PIG-13 PIG-13] || implement version; trivial change;
can be applied now || No ||
|| [http://issues.apache.org/jira/browse/PIG-101 PIG-101] || this was on 2/27 so it might
already be there; if not it is just test changes that can be applied now || No ||
|| [http://issues.apache.org/jira/browse/PIG-106 PIG-106] || Optimize Pig by replacing String
'+' and StringBuffer with StringBuilder; should be trivial to integrate || No ||
|| [http://issues.apache.org/jira/browse/PIG-122 PIG-122] || remove TokenMgrError and Co from
svn properties in src/org/apache/pig/tools/pigscript/parser; || No ||
|| [http://issues.apache.org/jira/browse/PIG-123 PIG-123] || escapes in the parse; can be
done now || No ||
|| [http://issues.apache.org/jira/browse/PIG-58 PIG-58] || parameter substitution; can be
moved now || No ||
|| [http://issues.apache.org/jira/browse/PIG-222 PIG-222] || param subst tests hardwire perl
location; can be committed now right after (58) || No ||
|| [http://issues.apache.org/jira/browse/PIG-220 PIG-220] || incorrect variable definition
in parameter substitution; can be committed now after (58) || No ||
|| [http://issues.apache.org/jira/browse/PIG-218 PIG-218] || param sub broken with some commands;
can be committed now right after (58) || No ||
|| [http://issues.apache.org/jira/browse/PIG-203 PIG-203] || pig parser hangs on input script
bigger ~1kb; can be committed right now after (58) || No ||
|| [http://issues.apache.org/jira/browse/PIG-219 PIG-219] || running all test cases in both
local amd MR mode; can be merged now || No ||
|| [http://issues.apache.org/jira/browse/PIG-156 PIG-156] || unit tests broken under windows
can be merged now || No ||
|| [http://issues.apache.org/jira/browse/PIG-18 PIG-18] || making Pig work with HOD 0.4; should
be doable now || No ||

=== Issues that do touch rewritten code and will be difficult ===

==== Streaming ====
The code is somewhat isolated to should not be too bad. Unit tests and end-to-end tests are
fairly comprehansive.

|| '''JIRA''' || '''Comments''' || '''Done'''||
|| [http://issues.apache.org/jira/browse/PIG-230 PIG-230] || auto ship broken is presence
of define for another streaming operator || No ||
|| [http://issues.apache.org/jira/browse/PIG-232 PIG-232] || Number of input/output rows in
the logs is invalid with BinaryStorage || No ||
|| [http://issues.apache.org/jira/browse/PIG-231 PIG-231] || validation of files in ship/cache
specs || No ||
|| [http://issues.apache.org/jira/browse/PIG-229 PIG-229] || streaming does not handle errors
on load function || No ||
|| [http://issues.apache.org/jira/browse/PIG-227 PIG-227] || parsing problems for streaming
define || No ||
|| [http://issues.apache.org/jira/browse/PIG-224 PIG-224] || streaming returns always returns
127 || No ||
|| [http://issues.apache.org/jira/browse/PIG-228 PIG-228] || names for secondary streaming
outputs don't match the spec || No ||
|| [http://issues.apache.org/jira/browse/PIG-226 PIG-226] || problem with streaming optimization
|| No ||
|| [http://issues.apache.org/jira/browse/PIG-216 PIG-216] || streaming does not handle commands
with pipes || No ||
|| [http://issues.apache.org/jira/browse/PIG-188 PIG-188] || streaming error || No ||
|| [http://issues.apache.org/jira/browse/PIG-184 PIG-184] || Parsing issue on the stream statement?
|| No ||
|| [http://issues.apache.org/jira/browse/PIG-182 PIG-182] || Broken pipe if excuting the streaming
script via the stream command directory || No ||
|| [http://issues.apache.org/jira/browse/PIG-94 PIG-94] || Streaming implementation || No
||
|| [http://issues.apache.org/jira/browse/PIG-181 PIG-181] || Bug in streaming || No ||
|| [http://issues.apache.org/jira/browse/PIG-174 PIG-174] || Bug in streaming || No ||
|| [http://issues.apache.org/jira/browse/PIG-180 PIG-180] || Bug in streaming || No ||
|| [http://issues.apache.org/jira/browse/PIG-154 PIG-154] || Move DEFINE and STORE parsing
into QueryParser from GruntParser || No ||

==== Config changes ====
|| '''JIRA''' || '''Comments''' || '''Done'''||
|| [http://issues.apache.org/jira/browse/PIG-215 PIG-215] || cleanup after PIG-111 || No ||
|| [http://issues.apache.org/jira/browse/PIG-111 PIG-111] || config changes || No ||
|| [http://issues.apache.org/jira/browse/PIG-236 PIG-236] || command line properties are ignored
needs manual testing; Pradeep should do this one || No ||

==== Custom splits ====
|| '''JIRA''' || '''Comments''' || '''Done'''||
|| [http://issues.apache.org/jira/browse/PIG-55 PIG-55] || || No ||
|| [http://issues.apache.org/jira/browse/PIG-204 PIG-204] || streaming fix after custom splits
|| No ||

==== Others ====
|| '''JIRA''' || '''Comments''' || '''Done'''||
|| [http://issues.apache.org/jira/browse/PIG-85 PIG-85] || making PigStorage work with control
characters || No ||
|| [http://issues.apache.org/jira/browse/PIG-250 PIG-250] || speculative execution is broken;
|| No ||
|| [http://issues.apache.org/jira/browse/PIG-198 PIG-198] || integration with hadoop 0.17;
|| No ||
|| [http://issues.apache.org/jira/browse/PIG-176 PIG-176] || memory management; depends on
PIG-111; || No ||
|| [http://issues.apache.org/jira/browse/PIG-151 PIG-151] || problems with bzip files || No
||
|| [http://issues.apache.org/jira/browse/PIG-207 PIG-207] || New illustrate command does not
work in mapreduce mode. || No ||
|| [http://issues.apache.org/jira/browse/PIG-59 PIG-59] || Illustrate command; I think should
be easy to integrate || No ||
|| [http://issues.apache.org/jira/browse/PIG-114 PIG-114] || reversable functions || No ||
|| [http://issues.apache.org/jira/browse/PIG-167 PIG-167] || memory management || No ||
|| [http://issues.apache.org/jira/browse/PIG-170 PIG-170] || memory management || No ||
|| [http://issues.apache.org/jira/browse/PIG-172 PIG-172] || NullPointerException thrown if
we catch exception with null message || No ||
|| [http://issues.apache.org/jira/browse/PIG-164 PIG-164] || purgin weak references || No
||
|| [http://issues.apache.org/jira/browse/PIG-129 PIG-129] || need to create temp files in
the task's working directory; should be easy to integrate || No ||
|| [http://issues.apache.org/jira/browse/PIG-118 PIG-118] || UNION/CROSS/JOIN operations should
not allow 1 operand; trivial change; || No ||
|| [http://issues.apache.org/jira/browse/PIG-120 PIG-120] || support hadoop map reduce in
loal mode || No ||
|| [http://issues.apache.org/jira/browse/PIG-113 PIG-113] || Make Grunt's explain output more
understandable; || No ||

== Step 2 ==
Merge from branch to trunk.

Mime
View raw message