[ https://issues.apache.org/jira/browse/HARMONY-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893108#action_12893108
]
Tim Ellison commented on HARMONY-6604:
--------------------------------------
I don't think we should try to match the RI in this case, it is way beyond the API requirements,
and is going to be sensitive not only to the Z_*_FLUSH option we use, but also the size of
any internal buffers we are using etc. so it will be difficult at best.
I suggest this is closed as a non-bug difference.
However, you say "this difference cause the 6 junit test case fail in hadoop." - maybe the
test is invalid if it depends upon this level of compression behavior.
> [classlib][archive] Deflater behavior different between ri and harmony
> ----------------------------------------------------------------------
>
> Key: HARMONY-6604
> URL: https://issues.apache.org/jira/browse/HARMONY-6604
> Project: Harmony
> Issue Type: Bug
> Reporter: Li Jing Qin
> Priority: Minor
> Attachments: arrayfile
>
>
> Given the test case below:
> File arrayfile = new File("arrayfile"); [0]
> BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(arrayfile)));
> String input = null;
> byte[] holder = new byte[262144];
> int curser = 0;
> byte[] output = new byte[262144];
> while ((input = reader.readLine()) != null) {
> String[] inputs = input.split(" ");
> for (String in : inputs) {
> String code = in;
> holder[curser] = Byte.decode(code).byteValue();
> curser++;
> }
>
> }
> Deflater deflater = new Deflater(Deflater.DEFAULT_COMPRESSION);
>
> int start = 0;
> int end = curser;
> int offset = 0;
>
> while (start < end) {
> if (deflater.needsInput()) {
> int size = 4092;
> if (start + size >= end) {
> size = end - start;
> }
> deflater.setInput(holder, start, size);
> start += size;
> } else {
> int def = deflater.deflate(output, offset, output.length - offset);
> if (def != 0) {
> System.out.println("def: " + def);
> offset += def;
> }
> }
> }
> deflater.finish();
> int def = deflater.deflate(output, offset, output.length - offset);
> System.out.println("end deflate: " + def);
>
> deflater.end();
> reader.close();
> The RI output is:
> def: 2
> def: 20670
> end deflate: 93
> Harmony is:
> def: 2
> end deflate: 20757
> From the output, it seems that RI thought input bytes enough early than Harmony does.
I guess ri maybe use different flush policy when the there are too many read bytes hold in
the deflater, but fail to test out which policy ri uses.
> The difference seems not so big. We are using Z_NO_FLUSH as the flush parameter, *which
allows deflate to decide how much data to accumulate before producing output, in order to
maximize the compression*. And the result shows that harmony need less bytes than RI. But
this difference cause the 6 junit test case fail in hadoop.
> Moreover, in zlib, there still another flush parameters (Z_PARTIAL_FLUSH, for example)
maybe we need add these.
> Does anyone has any idea about this?
> [0] arrayfile will be attached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
|