fleece-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Romain Manni-Bucau <rmannibu...@gmail.com>
Subject Re: New test cases and JsonStreamParser issues
Date Sun, 06 Jul 2014 17:03:24 GMT
you can create a checkstyle.xml copying/pasting the checkstyle config
of the pom.xml


Romain Manni-Bucau
Twitter: @rmannibucau
Blog: http://rmannibucau.wordpress.com/
LinkedIn: http://fr.linkedin.com/in/rmannibucau
Github: https://github.com/rmannibucau


2014-07-06 18:50 GMT+02:00 Hendrik Dev <hendrikdev22@gmail.com>:
> new testfiles added (FLEECE-4), i created new ones for those who
> potentially had IP problems. The new files contains only random data.
>
> Regarding code formatting: do you have a eclipse code style formatter
> export file (matching checkstyle) which i can import?
>
> Thanks
> Hendrik
>
> On Sat, Jul 5, 2014 at 11:36 AM, Romain Manni-Bucau
> <rmannibucau@gmail.com> wrote:
>> Hi
>>
>> Awesome,
>>
>> can you please:
>>
>> 1) fix the code style (add {} etc, all is configured in fleece pom now
>> to use checkstyle)
>> 2) check you have permissions to use jsons you added (some comes from
>> a well known project right?)
>> 3) create a  jira and attach you patch to let us applying it?
>>
>> Thank you a lot!
>>
>>
>>
>>
>> Romain Manni-Bucau
>> Twitter: @rmannibucau
>> Blog: http://rmannibucau.wordpress.com/
>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>> Github: https://github.com/rmannibucau
>>
>>
>> 2014-07-05 9:53 GMT+02:00 Hendrik Dev <hendrikdev22@gmail.com>:
>>> Hi,
>>>
>>> i have enhanced the set of test json files [1] (and related test
>>> cases) to make sure that the json parser works correctly and fails
>>> correctly if json is invalid.
>>>
>>> The current JsonStreamParser.java class does not pass all tests. So i
>>> started to fix this but JsonStreamParser was hard to understand for me
>>> and i discovered a main problem (see below) which i could not fix and
>>> so i wrote
>>> a new implementation (JsonByteBufferStreamParser and
>>> JsonCharBufferStreamParser [2]). These implementations did pass all
>>> new (and old) tests (some older tests had issues because of IMHO wrong
>>> tests, see [3]).
>>>
>>> The main problem with JsonStreamParser was that it fails if the last
>>> character in char[] loadedChars, before a direct overflow, is a ending
>>> doublequote. Than the overflow happens and something goes wrong with
>>> preserving the previously readed value.
>>> So if the buffer size (=org.apache.fleece.default-char-buffer) and the
>>> json input have the right "bad" correlation then parsing would fail.
>>> There are now testcases for this.
>>>
>>> I did some JMH based benchmarking for the the new parser
>>> implementations [4] (based on the ideas from [5]). They show more or
>>> less equals figures [6] compared to the original JsonStreamParser.
>>>
>>> Please let me know what you think about that.
>>> Thanks
>>> Hendrik
>>>
>>>
>>> [1]
>>> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/json
>>> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core
>>>
>>> [2]
>>> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonByteBufferStreamParser.java
>>> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonCharBufferStreamParser.java
>>>
>>> [3]
>>> https://github.com/salyh/fleece_tmp/commit/8377bdf88f86e0ca07b344a87a48241788e4fd7b#commitcomment-6902656
>>>
>>> [4]
>>> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/bench
>>> https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core/jmh/benchmark
>>>
>>> [5]
>>> https://github.com/RichardHightower/json-parsers-benchmark
>>>
>>> [6]
>>> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/avg_benchmark_jmh_result_f2_t16_w3_i5.txt
>>> https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/thr_benchmark_jmh_result_f2_t16_w3_i5.txt
>>>
>>> --
>>> Hendrik Saly (salyh, hendrikdev22)
>>> @hendrikdev22
>>> PGP: 0x22D7F6EC
>
>
>
> --
> Hendrik Saly (salyh, hendrikdev22)
> @hendrikdev22
> PGP: 0x22D7F6EC

Mime
View raw message