Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CAMDWjOPcds0aOLTkO4oMsK4Ckaq2-B1do-djRBF0kCrAZkz0Vw@mail.gmail.com>
References: <CAMDWjONSycsOksDEyAF_7Pw_4LbC0dV6tZJKvA0_ZvBapyN35A@mail.gmail.com>
 <CABuYRcqk7YLbJAF8_pLKK3B3=Kz3b0LvSXfUYO7=3e2JGgY8zQ@mail.gmail.com> <CAMDWjOPcds0aOLTkO4oMsK4Ckaq2-B1do-djRBF0kCrAZkz0Vw@mail.gmail.com>
From: Valentin Kulichenko <valentin.kulichenko@gmail.com>
Date: Sat, 11 Mar 2017 18:33:23 +0100
Message-ID: <CABuYRcqYHU4=mf4N0eVQJf198Rq7o7bTa38TEvL9jzz=i8Qhkg@mail.gmail.com>
Subject: Re: IGNITE-13 (ready for review)
To: =?UTF-8?B?0JLQsNC00LjQvCDQntC/0L7Qu9GM0YHQutC40Lk=?= <vaopolskij@gmail.com>
Cc: dev@ignite.apache.org, Vladimir Ozerov <vozerov@gridgain.com>
Content-Type: multipart/alternative; boundary=94eb2c05b790d42042054a77e184
archived-at: Sat, 11 Mar 2017 17:34:11 -0000

--94eb2c05b790d42042054a77e184
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Vadim,

According to these results, I don't think it makes sense to make this
change in the product. I closed the ticket.

-Val

On Fri, Mar 10, 2017 at 12:28 PM, =D0=92=D0=B0=D0=B4=D0=B8=D0=BC =D0=9E=D0=
=BF=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <vaopolskij@gmail.com>
wrote:

> Hi Valentin,
>
> I cant find out why it's better on long string, but worse on a short
> string. May be it needs to use special tools like such as oracle solaris
> studio performance analyzer or other.
>
> I've added links to the benchmark, unit test and results to the ticket an=
d
> switched status to patch available.
>
> Vadim Opolski
>
>
>
> 2017-03-09 13:57 GMT+03:00 Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
>
>> Hi Vadim,
>>
>> Results are a bit confusing. Any idea why it's better on long string, bu=
t
>> worse on a short string? If that's actually the case, there is no any
>> reason to make the change and I would just close the ticket.
>>
>> -Val
>>
>> On Thu, Mar 9, 2017 at 9:20 AM, =D0=92=D0=B0=D0=B4=D0=B8=D0=BC =D0=9E=D0=
=BF=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <vaopolskij@gmail.com>
>> wrote:
>>
>>> Hello everyone!
>>>
>>> Colleagues, take a look please at the results of measuring.
>>>
>>> Can I close this ticket ?
>>>
>>> Should I add JMH benchmark and unit test to Ignite project ?
>>>
>>> Results of measuring
>>> https://github.com/javaller/mybenchmark/blob/master/out.txt
>>>
>>> Benchmark
>>> https://github.com/javaller/mybenchmark/blob/master/src/main
>>> /java/org/sample/ExampleTest.java
>>>
>>> UTest
>>> https://github.com/javaller/mybenchmark/blob/master/src/main
>>> /java/org/sample/BinaryMarshallerSelfTest.java
>>>
>>> *results of measuring*
>>> Benchmark
>>> (message)                                              Mode  Cnt
>>> Score   Error  Units
>>> LatchBenchmark.binaryHeapOutputStreamDirect
>>> TestTestTestTestTestTestTestTestTest  avgt   50  128,036 =C2=B1 4,360  =
ns/op
>>> LatchBenchmark.binaryHeapOutputStreamDirect
>>> TestTestTest                    avgt   50    44,934 =C2=B1 1,463  ns/op
>>> LatchBenchmark.binaryHeapOutputStreamDirect
>>> Test                          avgt   50    21,254 =C2=B1 0,776  ns/op
>>> LatchBenchmark.binaryHeapOutputStreamInDirect
>>> TestTestTestTestTestTestTestTestTest avgt   50    83,262 =C2=B1 2,264  =
ns/op
>>> LatchBenchmark.binaryHeapOutputStreamInDirect
>>> TestTestTest                   avgt   50    58,975 =C2=B1 1,559  ns/op
>>> LatchBenchmark.binaryHeapOutputStreamInDirect
>>> Test                         avgt   50    48,506 =C2=B1 1,116  ns/op
>>>
>>>
>>> Vadim
>>>
>>> 2017-03-06 19:42 GMT+03:00 =D0=92=D0=B0=D0=B4=D0=B8=D0=BC =D0=9E=D0=BF=
=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <vaopolskij@gmail.com>:
>>>
>>>> Hello, everybody!
>>>>
>>>> Valentin, I've corrected benchmark and received the results:
>>>>
>>>> Benchmark
>>>> (message)                                              Mode  Cnt
>>>> Score   Error  Units
>>>> LatchBenchmark.binaryHeapOutputStreamDirect
>>>> TestTestTestTestTestTestTestTestTest  avgt   50  128,036 =C2=B1 4,360 =
 ns/op
>>>> LatchBenchmark.binaryHeapOutputStreamDirect
>>>> TestTestTest                    avgt   50    44,934 =C2=B1 1,463  ns/o=
p
>>>> LatchBenchmark.binaryHeapOutputStreamDirect
>>>> Test                          avgt   50    21,254 =C2=B1 0,776  ns/op
>>>> LatchBenchmark.binaryHeapOutputStreamInDirect
>>>> TestTestTestTestTestTestTestTestTest avgt   50    83,262 =C2=B1 2,264 =
 ns/op
>>>> LatchBenchmark.binaryHeapOutputStreamInDirect
>>>> TestTestTest                   avgt   50    58,975 =C2=B1 1,559  ns/op
>>>> LatchBenchmark.binaryHeapOutputStreamInDirect
>>>> Test                         avgt   50    48,506 =C2=B1 1,116  ns/op
>>>>
>>>> https://github.com/javaller/MyBenchmark/blob/master/out_06_03_17_2.txt
>>>>
>>>> Whats the next step ?
>>>>
>>>>  Do I have to add benchmark to Ignite project ?
>>>>
>>>> Vadim Opolskiy
>>>>
>>>> 2017-03-03 21:11 GMT+03:00 Valentin Kulichenko <
>>>> valentin.kulichenko@gmail.com>:
>>>>
>>>>> Hi Vadim,
>>>>>
>>>>> What do you mean by "copied benchmarks"? What changed singe previous
>>>>> iteration and why results are so different?
>>>>>
>>>>> As for duplicated loop, you don't need it. BinaryOutputStream allows
>>>>> to write a value to a particular position (even before already writte=
n
>>>>> data). So you can reserve 4 bytes for length, remember position, calc=
ulate
>>>>> length while encoding and writing bytes, and then write length.
>>>>>
>>>>> -Val
>>>>>
>>>>> On Fri, Mar 3, 2017 at 12:45 AM, =D0=92=D0=B0=D0=B4=D0=B8=D0=BC =D0=
=9E=D0=BF=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <vaopolskij@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> Valentin,
>>>>>>
>>>>>> What do you think about duplicated cycle in strToBinaryOutputStream =
?
>>>>>>
>>>>>> How to calculate StrLen =D0=B4=D0=BB=D1=8F outBinaryHeap without thi=
s cycle ?
>>>>>>
>>>>>> public class BinaryUtilsNew extends BinaryUtils {
>>>>>>
>>>>>>     public static int getStrLen(String val) {
>>>>>>         int strLen =3D val.length();
>>>>>>         int utfLen =3D 0;
>>>>>>         int c;
>>>>>>
>>>>>>         // Determine length of resulting byte array.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *for (int cnt =3D 0; cnt < strLen; cnt++) {            c =3D val.cha=
rAt(cnt);            if (c >=3D 0x0001 && c <=3D 0x007F)*                ut=
fLen++;
>>>>>>        *     else if (c > 0x07FF)*
>>>>>>                 utfLen +=3D 3;
>>>>>>             else
>>>>>>                 utfLen +=3D 2;
>>>>>>         }
>>>>>>
>>>>>>         return utfLen;
>>>>>>     }
>>>>>>
>>>>>>     public static void strToUtf8BytesDirect(BinaryOutputStream outBi=
naryHeap, String val) {
>>>>>>
>>>>>>         int strLen =3D val.length();
>>>>>>         int c, cnt;
>>>>>>
>>>>>>         int position =3D 0;
>>>>>>
>>>>>>         outBinaryHeap.unsafeEnsure(1 + 4);
>>>>>>
>>>>>> *   outBinaryHeap.unsafeWriteByte(GridBinaryMarshaller.STRING);     =
   outBinaryHeap.unsafeWriteInt(getStrLen(val));*
>>>>>>
>>>>>>
>>>>>>
>>>>>> * for (cnt =3D 0; cnt < strLen; cnt++) {            c =3D val.charAt=
(cnt);*
>>>>>>        *     if (c >=3D 0x0001 && c <=3D 0x007F)*
>>>>>>                 outBinaryHeap.writeByte((byte) c);
>>>>>>          *   else if (c > 0x07FF) {*
>>>>>>                 outBinaryHeap.writeByte((byte)(0xE0 | (c >> 12) & 0x=
0F));
>>>>>>                 outBinaryHeap.writeByte((byte)(0x80 | (c >> 6) & 0x3=
F));
>>>>>>                 outBinaryHeap.writeByte((byte)(0x80 | (c & 0x3F)));
>>>>>>             }
>>>>>>             else {
>>>>>>                 outBinaryHeap.writeByte((byte)(0xC0 | ((c >> 6) & 0x=
1F)));
>>>>>>                 outBinaryHeap.writeByte((byte)(0x80 | (c  & 0x3F)));
>>>>>>             }
>>>>>>         }
>>>>>>     }
>>>>>>
>>>>>>
>>>>>> Vadim
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2017-03-03 2:00 GMT+03:00 Valentin Kulichenko <
>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>
>>>>>>> Vadim,
>>>>>>>
>>>>>>> Looks better now. Can you also try to modify the benchmark so that
>>>>>>> marshaller and writer are created outside of the measured method? I=
.e. the
>>>>>>> benchmark methods should be as simple as this:
>>>>>>>
>>>>>>>     @Benchmark
>>>>>>>     public void binaryHeapOutputStreamDirect() throws Exception {
>>>>>>>         writer.doWriteStringDirect(message);
>>>>>>>     }
>>>>>>>
>>>>>>>     @Benchmark
>>>>>>>     public void binaryHeapOutputStreamInDirect() throws Exception {
>>>>>>>         writer.doWriteString(message);
>>>>>>>     }
>>>>>>>
>>>>>>> In any case, do I understand correctly that it didn't actually make
>>>>>>> any performance difference? If so, I think we can close the ticket.
>>>>>>>
>>>>>>> Vova, can you also take a look and provide your thoughts?
>>>>>>>
>>>>>>> -Val
>>>>>>>
>>>>>>> On Thu, Mar 2, 2017 at 1:27 PM, =D0=92=D0=B0=D0=B4=D0=B8=D0=BC =D0=
=9E=D0=BF=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <
>>>>>>> vaopolskij@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Valentin!
>>>>>>>>
>>>>>>>> I've created:
>>>>>>>>
>>>>>>>> new method strToUtf8BytesDirect in BinaryUtilsNew
>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main
>>>>>>>> /java/org/sample/BinaryUtilsNew.java
>>>>>>>>
>>>>>>>> new method doWriteStringDirect in BinaryWriterExImplNew
>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main
>>>>>>>> /java/org/sample/BinaryWriterExImplNew.java
>>>>>>>>
>>>>>>>> benchmarks for BinaryWriterExImpl doWriteString and
>>>>>>>> BinaryWriterExImplNew  doWriteStringDirect
>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main
>>>>>>>> /java/org/sample/ExampleTest.java
>>>>>>>>
>>>>>>>> This is a result of comparing:
>>>>>>>>
>>>>>>>> Benchmark
>>>>>>>> Mode  Cnt   Score               Error         UnitsExampleTest.bin=
aryHeapOutputStreamDirect
>>>>>>>> avgt   50  1128448,743 =C2=B1 13536,689  ns/opExampleTest.binaryHe=
apOutputStreamInDirect
>>>>>>>> avgt   50  1127270,695 =C2=B1 17309,256  ns/op
>>>>>>>>
>>>>>>>> Vadim
>>>>>>>>
>>>>>>>> 2017-03-02 1:02 GMT+03:00 Valentin Kulichenko <
>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>
>>>>>>>>> Hi Vadim,
>>>>>>>>>
>>>>>>>>> We're getting closer :) I would actually like to see the test for
>>>>>>>>> actual implementation of BinaryWriterExImpl#doWriteString method.
>>>>>>>>> Logic in binaryHeapOutputInDirect() confuses me a bit and I'm not=
 sure
>>>>>>>>> comparison is valid.
>>>>>>>>>
>>>>>>>>> Can you please do the following:
>>>>>>>>>
>>>>>>>>> 1. Create new BinaryUtils#strToUtf8BytesDirect method, copy-paste
>>>>>>>>> the code from existing BinaryUtils#strToUtf8Bytes and modify it s=
o that it
>>>>>>>>> takes BinaryOutputStream as an argument and writes to it directly=
. Do not
>>>>>>>>> create stream inside this method, as it's the same as creating ne=
w array.
>>>>>>>>> 2. Create new BinaryWriterExImpl#doWriteStringDirect, copy-paste
>>>>>>>>> the code from existing BinaryWriterExImpl#doWriteString and
>>>>>>>>> modify it so that it uses BinaryUtils#strToUtf8BytesDirect and
>>>>>>>>> doesn't call out.writeByteArray.
>>>>>>>>> 3. Create benchmark for BinaryWriterExImpl#doWriteString method.
>>>>>>>>> I.e., create an instance of BinaryWriterExImpl and call doWriteSt=
ring() in
>>>>>>>>> benchmark method.
>>>>>>>>> 4. Similarly, create benchmark for BinaryWriterExImpl#doWriteStri
>>>>>>>>> ngDirect.
>>>>>>>>> 5. Compare results.
>>>>>>>>>
>>>>>>>>> This will give us clear picture of how these two approaches
>>>>>>>>> perform. Your current results are actually promising, but I would=
 like to
>>>>>>>>> confirm them.
>>>>>>>>>
>>>>>>>>> -Val
>>>>>>>>>
>>>>>>>>> On Wed, Mar 1, 2017 at 6:17 AM, =D0=92=D0=B0=D0=B4=D0=B8=D0=BC =
=D0=9E=D0=BF=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <
>>>>>>>>> vaopolskij@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Valentin!
>>>>>>>>>>
>>>>>>>>>> Thank you for comments.
>>>>>>>>>>
>>>>>>>>>> There is a new method which writes directly to BinaryOutputStrea=
m
>>>>>>>>>> instead of intermediate array.
>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main
>>>>>>>>>> /java/org/sample/BinaryUtilsNew.java
>>>>>>>>>>
>>>>>>>>>> There is benchmark.
>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main
>>>>>>>>>> /java/org/sample/MyBenchmark.java
>>>>>>>>>>
>>>>>>>>>> Unit test
>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/main
>>>>>>>>>> /java/org/sample/BinaryOutputStreamTest.java
>>>>>>>>>>
>>>>>>>>>> Statistics
>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/out_01_0
>>>>>>>>>> 3_17.txt
>>>>>>>>>>
>>>>>>>>>> Benchmark
>>>>>>>>>>  Mode       Cnt    Score        Error  Units MyBenchmark.binaryH=
eapOutputIn
>>>>>>>>>> Direct            avgt          50  111,337 =C2=B1 0,742  ns/op
>>>>>>>>>> MyBenchmark.binaryHeapOutputStreamDirect   avgt          50
>>>>>>>>>> 23,847 =C2=B1 0,303    ns/op
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Vadim
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2017-02-28 4:29 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Hi Vadim,
>>>>>>>>>>>
>>>>>>>>>>> Looks like you accidentally removed dev list from the thread,
>>>>>>>>>>> adding it back.
>>>>>>>>>>>
>>>>>>>>>>> I think there is still misunderstanding. What I propose is to
>>>>>>>>>>> modify the BinaryUtils#strToUtf8Bytes so that it writes directl=
y to BinaryOutputStream
>>>>>>>>>>> instead of intermediate array. This should decrease memory cons=
umption and
>>>>>>>>>>> can also increase performance as we will avoid 'writeByteArray'
>>>>>>>>>>> step at the end.
>>>>>>>>>>>
>>>>>>>>>>> Does it make sense to you?
>>>>>>>>>>>
>>>>>>>>>>> -Val
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 27, 2017 at 6:55 AM, =D0=92=D0=B0=D0=B4=D0=B8=D0=BC=
 =D0=9E=D0=BF=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <
>>>>>>>>>>> vaopolskij@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi, Valentin!
>>>>>>>>>>>>
>>>>>>>>>>>> What do you think about using the methods of BinaryOutputStrea=
m:
>>>>>>>>>>>>
>>>>>>>>>>>> 1) writeByteArray(byte[] val)
>>>>>>>>>>>> 2) writeCharArray(char[] val)
>>>>>>>>>>>> 3) write (byte[] arr, int off, int len)
>>>>>>>>>>>>
>>>>>>>>>>>> String val =3D "Test";
>>>>>>>>>>>>     out.writeByteArray( val.getBytes(UTF_8));
>>>>>>>>>>>>
>>>>>>>>>>>>  String val =3D "Test";
>>>>>>>>>>>>     out.writeCharArray(str.toCharArray());
>>>>>>>>>>>>
>>>>>>>>>>>> String val =3D "Test"
>>>>>>>>>>>> InputStream stream =3D new ByteArrayInputStream(
>>>>>>>>>>>> exampleString.getBytes(StandartCharsets.UTF_8));
>>>>>>>>>>>> byte[] buffer =3D new byte[1024];
>>>>>>>>>>>> while ((buffer =3D stream.read()) !=3D -1) {
>>>>>>>>>>>> out.writeByteArray(buffer);
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> What else can we use ?
>>>>>>>>>>>>
>>>>>>>>>>>> Vadim
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2017-02-25 2:21 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Vadim,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Which method implements the approach described in the ticket?
>>>>>>>>>>>>> From what I see, all writeToStringX versions are still encodi=
ng into an
>>>>>>>>>>>>> intermediate array and then call out.writeByteArray. What we =
need to test
>>>>>>>>>>>>> is the approach where bytes are written directly into the str=
eam during
>>>>>>>>>>>>> encoding. Encoding algorithm itself should stay the same for =
now, otherwise
>>>>>>>>>>>>> we will not know how to interpret the result.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It looks like there is some misunderstanding here, so please
>>>>>>>>>>>>> let me know anything is still unclear. I will be happy to ans=
wer your
>>>>>>>>>>>>> questions.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Feb 22, 2017 at 7:22 PM, Valentin Kulichenko <
>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Vadim,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks, I will review this week.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Feb 22, 2017 at 2:28 AM, =D0=92=D0=B0=D0=B4=D0=B8=D0=
=BC =D0=9E=D0=BF=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <
>>>>>>>>>>>>>> vaopolskij@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Valentin!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-13
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I created BinaryWriterExImplNew (extended of
>>>>>>>>>>>>>>> BinaryWriterExImpl) and added new methods with changes
>>>>>>>>>>>>>>> described in the ticket
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/mai=
n
>>>>>>>>>>>>>>> /java/org/sample/BinaryWriterExImplNew.java
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I created a benchmark for BinaryWriterExImplNew
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/src/mai=
n
>>>>>>>>>>>>>>> /java/org/sample/ExampleTest.java
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I run benchmark and compared results
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://github.com/javaller/MyBenchmark/blob/master/totalst=
a
>>>>>>>>>>>>>>> t.txt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> # Run complete. Total time: 00:10:24
>>>>>>>>>>>>>>> Benchmark                                    Mode
>>>>>>>>>>>>>>> Cnt        Score       Error  Units
>>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream1          avgt   50
>>>>>>>>>>>>>>> 1114999,207 =C2=B1 16756,776  ns/op
>>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream2          avgt   50
>>>>>>>>>>>>>>> 1118149,320 =C2=B1 17515,961  ns/op
>>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream3          avgt   50
>>>>>>>>>>>>>>> 1113678,657 =C2=B1 17652,314  ns/op
>>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream4          avgt   50
>>>>>>>>>>>>>>> 1112415,051 =C2=B1 18273,874  ns/op
>>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStream5          avgt   50
>>>>>>>>>>>>>>> 1111366,583 =C2=B1 18282,829  ns/op
>>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStreamACSII   avgt   50
>>>>>>>>>>>>>>> 1112079,667 =C2=B1 16659,532  ns/op
>>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStreamUTFCustom  avgt   50
>>>>>>>>>>>>>>> 1114949,759 =C2=B1 16809,669  ns/op
>>>>>>>>>>>>>>> ExampleTest.binaryHeapOutputStreamUTFNIO        avgt   50
>>>>>>>>>>>>>>> 1121462,325 =C2=B1 19836,466  ns/op
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Is it OK? Whats the next step? Do I have to move this
>>>>>>>>>>>>>>> JMH benchmark to the Ignite project ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Vadim Opolski
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2017-02-21 1:06 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Vadim,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm not sure I understand your benchmarks and how they
>>>>>>>>>>>>>>>> verify the optimization discussed here. Basically, here is=
 what needs to be
>>>>>>>>>>>>>>>> done:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. Create a benchmark for BinaryWriterExImpl#doWriteString
>>>>>>>>>>>>>>>> method.
>>>>>>>>>>>>>>>> 2. Run the benchmark with current implementation.
>>>>>>>>>>>>>>>> 3. Make the change described in the ticket.
>>>>>>>>>>>>>>>> 4. Run the benchmark with these changes.
>>>>>>>>>>>>>>>> 5. Compare results.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Makes sense? Let me know if anything is unclear.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Feb 20, 2017 at 8:51 AM, =D0=92=D0=B0=D0=B4=D0=B8=
=D0=BC =D0=9E=D0=BF=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <
>>>>>>>>>>>>>>>> vaopolskij@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello everybody!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-13
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Valentin, I just have finished benchmark (with JMH) -
>>>>>>>>>>>>>>>>> https://github.com/javaller/MyBenchmark.git
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It collect data about time working of serialization.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> For instance - https://github.com/javaller/My
>>>>>>>>>>>>>>>>> Benchmark/blob/master/out200217.txt
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To start it you have to do next:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1) clone it - git colne https://github.com/javal
>>>>>>>>>>>>>>>>> ler/MyBenchmark.git
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2) install it - mvn install
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 3) run benchmarks -  java -Xms1024m -Xmx4096m -jar
>>>>>>>>>>>>>>>>> target\benchmarks.jar
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Vadim Opolski
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2017-02-15 0:52 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I think we misunderstood each other. My understanding of
>>>>>>>>>>>>>>>>>> this optimization is the following.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Currently string serialization is done in two steps (see
>>>>>>>>>>>>>>>>>> BinaryWriterExImpl#doWriteString):
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> strArr =3D BinaryUtils.strToUtf8Bytes(val); // Encode
>>>>>>>>>>>>>>>>>> string into byte array.
>>>>>>>>>>>>>>>>>> out.writeByteArray(strArr);                      // Writ=
e
>>>>>>>>>>>>>>>>>> byte array into stream.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> What this ticket suggests is to write directly into
>>>>>>>>>>>>>>>>>> stream while string is encoded, without intermediate arr=
ay. This both
>>>>>>>>>>>>>>>>>> reduces memory consumption and eliminates array copy ste=
p.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I updated the ticket and added this explanation there.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Vadim, can you create a micro benchmark and check if it
>>>>>>>>>>>>>>>>>> gives any improvement?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sun, Feb 12, 2017 at 10:38 PM, Vladimir Ozerov <
>>>>>>>>>>>>>>>>>> vozerov@gridgain.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> It is hard to say whether it makes sense or not. No
>>>>>>>>>>>>>>>>>>> doubt, it could speed up marshalling process at the cos=
t of 2x memory
>>>>>>>>>>>>>>>>>>> required for strings. From my previous experience with =
marshalling
>>>>>>>>>>>>>>>>>>> micro-optimizations, we will hardly ever notice speedup=
 in distributed
>>>>>>>>>>>>>>>>>>> environment.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> But, there is another sied - it could speedup our
>>>>>>>>>>>>>>>>>>> queries, because we will not have to unmarshal string o=
n every field
>>>>>>>>>>>>>>>>>>> access. So I would try to make this optimization option=
al and then measure
>>>>>>>>>>>>>>>>>>> query performance with classes having lots of strings. =
It could give us
>>>>>>>>>>>>>>>>>>> interesting results.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Feb 13, 2017 at 5:37 AM, Valentin Kulichenko <
>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Can you please take a look and provide your thoughts?
>>>>>>>>>>>>>>>>>>>> Can this be applied to binary marshaller? From what I =
recall, it serializes
>>>>>>>>>>>>>>>>>>>> string a bit differently from optimized marshaller, so=
 I'm not sure.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Fri, Feb 10, 2017 at 5:16 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Thu, Feb 9, 2017 at 11:26 PM, Valentin Kulichenko =
<
>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> > Hi Vadim,
>>>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>>>> > I don't think it makes much sense to invest into
>>>>>>>>>>>>>>>>>>>>> OptimizedMarshaller.
>>>>>>>>>>>>>>>>>>>>> > However, I would check if this optimization is
>>>>>>>>>>>>>>>>>>>>> applicable to
>>>>>>>>>>>>>>>>>>>>> > BinaryMarshaller, and if yes, implement it.
>>>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Val, in this case can you please update the ticket?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>>>> > -Val
>>>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>>>> > On Thu, Feb 9, 2017 at 11:05 PM, =D0=92=D0=B0=D0=B4=
=D0=B8=D0=BC =D0=9E=D0=BF=D0=BE=D0=BB=D1=8C=D1=81=D0=BA=D0=B8=D0=B9 <
>>>>>>>>>>>>>>>>>>>>> vaopolskij@gmail.com>
>>>>>>>>>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>>>> > > Dear sirs!
>>>>>>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>>>>>>> > > I want to resolve issue IGNITE-13 -
>>>>>>>>>>>>>>>>>>>>> > > https://issues.apache.org/jira/browse/IGNITE-13
>>>>>>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>>>>>>> > > Is it actual?
>>>>>>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>>>>>>> > > Vadim Opolski
>>>>>>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

--94eb2c05b790d42042054a77e184--