Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\))
Subject: Re: IGNITE-3244
From: Denis Magda <dmagda@apache.org>
In-Reply-To: <CAKr8Lfwf6EmibPYzfWqMe3ct5tc9CS-VJ0s+Lfd0TcCAtuYigw@mail.gmail.com>
Date: Wed, 8 Feb 2017 09:51:35 -0800
Cc: Valentin Kulichenko <valentin.kulichenko@gmail.com>,
 Vladimir Ozerov <vozerov@gridgain.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <62C391EA-931B-412C-9845-49C20C0CE08C@apache.org>
References: <CAKr8Lfw9sxwEKn-K4xBVre78FQe+Vw6pzjeSpKy_SEQ2N9rCYg@mail.gmail.com>
 <67BB3D30-4B36-410F-BD02-3C0A261E98A3@apache.org>
 <CAKr8Lfy32BftgxpPFWW28P8xh+Y_Mkh88TMosWrS0o4tXC9JAA@mail.gmail.com>
 <D41A945E-3707-4442-8F99-AE1F1506758F@apache.org>
 <CAKr8Lfy6mr03zhK8h=Fj=xX35VYDxaoVgUWCrQGBBywu-M3Msg@mail.gmail.com>
 <C2AD7308-005B-4817-AB30-D4593D91573F@apache.org>
 <CAGt6o8kU+obj6w5iveT8H5iOtOY5snXFHEvPZ7ONpz0OczTbqw@mail.gmail.com>
 <CAKr8LfwtwCfwQRhTZy+O=YYtTMO+29RJ=W85OspkuyGVzODbbg@mail.gmail.com>
 <CAKr8Lfx2NbxNJ0f_cHiJUo3KBRrH98sg6U8MtZtFRYXkO1EPNw@mail.gmail.com>
 <CAKr8Lfwf6EmibPYzfWqMe3ct5tc9CS-VJ0s+Lfd0TcCAtuYigw@mail.gmail.com>
To: dev@ignite.apache.org
archived-at: Wed, 08 Feb 2017 17:51:39 -0000

Hi Aleksey,

>> type IDs correctly attached to class descriptors. The issue comes =
from
>> BinaryUtils#doReadObjectArray() . Presently, unmarshalled array's =
elements
>> stored in Object[], one need to write the following :
>> Object[] arr =3D (Object[])Array.newInstance(compType, len);
>> instead of
>> Object[] arr =3D deserialize ? (Object[])Array.newInstance(compType, =
len)

This is for occasions when an array needs to be returned in NON =
deserialized form meaning that its elements still will be of binary =
type. When =E2=80=9Cdeserialize=E2=80=9D flag is set to =E2=80=9Ctrue=E2=80=
=9D the elements will be fully deserialized. I think that we should =
properly set =E2=80=9Cdeserialize=E2=80=9D or =E2=80=9CkeepBinary=E2=80=9D=
 flag somewhere.

Vovan, could join the conversation and share your thoughts?

>> Moreover, in CacheObjectBinaryProcessorImpl#marshalToBinary() we
>> explicitly marshal objects array into Object[] and then write it =
further. I
>> wonder why we dont just marshal -> unmarshal it ?


This kind of legacy code that was there for a while. I can=E2=80=99t =
recall the reason why we have it but I=E2=80=99m sure there is a sense =
for that.

Val, Vovan, do you remember?

=E2=80=94
Denis

> On Feb 7, 2017, at 5:05 AM, ALEKSEY KUZNETSOV =
<alkuznetsov.sb@gmail.com> wrote:
>=20
> I have a solution on this task, this is pull request into my forked
> repository : https://github.com/voipp/ignite/pull/6
>=20
> =D0=BF=D0=BD, 6 =D1=84=D0=B5=D0=B2=D1=80. 2017 =D0=B3. =D0=B2 16:03, =
ALEKSEY KUZNETSOV <alkuznetsov.sb@gmail.com>:
>=20
>> Moreover, in CacheObjectBinaryProcessorImpl#marshalToBinary() we
>> explicitly marshal objects array into Object[] and then write it =
further. I
>> wonder why we dont just marshal -> unmarshal it ?
>>=20
>> =D0=BF=D0=BD, 6 =D1=84=D0=B5=D0=B2=D1=80. 2017 =D0=B3. =D0=B2 15:58, =
ALEKSEY KUZNETSOV <alkuznetsov.sb@gmail.com>:
>>=20
>> type IDs correctly attached to class descriptors. The issue comes =
from
>> BinaryUtils#doReadObjectArray() . Presently, unmarshalled array's =
elements
>> stored in Object[], one need to write the following :
>> Object[] arr =3D (Object[])Array.newInstance(compType, len);
>> instead of
>> Object[] arr =3D deserialize ? (Object[])Array.newInstance(compType, =
len)
>>=20
>> =D0=BF=D0=BD, 6 =D1=84=D0=B5=D0=B2=D1=80. 2017 =D0=B3. =D0=B2 11:16, =
Vladimir Ozerov <vozerov@gridgain.com>:
>>=20
>> Denis,
>>=20
>> We need to ensure that type IDs are attached for both arrays and
>> collections/maps. May be we have it for arrays already, but I doubt =
it is
>> so for generic containers.
>>=20
>> On Fri, Feb 3, 2017 at 8:59 PM, Denis Magda <dmagda@apache.org> =
wrote:
>>=20
>>> If a value can be serialized to BinaryObject then it will be =
serialized.
>>> This piece of the code works perfectly well as far as I understand.
>>>=20
>>> I would agree with Vladimir that the binary protocol needs to be
>> revisited
>>> or the way we serialize/deserialize arrays. Presently, when an array =
of
>> any
>>> type is serialized with BinaryWriterExImpl.doWriteObjectArray we =
write
>>> that this is the array
>>>=20
>>> out.unsafeWriteByte(GridBinaryMarshaller.OBJ_ARR);
>>> and right below we store the actual type
>>>=20
>>> out.unsafeWriteInt(desc.typeId())
>>>=20
>>> In my understanding, in the layout of binary data we have all the =
data
>>> needed to deserialize an array back properly.
>>>=20
>>> Vovan, is there anything we are missing?
>>>=20
>>> =E2=80=94
>>> Denis
>>>=20
>>>> On Feb 3, 2017, at 7:09 AM, ALEKSEY KUZNETSOV <
>> alkuznetsov.sb@gmail.com>
>>> wrote:
>>>>=20
>>>> There is javax.cache.integration.CacheWriter#write() method which
>> writes
>>>> key and value, wrapped in CacheEntryImpl.
>>>> In case of class arrays, values must be in form of BinaryObject or
>>>> serialized class arrays ?
>>>>=20
>>>> =D1=87=D1=82, 2 =D1=84=D0=B5=D0=B2=D1=80. 2017 =D0=B3. =D0=B2 =
19:57, Denis Magda <dmagda@apache.org>:
>>>>=20
>>>>> Yes, this is exactly the reason why this ticket is created. =
Presently,
>>>>> binary marshaller ignores custom type for arrays. We need to find =
a
>> way
>>> how
>>>>> to handle this.
>>>>>=20
>>>>> =E2=80=94
>>>>> Denis
>>>>>=20
>>>>>> On Feb 2, 2017, at 6:23 AM, ALEKSEY KUZNETSOV <
>>> alkuznetsov.sb@gmail.com>
>>>>> wrote:
>>>>>>=20
>>>>>> I founded that cache.get(i) actually returns an array, containing
>>>>>> TestObject. But somehow the type of returned value is Object[] =
not
>>>>>> TestObject[]
>>>>>>=20
>>>>>> =D1=81=D1=80, 1 =D1=84=D0=B5=D0=B2=D1=80. 2017 =D0=B3. =D0=B2 =
22:26, Denis Magda <dmagda@apache.org>:
>>>>>>=20
>>>>>>> Excellent, please share the way you want to fix the issue with =
the
>>>>>>> community. You might get a valuable feedback before getting down =
to
>>>>> coding.
>>>>>>>=20
>>>>>>> =E2=80=94
>>>>>>> Denis
>>>>>>>=20
>>>>>>>> On Feb 1, 2017, at 1:18 AM, ALEKSEY KUZNETSOV <
>>>>> alkuznetsov.sb@gmail.com>
>>>>>>> wrote:
>>>>>>>>=20
>>>>>>>> will take https://issues.apache.org/jira/browse/IGNITE-3244
>>>>>>>> --
>>>>>>>>=20
>>>>>>>> *Best Regards,*
>>>>>>>>=20
>>>>>>>> *Kuznetsov Aleksey*
>>>>>>>=20
>>>>>>> --
>>>>>>=20
>>>>>> *Best Regards,*
>>>>>>=20
>>>>>> *Kuznetsov Aleksey*
>>>>>=20
>>>>> --
>>>>=20
>>>> *Best Regards,*
>>>>=20
>>>> *Kuznetsov Aleksey*
>>>=20
>>>=20
>>=20
>> --
>>=20
>> *Best Regards,*
>>=20
>> *Kuznetsov Aleksey*
>>=20
>> --
>>=20
>> *Best Regards,*
>>=20
>> *Kuznetsov Aleksey*
>>=20
> --=20
>=20
> *Best Regards,*
>=20
> *Kuznetsov Aleksey*