ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: By bytes access to binary format
Date Thu, 29 Jun 2017 21:11:19 GMT
Hi Vlad,

I am not quite sure I understand the problem. Can you show how the API you
propose would look like? Remember that "field" method can return anything
from primitive, String or byte array, to another BinaryObject. And returned
BinaryObject can have references outside of itself, so it cannot be
serialized easily without full rebuild. .

On Thu, Jun 29, 2017 at 10:16 AM, Vladislav Pyatkov <vpyatkov@gridgain.com>
wrote:

> Val,
>
> I proposal, access as a stream to binary object, because we have doubled
> copy on touch a field (first at copy from cache and second - on getting a
> field).
>
> For the stream in/out to cache I will be used IGFS.
> Main idea to avoid GC pressure when make a massive read from key-value
> storage.
>
> On Wed, Jun 28, 2017 at 9:36 PM, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
>
> > Vladislav,
> >
> > Are you suggesting to stream directly from cache. or from a binary object
> > that is already copied from cache?
> >
> > -Val
> >
> > On Wed, Jun 28, 2017 at 2:52 AM, Vladislav Pyatkov <
> vpyatkov@gridgain.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Recently, from one of Ignite user, I listened interest idea.
> > > What if I want to pass some date to java stream from cache.
> > >
> > > With binary I do it like this:
> > >
> > > BinaryObject get = (BinaryObject) cache.get(key);
> > > byte[] dataFromCache = get.<byte[]>field("data");
> > > System.out.write(dataFromCache, 0, dataFromCache.length);
> > >
> > > But in this case we got garbage a lot, due to each time new bytes array
> > is
> > > creating.
> > >
> > > This will lead to many GC events in case we load a some of million
> > entries.
> > > Could we offer additional API for working with java stream:
> > >
> > > BinaryObject.writeBytesToBuf("data", ByteBuffer.allocate(1024));
> > >
> > > or with buffer
> > >
> > > BinaryObject.writeBytesToBuf("data", new byte[1000], 100);
> > >
> > > I already created a Jira ticket.
> > > https://issues.apache.org/jira/browse/IGNITE-5602
> > >
> > > --
> > > Vladislav Pyatkov
> > > Architect-Consultant "GridGain Rus" Llc.
> > > +7 963 716 68 99
> > >
> >
>
>
>
> --
> Vladislav Pyatkov
> Architect-Consultant "GridGain Rus" Llc.
> +7 963 716 68 99
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message