Partial answers are super=C2= =A0helpful!

I'm happy to break this up if it's too m= uch for 1 question @moderators=C2=A0

Sam

On Sat, Feb 27, 2021 at 1:27 PM, Sam Shl= eifer <sshleifer@gmail.com> wrote:

Hi!
I am trying= to use plasma store to reduce the memory usage of a pytorch dataset/datalo= ader combination, and had 4=C2=A0 questions. I don=E2=80=99t think any of t= hem require pytorch knowledge. If you prefer to comment inline there is a q= uip with identical content and prettier formatting here https:= //quip.com/3mwGAJ9KR2HT

*1)* My script starts the plasma-store from python with 200 GB:
=

nbytes =3D (1024 ** 3) * 200
_server =3D subprocess.Popen(["plasma_store", "= ;-m", str(nbytes), "-s", path])
where nbytes is chosen arbitrarily. From my experiments i= t seems that one should start the store as large as possible within the lim= its of dev/shm . I wanted to verify whether this is actually the best pract= ice (it would be hard for my app to know the storage needs up front) and al= so whether there is an automated way to figure out how much storage to allo= cate.

*2)* Does plasma store support simultaneous r= eads? My code, which has multiple clients all asking for the 6 arrays from = the plasma-store thousands of times, was segfaulting with different errors,= e.g.
Check failed: Remove= FromClientObjectIds(object_id, entry, client) =3D=3D 1
until I added a lock around my client.get
=

if self.use_lock: # Fix segfault
=C2=A0=C2=A0=C2=A0 with FileLock("/tmp/pla= sma_lock"):
=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ret =3D self.client.get(self.object_id)
else:
=C2=A0=C2=A0=C2=A0 ret =3D self.client.get(sel= f.object_id)

which fixes.

H= ere is a full traceback of the failure without the lock https://gist.github.= com/sshleifer/75145ba828fcb4e998d5e34c46ce13fc
Is this expected behavior?
<= div class=3D"sh-color-black sh-color">
*3)* Is there a simple way to add many objects to the plasma s= tore at once? Right now, we are considering changing,

oid =3D client.put(array)
to
oids =3D [client.pu= t(x) for x in array]

=
so that we can fetch one entry= at a time. but the writes are much slower.

* 3a) I= s there a lower level interface for bulk writes?
* 3b) Or is it recommended to chunk the array and ha= ve different python processes write simultaneously to make this faster?

*4)* Is there a way to save/load the contents of the p= lasma-store to disk without loading everything into memory and then saving = it to some other format?
<= br/>
Replication

Setup instructions for fairseq+replicating the segfault:=C2=A0https://gist.github.com/sshleifer/bd6982b3f632f1d4bcefc9feceb30b1a<= /a>
My code is here: https://github.com/pytorch= /fairseq/pull/3287

Thanks!
Sam

<= /div>