The problem I have is the following : we have to remember the pointer to the last entry we have sent back to the client
How should we do ? My first approach was pretty naive : we are using a cursor, so it's easy, we simply store the cursor into the session, and the next request will just have to get back this cursor from the session, and get the N next elements from this cursor.
This has the advantage of being simple, but there are some very important cons :
- it's memory consuming, as we may keep those cursor in the session for a very long time
- we will have to close all the cursors when the session is closed (for whatever reason)
- if some data has been modified since the cursor creation, it may contain invalid data
- if the user don't send and abandon search request, those cursors will remain in the session until it's closed (this is very likely to happen)
So I'm considering an alternative - though more expensive and less performant - approach :
- we build a new cursor for each request,
- we move forward the Nth entry in the newly created cursor, and return back the M requested elements
- and when done, we discard the cursor.
The pros are
- we don't have to keep n cursors in memory for ever.
- from the client POV, it respects the PagedSeach contract
- it's easier to implement as we have less information to keep in the session, and to restore back
The cons are :
- it's time consuming, as if we have N entry to return, with a P page size, we will construct N/P cursors.