olingo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Haller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OLINGO-1183) JPAPage - Load all entities from DB, no paging on DB query
Date Mon, 08 Oct 2018 12:37:00 GMT

    [ https://issues.apache.org/jira/browse/OLINGO-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641761#comment-16641761
] 

Patrick Haller commented on OLINGO-1183:
----------------------------------------

*Analysis*

If using the paging with $top, $skip and $inlinecount=allpages the class JPAPage load all
entities ...
 * *JPAProcessorImpl.handlePaging( Query, GetEntitySetUriInfo )* is only setting query.setFirstResult()
and query.setMaxResult() if no $inlinecount is present at all. Hence if an $inlinecount is
present, it potentially reads hundreds of thousands of entities into memory. This has massive
impact on database performance and JVM memory consumption.
 * *ODataJPAResponseBuilderDefault.getInlineCountForNonFilterQueryEntitySet()* is using that
non-paged list of all entities of a query to determine the count. Only after that, the returned
result is reduced to page size.

 

This is a quite inefficient way to count entities. This inefficiency is unexpected to calling
clients and, even if it was, cannot always be circumvented.

A proposal would be that the JPAProcessorImpl is actually issuing a 2^nd^ database call during
GetEntitySet in conjunction with $inlinecount to just count the entities, without reading
them.

> JPAPage - Load all entities from DB, no paging on DB query
> ----------------------------------------------------------
>
>                 Key: OLINGO-1183
>                 URL: https://issues.apache.org/jira/browse/OLINGO-1183
>             Project: Olingo
>          Issue Type: Bug
>          Components: odata2-jpa
>    Affects Versions: V2 2.0.8, V2 2.0.9
>            Reporter: Francesco longhitano
>            Priority: Major
>
> If i using the paging with $top, $skip and $inlinecount=allpages the class JPAPage load
all entities from DB (howewer filtered by the where), no setting on top and skip for paginated
query directly from db. That's a problem if the query return many result (in our case ~100000
of complex entity).
> I'm looking into the code and the problem is not so easy to resolve. The inlinecount
is managed by ODataJPAResponseBuilderDefault, as an size() called on the result of query.
Then if there is top or skip the class do a filter on the result list, to return to client
only the correct entities. So even if i correct set top and skip on JPAPage modifing the method
handlePaging using an custom implementation of JPAProcessor, then the result __count show
an wrong result.
> Example URL
> Entities?$skip=10&$top=10&$inlinecount=allpages



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message