felix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guillaume Nodet (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FELIX-4656) Improve memory usage and speed of the resolver
Date Fri, 17 Apr 2015 16:29:59 GMT

    [ https://issues.apache.org/jira/browse/FELIX-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500133#comment-14500133
] 

Guillaume Nodet commented on FELIX-4656:
----------------------------------------

[~tjwatson] I've applied your patch.  I don't have any real comments on it.  I initially called
it path because we add the tried permutation to the previous set, but given it's unordered
by design, the term delta is much more appropriate.

I'd like to keep the uses / imports permutations to try in an ordered collection (it's a ArrayList,
but it could be a LinkedHashSet) so that the resolver remains predictable, i.e. the given
input always return the same output.   I've fixed a couple of locations where things used
to be put in a simple HashMap, which can lead to different results depending on the hash code
of the objects.
One of the idea I had, which I did not came to try yet, was to allow some pluggable heuristics
in the resolver, mostly to pick the next permutation to try.

I must admit I haven't really gone through the whole algorithm in depth while doing this optimization,
as my goal was really to not change the algorithm at all.  But at first glance, I think you're
right that we could use the already processed delta set to mostly get rid of the permutateIfNeeded()
method.  I'd have to double check though.
We could even avoid the full copy of the Candidates object, by creating the new delta first
and verifying it has not been already processed and it's not in the list of permutations to
try yet.

That said, I'd rather create new JIRA issues for such enhancements and do a release with what
we have now.




> Improve memory usage and speed of the resolver
> ----------------------------------------------
>
>                 Key: FELIX-4656
>                 URL: https://issues.apache.org/jira/browse/FELIX-4656
>             Project: Felix
>          Issue Type: Improvement
>          Components: Resolver
>            Reporter: Guillaume Nodet
>            Assignee: Guillaume Nodet
>             Fix For: resolver-1.2.0
>
>         Attachments: FELIX-4656.patch
>
>
> During big resolutions (> 100 bundles), the memory consumption can become very huge,
mostly by keeping a lot of copies of the Candidates object.
> I want to lower the memory requirements of the resolver without touching the algorithm
at all (which would be a different improvement).
> This can be done by using :
>   * lower memory intensive collections
>   * do smart copies of those collections (where they would only actually copy the data
when modify)
> The second item is slightly more difficult to achieve, as the maps in the Candidate objects
contains Set and List, which would mean that those must be copied too.  So it could actually
be complementary, if achievable.
> For the first one, the HashMap and HashSet are very memory intensive.  I'll introduce
two new collections which will lower the requirements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message