systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Boehm (JIRA)" <j...@apache.org>
Subject [jira] [Deleted] (SYSTEMML-2172) Repartitioning before caching ulta-sparse matrices
Date Wed, 07 Mar 2018 08:26:00 GMT

     [ https://issues.apache.org/jira/browse/SYSTEMML-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matthias Boehm deleted SYSTEMML-2172:
-------------------------------------


> Repartitioning before caching ulta-sparse matrices
> --------------------------------------------------
>
>                 Key: SYSTEMML-2172
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2172
>             Project: SystemML
>          Issue Type: Bug
>            Reporter: Matthias Boehm
>            Priority: Major
>
> Ultra-sparse matrices have dedicated serialized block representation which means that
their in-memory storage in CSR can be much larger than on disk which leads to a blow-up of
128MB partitions to >1GB partitions. Accordingly, we should repartition the data before
the initial caching in order to remove memory pressure and exploit the full parallelism.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message