systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremy Nilmeier" <nilme...@us.ibm.com>
Subject Re: distributed cholesky on systemml
Date Sun, 22 Apr 2018 05:03:00 GMT


Hi Team:

I can try to answer some questions about these scripts...Imran and I had
some discussions about them some time ago.

Cheers, J

Sent from my iPhone

> On Apr 21, 2018, at 7:05 PM, Qifan Pu <qifan.pu@gmail.com> wrote:
>
> Hi Matthias,
>
> Thanks for the fast response and detailed information. This is really
> helpful.
>
> I just tried to run it, and was tracing down a indexing bug that can be
> repeated by simply running the test script of triangle solve[1]
> Caused by: org.apache.sysml.runtime.DMLRuntimeException: Invalid values
for
> matrix indexing: [1667:3333,1:1666] must be within matrix dimensions
> [1000,1000]
>
>
> Am I missing some configuration here?
>
>
> [1]
>
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_systemml_blob_master_scripts_staging_scalable-5Flinalg_test_test-5Ftriangular-5Finv.dml&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=3mYOfURw_FSirAnoSv2pWvLSi1psso4F9RdGjEWL6yc&m=EGIIC2kHgLOJOxCqvFgtDxc4AhkH5hragWnnueZX_e8&s=xknejm8MxIyJ1kwEEWEGx-lLMlRe2mky9J5JH0gNVbM&e=

>
>
> Best,
> Qifan
>
>
>> On Sat, Apr 21, 2018 at 4:06 PM, Matthias Boehm <mboehm7@gmail.com>
wrote:
>>
>> Hi Qifan,
>>
>> thanks for your feedback. You're right, the builtin functions
>> cholesky, inverse, eigen, solve, svd, qr, and lu are currently only
>> supported as single-node operations because they're still implemented
>> via Apache commons.math.
>>
>> However, there is an experimental script for distributed cholesky [1]
>> which uses a recursive approach (with operations that allow for
>> automatic distributed computation) for matrices larger than a
>> user-defined block size. Once blocks become small enough, we use again
>> the builtin cholesky. Graduating this script would require a broader
>> set of experiments (and potential improvements) but it simply did not
>> have the highest priority so far. You might want to give it a try
>> though.
>>
>> Thanks again for your feedback - we'll consider a higher priority for
>> these distributed operations when discussing the roadmap for the next
>> releases.
>>
>> [1]
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_systemml_blob_master_scripts_&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=3mYOfURw_FSirAnoSv2pWvLSi1psso4F9RdGjEWL6yc&m=EGIIC2kHgLOJOxCqvFgtDxc4AhkH5hragWnnueZX_e8&s=gSIeBektBhtRHzozya6is0WfS2hSyUjkKi3TPawTBxc&e=

>> staging/scalable_linalg/cholesky.dml
>>
>> Regards,
>> Matthias
>>
>>> On Sat, Apr 21, 2018 at 2:15 PM, Qifan Pu <qifan.pu@gmail.com> wrote:
>>> Hi,
>>>
>>> I would love to do distributed cholesky on large matrix with SystemML.
I
>>> found two related jiras (SYSTEMML-1213, SYSTEMML-1163), but AFAIK, this
>> is
>>> currently not implemented? I just wanted to check.
>>>
>>> Best,
>>> Qifan
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message