systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deron Eriksson (JIRA)" <>
Subject [jira] [Commented] (SYSTEMML-453) JMLC in Standalone mode and dependency on Hadoop
Date Wed, 14 Jun 2017 04:53:00 GMT


Deron Eriksson commented on SYSTEMML-453:

Looking over the code, it looks like Hadoop is very tightly coupled into certain data structures
such as MatrixBlock and OutputInfo. As a result, it is probably hard to remove Hadoop classes
completely when using JMLC, but it should be possible to greatly reduce the memory footprint
required to run JMLC using an in-memory jar. The need for most Hadoop classes (such as those
in org.apache.hadoop.conf) when using JMLC can probably be eliminated with some refactoring.

I'll have a look at it.

cc [~reinwald] [~mboehm7]

> JMLC in Standalone mode and dependency on Hadoop
> ------------------------------------------------
>                 Key: SYSTEMML-453
>                 URL:
>             Project: SystemML
>          Issue Type: Improvement
>          Components: APIs
>            Reporter: Laura Chiticariu
>            Assignee: Deron Eriksson
> Hello,
> I am using the JMLC API in standalone mode. But Instantiating org.apache.sysml.api.jmlc.Connection
fails with NoClassDefFoundError: org/apache/hadoop/conf/Configuration.
> Is it possible to remove the dependency on Hadoop when running SystemML in standalone
> Thanks,
> Laura

This message was sent by Atlassian JIRA

View raw message