systemml-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deron Eriksson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SYSTEMML-453) JMLC in Standalone mode and dependency on Hadoop
Date Wed, 14 Jun 2017 04:53:00 GMT

    [ https://issues.apache.org/jira/browse/SYSTEMML-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048706#comment-16048706
] 

Deron Eriksson commented on SYSTEMML-453:
-----------------------------------------

Looking over the code, it looks like Hadoop is very tightly coupled into certain data structures
such as MatrixBlock and OutputInfo. As a result, it is probably hard to remove Hadoop classes
completely when using JMLC, but it should be possible to greatly reduce the memory footprint
required to run JMLC using an in-memory jar. The need for most Hadoop classes (such as those
in org.apache.hadoop.conf) when using JMLC can probably be eliminated with some refactoring.

I'll have a look at it.

cc [~reinwald] [~mboehm7]


> JMLC in Standalone mode and dependency on Hadoop
> ------------------------------------------------
>
>                 Key: SYSTEMML-453
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-453
>             Project: SystemML
>          Issue Type: Improvement
>          Components: APIs
>            Reporter: Laura Chiticariu
>            Assignee: Deron Eriksson
>
> Hello,
> I am using the JMLC API in standalone mode. But Instantiating org.apache.sysml.api.jmlc.Connection
fails with NoClassDefFoundError: org/apache/hadoop/conf/Configuration.
> Is it possible to remove the dependency on Hadoop when running SystemML in standalone
mode?
> Thanks,
> Laura



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message