manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chalitha Perera <cper...@zaizi.com>
Subject Getting class not found exception with DistributedFileSystem (New Connector for MICO)
Date Tue, 08 Mar 2016 05:13:03 GMT
Hi All,

We are writing a new transformation connector[1] to utilize MICO
platform[2]. Platform back-end uses hadoop for storage.
When trying to get the outputstream within connector which is a
HdfsDataOutputStream I am getting class not found exception with
DistributedFileSystem. ManifoldCF already includes hadoop-hdfs jar in
connectors-lib which should contain this class.

This error is generated only when used within manifoldcf. We have tried the
platform utilization code from a separate jar and it worked without any
errors.

Any help for this matter is greatly appreciated.

Here is the error I am getting,

Error tossed: java.lang.ClassNotFoundException: Class
org.apache.hadoop.hdfs.DistributedFileSystem not found
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.hdfs.DistributedFileSystem not found
    at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2074)
    at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2578)
    at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
    at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
    at
eu.mico.platform.storage.impl.StorageServiceHDFS.getOutputStream(StorageServiceHDFS.java:62)
    at
eu.mico.platform.persistence.impl.MarmottaContent.getOutputStream(MarmottaContent.java:266)
    at
org.apache.manifoldcf.agents.transformation.mico.MicoExtractor.addOrReplaceDocumentWithException(MicoExtractor.java:182)
    at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3221)
    at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3072)
    at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2706)
    at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
    at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
    at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
    at
org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:404)
    at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.hdfs.DistributedFileSystem not found
    at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1980)
    at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2072)


Thanks,
Chalitha


[1] https://github.com/zaizi/sensefy-mico-connector
[2] https://bitbucket.org/mico-project/platform

-- 

------------------------------
This message should be regarded as confidential. If you have received this 
email in error please notify the sender and destroy it immediately. 
Statements of intent shall only become binding when confirmed in hard copy 
by an authorised signatory.

Zaizi Ltd is registered in England and Wales with the registration number 
6440931. The Registered Office is Brook House, 229 Shepherds Bush Road, 
London W6 7AN. 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message