hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Hadoop in Web applications
Date Thu, 17 Jun 2010 10:13:42 GMT
Kilbride, James P. wrote:
> All,
> 
> What's the preferred way to use hadoop within java code running as a web
> service in something like tomcat? Would you edit the tomcat
> configuration to have access to the hadoop directory? Or simply include
> the hadoop jar and have the web service set some configuration somehow
> before calling submit(assuming the webservice implements the Tool
> interface)?

1. run hadoop services outside of the web server process,
2. bring up the Hadoop client libraries passing in a jobconf that binds 
to the filesystem/JT in the cluster
3. talk to the things remotely as you desire

there's no need to do the tool interface, just talk using the DfsClient 
and Filesystem client APIs.


e.g to submit work, call JobClient.submitJob(jobConf) on a client you've 
created, to work with a dfs given its URI and a conf file:

             FileSystem dfs = FileSystem.newInstance(uri, conf);
             dfs.initialize(uri, conf);

Mime
View raw message