incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From JiHyoun Park <>
Subject pluggable s4r fetching strategies
Date Tue, 16 Apr 2013 10:08:05 GMT

I am trying to apply 'pluggable s4r fetching strategies' of S4-25 to S4

Changes that I made are

    protected void configure() {
        // added this codes
        Multibinder<ArchiveFetcher> archiveFetcherMultibinder =
Multibinder.newSetBinder(binder(), ArchiveFetcher.class);


    // added this codes
    boolean handlesProtocol(URI uri);

    // added this codes
    public boolean handlesProtocol(URI uri) {
        return "file".equalsIgnoreCase(uri.getScheme());

    // added this codes
    public boolean handlesProtocol(URI uri) {
        return ("http".equalsIgnoreCase(uri.getScheme()) ||

    // added this codes
    private final Set<ArchiveFetcher> archiveFetchers;

    // added this codes
    public RemoteFileFetcher(Set<ArchiveFetcher> archiveFetchers) {
        this.archiveFetchers = archiveFetchers;

    public InputStream fetch(URI uri) throws ArchiveFetchException {
        /* removed this codes
        if ("file".equalsIgnoreCase(scheme)) {
            return new FileSystemArchiveFetcher().fetch(uri);
        if ("http".equalsIgnoreCase(scheme) ||
"https".equalsIgnoreCase(scheme)) {
            return new HttpArchiveFetcher().fetch(uri);

        // added this codes
        for (ArchiveFetcher archiveFetcher : archiveFetchers) {
            if (archiveFetcher.handlesProtocol(uri)) {
                return archiveFetcher.fetch(uri);

6) I passed the "-modulesClasses=my.HdfsFetcherModule" argument when I
deploy my s4r file. The HdfsFetcherModule is the same HdfsFetcherModule
from S4-25.

But, when I run it, I got this error.

17:46:24.151 [S4 platform loader] ERROR org.apache.s4.core.S4Bootstrap
- Cannot start S4 node
org.apache.s4.deploy.DeploymentFailedException: Cannot deploy
application [SimpleApp] from URI
	at org.apache.s4.core.S4Bootstrap.loadApp(
	at org.apache.s4.core.S4Bootstrap.startS4App(
	at org.apache.s4.core.S4Bootstrap.access$000(
	at org.apache.s4.core.S4Bootstrap$
	at [na:1.6.0_22]
Caused by: org.apache.s4.core.util.ArchiveFetchException: Unsupported
protocol hdfs
	at org.apache.s4.core.util.RemoteFileFetcher.fetch(
	at org.apache.s4.core.S4Bootstrap.loadApp(
	... 4 common frames omitted

I think there are some missing links to make S4 recognize the custom module.
Is it related with the new parameter "-modulesURIs"? How can I use it?
I found some description for this parameter at
        @Parameter(names = { "-modulesURIs", "-mu" }, description = "URIs
for fetching code of custom modules")
        List<String> modulesURIs = new ArrayList<String>();

But I have no idea about how to use it. My custom module will be located in
HDFS with the s4r file that I want to deploy.
Or are there any other things that I have to take into consideration in the
above implementation?

Best Regards

View raw message