whirr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paolo Castagna (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (WHIRR-425) Define a distinct mechanism that handles the work of transfering artefacts on the remote machines
Date Wed, 02 Nov 2011 16:09:33 GMT

    [ https://issues.apache.org/jira/browse/WHIRR-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142243#comment-13142243
] 

Paolo Castagna commented on WHIRR-425:
--------------------------------------

I experienced an issue which is probably related to this, details here: http://markmail.org/message/3ldqgbl5ppj7rg5c

I started an Hadoop cluster of 17 nodes, but the NameNode reported only 8 live nodes.
Trying to connect to one of the instances not listed by the NameNode, I verified no Java process
was running and there were errors in the /tmp/logs/stderr.log file.
                
> Define a distinct mechanism that handles the work of transfering artefacts on the remote
machines
> -------------------------------------------------------------------------------------------------
>
>                 Key: WHIRR-425
>                 URL: https://issues.apache.org/jira/browse/WHIRR-425
>             Project: Whirr
>          Issue Type: Improvement
>          Components: core
>            Reporter: Andrei Savu
>
> As we begin to start larger clusters we need to start to think about a different, more
reliable and more scalable way of bringing the required artefacts on the remote machines.
I suggest we should create some abstraction so that we can use different policies based on
needs (e.g. for small clusters direct download for apache.org, for larger clusters build a
cache in s3 or download from machines inside the same cluster in a peer to peer fashion).


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message