cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Min Chen <min.c...@citrix.com>
Subject Re: Sending to LocalHostEndpoint, Is there any resolution for this?
Date Thu, 31 Oct 2013 18:02:55 GMT
Yes, this issue exist in 4.2/4.2.1 since DefaultEndpointSelector was
introduced in 4.2.

Thanks
-min

On 10/31/13 10:56 AM, "Darren Shepherd" <darren.s.shepherd@gmail.com>
wrote:

>Does this issue exist in 4.2/4.2.1?  I'm going to put the bug as blocker,
>if you disagree we can lower it.  I really think it is quite a bad
>scenario
>to mount NFS to the mgmt server.
>
>Darren
>
>
>On Thu, Oct 31, 2013 at 10:48 AM, Min Chen <min.chen@citrix.com> wrote:
>
>> Delaying them until ssvm is up may not that easy to handle in current
>> CloudStack. We may be able to send an error in this case by
>>distinguishing
>> S3 register template case from other scenarios. Would you mind filing a
>> JIRA bug on this? Your issue seems different from the issue filed by
>>Daan
>> (https://issues.apache.org/jira/browse/CLOUDSTACK-4759).
>>
>> Thanks
>> -min
>>
>>
>> On 10/31/13 10:38 AM, "Darren Shepherd" <darren.s.shepherd@gmail.com>
>> wrote:
>>
>> >The behaviour should be to delay until there is a SSVM available.  I
>>don't
>> >know if that is even possible in CloudStack, I don't want you to
>>sleep().
>> >Is there any way to delay?  Something else in the system should be
>> >ensuring
>> >that a SSVM is running and available (what I don't know...).  If you
>>can't
>> >delay, it is far better in my mind to send an error to the user than to
>> >try
>> >to do the operation on the mgmt server.
>> >
>> >I think it is very dangerous to have this code that routes to localhost
>> >because in a production setup ACS will have the sudo access to
>> >mount/mkdir.  But the server may not have network access, so you run
>>the
>> >chance of mounting the NFS and having it hang almost indefinitely which
>> >can
>> >cause system instability for the mgmt server.
>> >
>> >The specific issue I have right now is that for some reason on master,
>>the
>> >allocator is not working for me anymore.  So SSVM doesn't deploy.  If I
>> >restart the mgmt stack, on host connect it tries to download templates
>>so
>> >it sends a command and it goes to LocalHostEndPoint.
>> >
>> >Darren
>> >
>> >
>> >On Thu, Oct 31, 2013 at 10:28 AM, Min Chen <min.chen@citrix.com> wrote:
>> >
>> >> Yes, that is true, it is a bug in that case. The logic of choosing
>> >> endpoint to send Command will not work properly for
>>destroying/Stopping
>> >>VM
>> >> case. Our assumption is that registering template is done when ssvm
>>is
>> >>up
>> >> and running. Otherwise, it will send to LocalHost to execute
>>command. In
>> >> your opinion, what should be expected behavior if user is trying to
>> >> register template when ssvm is crashed? Directly reporting error
>>instead
>> >> of trying anything.
>> >>
>> >> Thanks
>> >> -min
>> >>
>> >> On 10/31/13 10:18 AM, "Darren Shepherd" <darren.s.shepherd@gmail.com>
>> >> wrote:
>> >>
>> >> >It's happening when I first create the zone.  I'll debug it further
>>to
>> >> >tell
>> >> >you exactly what its happening.  But I don't understand how the
>> >>following
>> >> >code in DefaultEndpointSelector could be correct.  If I have a zone,
>> >>and I
>> >> >delete/stop/crash the SSVM, and at the same time register a new
>> >>template,
>> >> >it seems the DownloadCommand will go to LocalHostEndPoint.
>> >> >
>> >> >    protected EndPoint findEndpointForImageStorage(DataStore store)
>>{
>> >> >        Long dcId = null;
>> >> >        Scope storeScope = store.getScope();
>> >> >        if (storeScope.getScopeType() == ScopeType.ZONE) {
>> >> >            dcId = storeScope.getScopeId();
>> >> >        }
>> >> >        // find ssvm that can be used to download data to store. For
>> >> >zone-wide
>> >> >        // image store, use SSVM for that zone. For region-wide
>>store,
>> >> >        // we can arbitrarily pick one ssvm to do that task
>> >> >        List<HostVO> ssAHosts =
>> >> >listUpAndConnectingSecondaryStorageVmHost(dcId);
>> >> >        if (ssAHosts == null || ssAHosts.isEmpty()) {
>> >> >            s_logger.info("No running ssvm is found, so command will
>> be
>> >> >sent to LocalHostEndPoint");
>> >> >            return LocalHostEndpoint.getEndpoint(); // use local
>>host
>> >>as
>> >> >endpoint in
>> >> >            // case of no ssvm existing
>> >> >        }
>> >> >        Collections.shuffle(ssAHosts);
>> >> >        HostVO host = ssAHosts.get(0);
>> >> >        return
>> >>RemoteHostEndPoint.getHypervisorHostEndPoint(host.getId(),
>> >> >host.getPrivateIpAddress(),
>> >> >                host.getPublicIpAddress());
>> >> >    }
>> >> >
>> >> >Darren
>> >> >
>> >> >
>> >> >On Thu, Oct 31, 2013 at 9:42 AM, Min Chen <min.chen@citrix.com>
>>wrote:
>> >> >
>> >> >> Hi Darren,
>> >> >>
>> >> >>         The logic of sending command to Local in case of no SSVM
>>is
>> >>up
>> >> >>is
>> >> >> to
>> >> >> support S3 as secondary storage. In case of S3 as secondary
>>storage,
>> >> >> system vm template is automatically downloaded to S3 when S3 is
>>added
>> >> >>into
>> >> >> CloudStack, unlike NFS secondary storage case where we used a
>>script
>> >>to
>> >> >> pre-populate system vm template. The thing I don't get is: what
>> >>scenario
>> >> >> are you trying to do to trigger DownloadCommand when SSVM is not
>>up?
>> >> >>
>> >> >>         Thanks
>> >> >>         -min
>> >> >>
>> >> >> On 10/31/13 9:32 AM, "Darren Shepherd"
>><darren.s.shepherd@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >> >No, that's not the problem.  The problem is that the
>>DownloadCommand
>> >> >> >is supposed to go to the SSVM, but it sending it to the
>> >> >> >LocalHostEndPoint which is the mgmt server.  So the mgmt server
>>can
>> >> >> >not create /var/cloudstack/mnt/
>> >> >> >secStorage/64d6e26f-e656-3ba3-908f-ce6610ede011.  I have my
mgmt
>> >>setup
>> >> >> >so that it does not have sudo access.  A standard cloudstack
>>install
>> >> >> >has sudo access so it will create those folders and mount on
the
>> >>mgmt
>> >> >> >server, but it shouldn't.
>> >> >> >
>> >> >> >Please don't tell me we made a change so that the mgmt server
is
>> >> >> >expected to have NFS access to all secondary storages.
>> >> >> >
>> >> >> >Darren
>> >> >> >
>> >> >> >On Thu, Oct 31, 2013 at 8:54 AM, Wei ZHOU <ustcweizhou@gmail.com>
>> >> >>wrote:
>> >> >> >> Please check if you can
>> >> >> >> create
>> >> >>
>> >>>>/var/cloudstack/mnt/secStorage/64d6e26f-e656-3ba3-908f-ce6610ede011,
>> >> >> >> and mount nfs point.
>> >> >>
>> >> >>
>> >>
>> >>
>>
>>


Mime
View raw message