cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrija Panic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-8451) Static Nat show wrong remote IP in VM behind VPC
Date Tue, 26 May 2015 16:07:18 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-8451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14559300#comment-14559300
] 

Andrija Panic commented on CLOUDSTACK-8451:
-------------------------------------------

*From agent logs...(setting source nat for eth2)*

2015-05-26 17:30:24,873 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-3:null)
Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh vpc_snat.sh 169.254.2.49
 -A  -l XXX.YYY.147.26 -c eth2
2015-05-26 17:30:25,001 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-3:null)
Execution is successful.
2015-05-26 17:30:25,001 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-3:null)
iptables: Bad rule (does a matching rule exist in that chain?).
iptables: No chain/target/match by that name.

*From management logs (grep -i eth2 didnt give any explicit commands sent from management
server side, nor id=2 or similar)*
(mgmt log looks fine to me...)

2015-05-26 17:27:30,862 DEBUG [c.c.n.r.VpcVirtualNetworkApplianceManagerImpl] (Job-Executor-19:ctx-d99dacc4
ctx-bb38451d) Removing nic NicProfile[4903-2916-null-XXX.YYY.147.26-vlan://untagged of type
Public from the nics passed on vm start. The nic will be plugged later
2015-05-26 17:27:30,866 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Job-Executor-19:ctx-d99dacc4
ctx-bb38451d) Boot Args for VM[DomainRouter|r-2916-VM]:  vpccidr=10.0.0.0/8 domain=cs2cloud.internal
dns1=8.8.8.8 dns2= template=domP name=r-2916-VM eth0ip=169.254.2.49 eth0mask=255.255.0.0 type=vpcrouter
disable_rp_filter=true
2015-05-26 17:27:30,947 DEBUG [c.c.n.r.VpcVirtualNetworkApplianceManagerImpl] (Job-Executor-19:ctx-d99dacc4
ctx-bb38451d) Found 0 static routes to apply as a part of vpc route VM[DomainRouter|r-2916-VM]
start
2015-05-26 17:27:30,968 DEBUG [c.c.a.t.Request] (Job-Executor-19:ctx-d99dacc4 ctx-bb38451d)
Seq 793-1613229855: Sending  { Cmd , MgmtId: 161344838950, via: 793(cs12.domain.net), Ver:
v1, Flags: 100111, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":2916,"name":"r-2916-VM","type":"DomainRouter","cpus":1,"minSpeed":166,"maxSpeed":1000,"minRam":268435456,"maxRam":268435456,"arch":"x86_64","os":"Debian
GNU/Linux 7(64-bit)","bootArgs":" vpccidr=10.0.0.0/8 domain=cs2cloud.internal dns1=8.8.8.8
dns2= template=domP name=r-2916-VM eth0ip=169.254.2.49 eth0mask=255.255.0.0 type=vpcrouter
disable_rp_filter=true","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"ca8af10f1fd5804c","params":{"memoryOvercommitRatio":"1.0","cpuOvercommitRatio":"6.0"},"uuid":"518baeec-df0c-413e-9f26-07b7fb823601","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"59eab08e-4814-4e09-b1ee-34b357a430b2","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"5b93422e-1a66-353d-88a8-2203f79b1dc6","id":209,"poolType":"RBD","host":"cephmon.domain.net","path":"cloudstack","port":6789,"url":"RBD://cephmon.domain.net/cloudstack/?ROLE=Primary&STOREUUID=5b93422e-1a66-353d-88a8-2203f79b1dc6"}},"name":"ROOT-2916","size":2621440000,"path":"59eab08e-4814-4e09-b1ee-34b357a430b2","volumeId":8416,"vmName":"r-2916-VM","accountId":2,"format":"RAW","id":8416,"deviceId":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"59eab08e-4814-4e09-b1ee-34b357a430b2","type":"ROOT","_details":{"managed":"false","storagePort":"6789","storageHost":"cephmon.domain.net","volumeSize":"2621440000"}}],"nics":[{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"2a657fb8-c645-47c2-a335-e2d2c7da030c","ip":"169.254.2.49","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:31","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"hostIp":"10.xxx.yyy.120","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.2.49","port":3922,"interval":6,"retries":100,"name":"r-2916-VM","wait":0}},{"com.cloud.agent.api.GetDomRVersionCmd":{"accessDetails":{"router.ip":"169.254.2.49","router.name":"r-2916-VM"},"wait":0}},{"com.cloud.agent.api.PlugNicCommand":{"nic":{"deviceId":1,"networkRateMbps":99999,"defaultNic":true,"uuid":"de8f637c-195d-4455-9035-81f8d4f74e09","ip":"XXX.YYY.147.26","netmask":"255.255.255.128","gateway":"XXX.YYY.147.1","mac":"06:f3:72:00:01:b2","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://untagged","isolationUri":"vlan://untagged","isSecurityGroupEnabled":false,"name":"breth1-500"},"instanceName":"r-2916-VM","vmType":"DomainRouter","wait":0}},{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":2,"publicIp":"XXX.YYY.147.26","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":false,"broadcastUri":"vlan://untagged","vlanGateway":"XXX.YYY.147.1","vlanNetmask":"255.255.255.128","vifMacAddress":"06:f3:72:00:01:b2","networkRate":99999,"trafficType":"Public","networkName":"breth1-500"}],"accessDetails":{"router.guest.ip":"XXX.YYY.147.26","zone.network.type":"Advanced","router.ip":"169.254.2.49","router.name":"r-2916-VM"},"wait":0}},{"com.cloud.agent.api.routing.SetSourceNatCommand":{"ipAddress":{"accountId":2,"publicIp":"XXX.YYY.147.26","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":false,"broadcastUri":"vlan://untagged","vlanGateway":"XXX.YYY.147.1","vlanNetmask":"255.255.255.128","vifMacAddress":"06:f3:72:00:01:b2","networkRate":99999,"trafficType":"Public","networkName":"breth1-500"},"add":true,"accessDetails":{"zone.network.type":"Advanced","router.ip":"169.254.2.49","router.name":"r-2916-VM"},"wait":0}},{}]
}
...
...
2015-05-26 17:30:25,144 DEBUG [c.c.a.t.Request] (AgentManager-Handler-13:null) Seq 793-1613229855:
Processing:  { Ans: , MgmtId: 161344838950, via: 793, Ver: v1, Flags: 110, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":2916,"name":"r-2916-VM","type":"DomainRouter","cpus":1,"minSpeed":166,"maxSpeed":1000,"minRam":268435456,"maxRam":268435456,"arch":"x86_64","os":"Debian
GNU/Linux 7(64-bit)","bootArgs":" vpccidr=10.0.0.0/8 domain=cs2cloud.internal dns1=8.8.8.8
dns2= template=domP name=r-2916-VM eth0ip=169.254.2.49 eth0mask=255.255.0.0 type=vpcrouter
disable_rp_filter=true","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"ca8af10f1fd5804c","vncAddr":"10.44.253.120","params":{"memoryOvercommitRatio":"1.0","cpuOvercommitRatio":"6.0"},"uuid":"518baeec-df0c-413e-9f26-07b7fb823601","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"59eab08e-4814-4e09-b1ee-34b357a430b2","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"5b93422e-1a66-353d-88a8-2203f79b1dc6","id":209,"poolType":"RBD","host":"cephmon.domain.net","path":"cloudstack","port":6789,"url":"RBD://cephmon.domain.net/cloudstack/?ROLE=Primary&STOREUUID=5b93422e-1a66-353d-88a8-2203f79b1dc6"}},"name":"ROOT-2916","size":2621440000,"path":"59eab08e-4814-4e09-b1ee-34b357a430b2","volumeId":8416,"vmName":"r-2916-VM","accountId":2,"format":"RAW","id":8416,"deviceId":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"59eab08e-4814-4e09-b1ee-34b357a430b2","type":"ROOT","_details":{"managed":"false","storagePort":"6789","storageHost":"cephmon.domain.net","volumeSize":"2621440000"}}],"nics":[{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"2a657fb8-c645-47c2-a335-e2d2c7da030c","ip":"169.254.2.49","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:31","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"result":true,"wait":0}},{"com.cloud.agent.api.check.CheckSshAnswer":{"result":true,"wait":0}},{"com.cloud.agent.api.GetDomRVersionAnswer":{"templateVersion":"Cloudstack
Release 4.3.2 (64-bit) Wed Jan 28 18:38:51 UTC 2015","scriptsVersion":"253cafa254fc386e9fce204d9395a181","result":true,"details":"Cloudstack
Release 4.3.2 (64-bit) Wed Jan 28 18:38:51 UTC 2015&253cafa254fc386e9fce204d9395a181","wait":0}},{"com.cloud.agent.api.PlugNicAnswer":{"result":true,"details":"success","wait":0}},{"com.cloud.agent.api.routing.IpAssocAnswer":{"results":["XXX.YYY.147.26
- success"],"result":true,"wait":0}},{"com.cloud.agent.api.routing.SetSourceNatAnswer":{"result":true,"details":"success","wait":0}},{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-2916-VM","bytesSent":0,"bytesReceived":0,"result":true,"wait":0}}]
}

Any idea why the script is run with eth2 as parameter when creating empty new VPC (eth0, eth1,
and lo present only) ?

I suspect this might have something to do with the fact that we dont use vlan tags on public
network, 
I will try to test with tagging, since there was also some bugs arround this *vlan://untagged*
previously...so that is my best guess now :(


> Static Nat show wrong remote IP in VM behind VPC
> ------------------------------------------------
>
>                 Key: CLOUDSTACK-8451
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8451
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: KVM, Network Controller, Virtual Router
>    Affects Versions: 4.4.3, 4.3.2, 4.5.1
>         Environment: Ubuntu 14.04, ACS 4.5.1-SNAPSHOT
>            Reporter: Andrija Panic
>            Assignee: Rohit Yadav
>
> When configuring Port FOrwarding or Static NAT on VPC VR, and connect from outside world
to VPC IP address, traffic gets forwarded to VM behind VPC.
> But if you run "netstat -antup | grep $PORT" (where port is i.e. ssh port) - given result
will show that remote connections come from the Source NAT IP of the VR, instead of the real
remote client IP.
> Example:
> private VM: 192.168.10.10
> Source NAT IP on VPC VR: 1.1.1.1
> Additional Public IP on VPC VR. 1.1.1.2
> Remote client public IP: 4.4.4.4 (external to VPC)
> Test:
> from 4.4.4.4 SSH to 1.1.1.2 port 22 (or any other port)
> inside 192.168.10.10 do "netstat -antup | grep 22"
> Result: Remote IP show is 1.1.1.1 instead of 4.4.4.4
> We found a solution (somwhat tested, and not sure if this would break anything...)
> Problem is in VRs iptables NAT table, POSTROUTING chain, rule:
> SNAT all -- * eth2 0.0.0.0/0 0.0.0.0/0 to:1.1.1.1
> where 1.1.1.1 is public IP of VR
> eth2: is Public Interface of VR
> When this rule is deleted, NAT is working fine.
> This is serious issue for anyone using VPC, since there is no way to see real remote
client IP, and this no firewall funtionality inside VM, SIP doesnt work, web server logs are
useless etc.
> I also experienced this problem with 4.3.x releases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message