httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 45449] [PATCH] Add support to WebDav to MOVE/COPY to remote servers
Date Wed, 07 Oct 2009 17:39:57 GMT
https://issues.apache.org/bugzilla/show_bug.cgi?id=45449

--- Comment #6 from Pierre Delaage <delaage.pierre@free.fr> 2009-10-07 10:39:50 PDT
---
(In reply to comment #5)
> Reset assignee so mails go to list.

Provided that proper usage of Apache Reverse Proxying is made, then NO PATCH is
needed to make MOVE/COPY work as expected.

I Highly recommend that this patch SHOULD NOT be included in the apache source
tree, although this is a good programming effort.

Here is a detailed memo I wrote on the subject, with 3 different solutions to
this problem:

====================================================

I/ The problem: 
Briefly: one cannot rename, move or copy (from remote to remote) files on a
https webdav location accessed via stunnel, from dav client such as Windows
Webdav Mini-Redir (ie "net use" mapped drive).

However rename or copy work with "Add Network Place" wizard, but is very long
for big files.
Also, via stunnel, accessing a classic www location works fine (provided that
all url in html files are relative!)



II/ CAUSE:

Quick comments :
there is no bug in Apache,
no bug in stunnel,
and no bug (related to this pb)  in Windows webdav miniredir client

so what?...

The method MOVE and COPY of the dav protocol are most often implemented in dav
clients by using an absolute destination url such as
"http://myserver.mydomain/folder1" or "https://sslserver.otherdomain/folder2",
WHILE the "source" document url is just expressed as a relative path to the
"Host" designated in the http header of the request.
All other http and dav methods, which are essentially some kind of "get
something methods" are most often implemented by using relative path such
"/folderX/adoc" to designate a document of interest, "relative" means relative
to the "host" field defined in the header of the http request. Note:  The PUT
method is built the same way as the GET methods: ie with a relative path to the
implicit "document root" of the server designated in the "Host:" http header
field of the request (see RFC 2616 for http, RFC4918 for dav).
Well, from a vocabulary point of view, rfc2626 call my "relative path" as
"abs-path" because they start with a "/", and not a dot (see chapter 3.2.1 of
rfc2616).

That said, to secure a dav communication between a XP client and a DAV server,
some people need to set up an ssl (s)tunnel between a WXP/Dav MiniRedir client
and a HTTPS/dav server: this because WXP client are not able to manage "https
connection" directly through "net use". So they usually use a setup like this
on their client machine  stunnel.conf file:
accept: localhost:80
connect: remotehost:443
+ some other stuff for cert management.
The XP webdav miniredir client is only able to connect to 80 location. On XP
"net use" does not support https locations nor forcing port number to any
desired value.

So, with such setup, clients are about to build url like this
"http://localhost/remotefolder/remotedoc" in their dav client, then put this
request in an ssl tunnel (stunnel) which will transport this request to the 443
port of a remote server. 
Let's say that the remote server is an Apache one (for example).

A classic configuration of Apache is to setup two ip-based vhosts to access the
same locations (it is very simple and even recommended because apache does not
support ssl access on name-based vhost):
- one ip-based on 80 port (0.0.0.0:80 is the good way to write this, _default_ 
and * are not appropriate in that case)
- the other ip-based on 443 port
Some may restrict the 80 vhost to "apache server local network", and forbid
access from the whole internet, and keep 443 access with client cert validation
for external access.

So what happens when MOVE/RENAME is rejected :
the http client will build a MOVE request specifying :
  - a source document as a relative "file path", relative to "localhost", which
is an "happily" AMBIGUOUS name (or address if used as 127.0.0.1)  that will
allow BOTH the client to send its resquest in its local tunnel-end AND the
server to interpret url as relative to ITSELF (ie the server, not the client)
at the other end of the (s)tunnel, so that requested document can be reaaly
reached.
I will explain this further but you have to understand that it is really a
chance that the name "localhost" can serve two different purposes on two
different machines, when it is used an an url exchanged between the two
machines, because an url should express a unic "place" -not two! 
I call that the ubiquity of localhost based url, such as "/localhost/src_doc",
relative to the "localhost machine" but in fact relevant on the SERVER machine
  - a destination location as an ABSOLUTE url, such as
"http://localhost/newpath/newdoc_name", also relevant in fact only on the
server machine

Then there are at least two potential sources of pb:
1/ nothing assures the client that the server will be able/allowed to
reach/write to such a location "http://someserver" (even if someserver is the
"itself" remote server contacted by the client, because http urls are not "the
same" as https urls)
2/ even if that location is reachable, the server receives this request on its
443 port and may expect that url be specified as https, and may consider
receiving an http url as a bug
(because Apache does not want to proxy by default between two of its vhosts:
you have to ask it ...and I think this in NORMAL).
This is where the bug mainly is !

III/ Various comments and questions

1/ the "localhost" ubiquity good bug 
-----------------------------------------
There should be a 3rd problem, but this will fortunately not be the case
because of the "ubiquity/ambivalence" of the "locahost" hostname both in the
client network space, and on the server network space:
normally, when the client builds an url beginning with "http://localhost", it
really wants to address and truly adresses a http request to ITS own host
machine: 
by the stunnel magic that request comes to the REMOTE server which, except on
particular requests, usually accepts this and serves it : 
that means that the server, when seeing such request,  interprets "localhost"
as "ITSELF", and as "itself" is in general allowed to access all its web
locations, then the server accepts to serve the request and send back a
document in the tunnel back to the client.
You have to understand that this is NOT (so) normal because the "locahost"
requested by the client is TRULY not the same host machine as that "understood"
by the server.
This is what I call the "localhost" ubiquity good-bug : without this bug no ssl
access through stunnel would work (no www, no dav, no http, no https,
nothing!).
This property is not at all related to a kind of "proxying" role of stunnel:
this is something totally different.
Remember also that stunnel is NOT an http proxy so it transports http requests
"as is".

2/ HTTP + SSL is NOT HTTPS
------------------------------------
Because absolute http url requested by the client are directed to an https
server: as the client does not know that it is talking to an httpS server, it
does not build correct urls.
So it is even a chance that most request works (due to localhost ambivalence
and usage of relative path)...

3/ role of stunnel in HTTP over SSL 
-----------------------------------------

In that matter stunnel is not acting as a transparent proxy (this is not
related to the "transparent proxy option" of stunnel) : if it were perfectly
transparent it would not redirect a request to port 80 to a 443 port, and will
not "route" requests for THE localhost to a machine on the internet.
It is also not used as a proxy because the client does not deal with the remote
adress but needs to know the adress of the stunnel client machine (localhost in
the present case).

In fact stunnel, when used only on one side of a end-to-end communication, is
acting as a GATEWAY between two networks : one talking ssl, the other not.
As such it can translate protocols by removing the payload from an ordinary
tcp/ip packet on one side and putting it in a new ssl tcp/ip packet on the
other side.
We can also say that stunnel acts as a "reverse proxy" hiding a remote machine
behind itself...

Well do not think that tricking your stunnel conf file with "accept 443" and
"connect 443" will solve your problem: because absolute urls in your requests
still begin with http ! and Apache will still not appreciate it (no implicit
proxying).
Moreover MS XP clients will not allow you to mix http / 443 in a net use
command  (it is possible on Vista but useless for other reasons: on vista you
can directly connect to https locations with net use)

Stunnel is not either acting as a ...tunnel ! No it does not, because a pure
tunnel will encapsulate requests at one end and remove encapsulation at the
other end.

4/ Some workarounds (but the true solution is later)
-----------------------------------------------------------
WORKAROUND 1 : 
if you are impatient  to solve your problem, a good solution is to put stunnel
at both ends AND remove SSL support from Apache: 
so put stunnel both on the client and on the server to REALLY build a
TRANSPARENT tunnel: I mean transparent to the fact that both ends believes to
talk directly "http on 80 port", as they are in fact tunneled through an ssl
session.
Also note that this true ssl tunnel will work thanks to the "localhost
ambivalence": without that ambivalence, not even a simple get request would
work. 
NOTE: please check your apache conf with various directives such as
"usecanonicalname" : I put if off in my conf, 

WORKAROUND 2: 
of course something else which works is "true https-able client connecting
directly to true https server": this is possible by using the Windows Vista
webdav mini-redir directly talking to apache server on 443. Well in that case
you will still have ANOTHER MS BUG : "net use" DOES not support BOTH "client
cert auth" AND "basic auth by username/password" to access directly a folder on
the server (whereas "add network place" does!). If you want the Vista client to
work you will be obliged to remove "basic auth restriction" at least at the
(dav-)root / location on the server: this is not a good new.
Anyway this will work perfectly if you only use SSL to authenticate the server,
not the client, and keep your basic auth settings unchanged.
Well this solution does not involve stunnel so let us continue...


5/ BUT why other requests such as put/get are working?
--------------------------------------------------------------------
Thanks to the localhost ambivalence AND to relative path in http requests other
than MOVE and COPY (which are dav requests, and which are the SOLE requests
having a destination field, and where this field is implemented as an absolute
URL).

6/ NOTE about absolute url in MOVE/COPY destination field
----------------------------------------------------------------------
Some people think that the RFC4918 does not allow "relative url" in the
destination field : this is WRONG.
But it is a fact that most clients implement "destination" just as "absolute
urls": this is like it is.

Some people may think  "absolute url, well ok, so it implies that moving a doc
between two servers or two "services" on the same server should be possible!" :
yes BUT:
1/ the RFC does not say that it is mandatory ! and 
2/ Moreover "miracles" are just not possible if various restrictions disallow
this.
3/ Moreover requesting a "write access" on a location that is not in the "name
space" of the server currently servicing the request is just not logical, here
is why : 
if it were the case, it would mean that any server should act as a proxy to any
other : although proxy server exist and are described in the rfc, a http server
is not automatically a proxy server; proxy servers are a special family of
servers that require EXPLICIT configuration, NOT implicit configuration. So
"implicit proxying", although not explicitely forbidden by the RFC, is  a NON
SENSE vs the RFC. 
Further it must be noted that the proxying mechanisms described in the RFC2616
are NOT involved when performing HTTP-DAV over SSL ! because this is a
particular case of "proxying at tcp/ip level", not http level, no the RFC is
not relevant to solve this case: ie a server is not supposed to serve a request
in a situation not described in the RFC.


7/ BUT Why "rename is working with "add network place wizard", or some other
dav client:
--------------------------------------------------------------------------------------------------------
Because your dav client implements some tricks :
Some clients (such as Windows "Add Network Place" wizard) first try a true MOVE
or COPY method,
then they receive a 502 error ("Bad gateway") that I really think is a logical
reason for rejecting the request:
the reached host (127.0.0.1:443)  is NOT implicitely a PROXY to 127.0.0.1:80
(as implied by the destination url in the MOVE/COPY request, forged as such by
a client believing to act in a http/80 world), so it just CANNOT serve your
request.
So when copy/move fail, those clients (as "Add Network Place") try a GET (to
local hard drive) and PUT (to remote machine) :  well ok, it seems a good idea
but it is NOT because of network speed (often  low in upload) and delays in
case of big files.


IV/ SO what is the solution ?

Remember I already stated two workarounds :
1/ true SSL tunneling by using stunnel BOTH on the client AND the server, to
join a http client/80 and a http/80 server, and removing of SSL support in
Apache: 
to secure your server just use apache or firewall restrictions such as "port 80
only reachable from lan or loopback or localhost!"). 
If 80 is restricted to localhost, the stunnel server must be installed on the
http server, of course...

2/ On Vista, by directly using "net use https://", forgetting stunnel, but
provided that you disable "basic auth" at least at the root of the mounted
remote location.

But these are just workarounds.

**********

A solution based on stunnel client and apache classic conf directives exists:

Well you have understood that the problem comes from the fact that Apache DOES
NOT want to act AS A PROXY from "127.0.0.1:443 service" to "127.0.0.1:80"
service.
And I really think it is normal.

So you may have 2 solutions :
1/ by doing some "rewrite" magic in your 443 vhost:
1.1/ either rewrite the destination field as A RELATIVE url
1.2/ either rewrite the destination field as beginning with https://localhost
BUT ALL THIS WILL NOT be possible using standard apache directives because
apache rewrite engine does not give the ability to rewrite this field. Should
it be able to do that : not sure...and I do not request the apache team to do
that.

2/ the TRUE solution is here :
So you cannot rewrite the request, hmm, BUT you COULD REDIRECT IT to a service
that COULD understand it as it is (because in fact the request can be
considered as well-formed when received by an http server, although not by an
httpS server), by using one of the apache redirection and/or proxying features.
Well, in that particular case, redirect is NOT the good solution.
The solution is in "PROXYING" !


The PROXYING solution :
When you have finished this paper you will say "all this blah blah" for a few
lines of config...yes but it is necessary to really master apache conf, in
general and because this conf is also able to open big security holes.

"Forward proxy" is not the solution (you are lucky, I will NOT explain why).
The solution is in "reverse proxying".
See the apache doc about ProxyPass directive, as this is the basis for setting
up reverse proxy...

Anyway, If you roughly configure "ProxyPass" directives, it will work for
MOVE/COPY with stunnel and "net use" but will make FAIL MANY OTHER things, and
in particular for "true https clients talking directly to the 443 server":
because  if you have a WELL formed destination url beginning with HTTPS, a raw
proxying will send this to the "80 service" which  will REJECT IT !

SO the solution is to only (reverse) proxy requests that really need to be
rev-proxied !
For that the only solution is to use the " rewritecond...rewriterule ...[P]
flag" magics,
to ONLY rev-proxy <<MOVE and COPY methods arriving on 443 vhost AND having a
destination field beginning with "http://">>, which defines perfectly http
requests sent via a client SSL stunnel gateway.

Then here is the code to add this slective reverse proxying in the apache
configuration : 

# ================================================================
#                       IP BASED VIRTUAL HOSTS 
# ================================================================

<VirtualHost 0.0.0.0:80>
# ServerName (not used, this is a reminder related to usecanonicalname-on; we
prefer to use usecanonicalname Off!)
ServerName http://LAN-DNS-NAME:80
DocumentRoot "/var/www/html"
</VirtualHost>


<VirtualHost 0.0.0.0:443>

# ServerName useless but it is a reminder (see recommendation in ServerName
apache doc however fort https vhost)
ServerName https://WAN-DNS-NAME:443
DocumentRoot "/var/www/html"

# SSL configuration: PUT WHAT YOU WANT HERE provided that you know what you are
doing !
# -----------------

SSLEngine on
SSLOptions +StrictRequire

#   SSL Cipher Suite:
SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW

#   SSL Protocol support:
SSLProtocol all -SSLv2

# I only accept users from my CA 
SSLCACertificateFile /etc/pki/tls/cacert.pem

# CRL check
SSLCARevocationFile /etc/pki/tls/cacrl.pem

# Server Certificate Chain File
SSLCertificateChainFile /etc/pki/tls/certs/localhost-chain.crt

#   Server Certificate:
SSLCertificateFile /etc/pki/tls/certs/localhost.crt

#   Server Private Key:
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

# VERY IMPORTANT 

SSLVerifyClient require

SSLVerifyDepth 1

<Location />
SSLRequireSSL
</Location>

# REVERSE PROXYING : solution to "WEBDAV over STUNNEL and RENAME issue", the
solution starts here
#
----------------------------------------------------------------------------------------------------------------------------

RewriteEngine On

# Some env vars useful later

RewriteRule "(.*)" - [env=V443_URL:$1]

RewriteCond %{SSL:SSL_CLIENT_S_DN} "^(.*)$" [NC]
RewriteRule "(.*)" - [env=MY_SSL_CLIENT:%1]

# REVERSE PROXYING : solution to "WEBDAV over STUNNEL and RENAME issue"
#
-----------------------------------------------------------------------------------------
# This is also the solution to problem in renaming files on a subversion server
accessed through dav.

RewriteCond %{REQUEST_METHOD} (MOVE|COPY) [NC]
RewriteCond %{HTTP:Destination} "^(http://.*)$" [NC]
RewriteRule ^/(.*)$ http://127.0.0.1:80/$1 [proxy]
# Note : [proxy] can be abreviated as [P]
# RAF : b flag not supported ? (cf unescape)

# SOME security on the proxy: DO NOT HESITATE TO HARDEN THIS more  than I did
<Proxy http://127.0.0.1:80/>
Order allow,deny
Allow from env=MY_SSL_CLIENT
</Proxy>

</VirtualHost>

# Some useful logs
CustomLog logs/mylogs.log "SSL_DN is  %{MY_SSL_CLIENT}e"
CustomLog logs/mylogs.log "V443URL is  %{V443_URL}e"

V/ Miscellaneous comments
--------------------------------------

1/ Some people have the same kind of problem:
- when using dav access to subversion location,
- with various dav clients (cadaver, goliath)
- when dealing with error 502 and MOVE/COPY methods
All this problems have the same cause and solution described above : reverse
proxying config on your https server.

2/ a little note about www access (not dav):
Be careful to only have relative url in your various "href" properties in your
links, and "usecanonicalname off" on apache, in order to have both http and
https access working on the same directories.

3/ General note
Think also to have SSLRequireSSL directive in apache conf: do not think to
solve the "move/rename/copy" problem with dav over stunnel by removing this
important directive:
 you will only open security holes.

4/ Some people have proposed some similar solutions either by rewriting or by
proxying, but these solutions usually propose to write some kind of Apache
module or filters, and in fact circumvent apache security by activating a kind
of "implicit proxying" which is really a bad idea.

5/ use apache logs (and particularly logs of envvars) and wireshark
(ex-ethereal) to diagnose your dav-client behavior: but remember that you can
only sniff clear traffic SO it in only possible on the client BUT unfortunately
it is NOT possible to sniff the loopback interface on a XP/Vista machine : the
trick is to sniff dav traffic on a clear local lan without SSL support, to
understand how your client manage various http methods, then to extrapolate
when this traffic is transferred through ssl.
In pure ssl traffic, apache logs can be enriched to track the "Destination
field", with  an envvar extracted by the headers through some rewriterule.


VI/ Some useful links
------------------------.
Good discussion about dav+https (follow the 2 links at the bottom of the page:
one to simple-groupware CMS, the other to "smallvoid" nett use guide)
http://devwfb.blogspot.com/2008/07/how-to-map-https-only-web-folder-to.html
http://www.simple-groupware.de/cms/WebDAV
http://smallvoid.com/article/winnt-webdav-network-drive.html


DAV and SUBVERSION:
http://www.phwinfo.com/forum/alt-apache-configuration/351885-webdav-over-ssl-move-copy.html
http://sethd.org/Site/SubversionProxyFix.html

Good info about webdav implementation by MS:
http://greenbytes.de/tech/webdav/webdav-redirector-list.html

an howto to setup dav+stunnel
http://www.autohotkey.com/forum/topic44471.html

some other howto:
http://librenix.com/?page=SSL
http://www.howtoforge.com/webdav_with_ssl_and_two_factor_authentication

Apache "bug" that I recommend NOT to fix :
https://issues.apache.org/bugzilla/show_bug.cgi?id=45449

==============

Yours sincerely,
Pierre Delaage

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


Mime
View raw message