Hello again, 

I have made a few steps forward and a few back.

First I decided to move from ATS 5.3 (latest rpm available on RHEL by default) to 7.0.0 (latest RHEL rpm build I could find). I did this becuase I found a few settings that led me to beleive this version was more CONNECT method aware (proxy.config.http.forward_connect_method, proxy.local.http.parent_proxy.disable_connect_tunneling, etc).

I hit a snag here though. HTTP works fine but HTTPS reports an Unknown Host error due to DNS lookup failure of the https host. The issue is I expect the DNS lookup that ATS performs to fail and have set 'no_dns_just_forward_to_parent' to 1.

I have enabled debugging and it looks like the requests are both mapping correctly. Looking at the logs and GitHub source - I can see where the request processing diverges:

HTTP Request:

    <HttpTransact.cc:1812 (HandleRequestAuthorized)> (http_trans) Next action SM_ACTION_API_OS_DNS; HttpTransact::DecideCacheLookup

HTTPS Request:

    <HttpTransact.cc:1350 (HandleRequest)> (http_trans) Next action SM_ACTION_DNS_LOOKUP; HttpTransact::OSDNLookup

... looking in GitHub I can see the following line decides which code path is taken: 


This made me think maybe how I'm defining parent hosts wasn't causing the parent proxy to match to https requests. Playing with the parent hosts settings didn't seem to change anything - I have tried the following:

    dest_domain=. parent=<corp proxy host>:<port>
    dest_domain=. parent=<corp proxy host>:<port> go_direct=false
    dest_domain=. scheme=https parent=<corp proxy host>:<port>
    dest_host=registry.npmjs.org scheme=https parent=<corp proxy host>:<port>

If I can overcome the problem above then I can try another PCAP to see what the parent proxy is sent.

Finally, I'm not sure ATS supports initiating a HTTP CONNECT tunnel to a parent proxy, only that ATS can be the proxy that does the tunneling.

Any suggestions you might have will be very helpful! I'm a bit stuck again now.

Thanks again,

On 21 March 2017 at 17:31, Jim Groffen <jim.groffen@gmail.com> wrote:
Thank you for this response Jeremy, I've only now had a chance to try your suggestions.

As you suspect the corporate proxy does not support TLS Termination but does support HTTP CONNECT Tunnelling.

My existing Java based proxy can connect to https://registry.npmjs.org and is using HTTP CONNECT tunelling. I can see the TLS handshakes etc in the PCAP now.

I haven't worked out what ATS is attempting to do yet - I'm having unrelated issues getting to my test machines. Once I've got a handle on that I'll send another update.

Thanks again for such a fast response, and apologies for my tardy reply!

On 17 March 2017 at 00:01, Jeremy Payne <jp557198@gmail.com> wrote:

can your corporate proxy terminate requests for  https://registry.npmjs.org ?
usually corp proxies do not unless they support something like squid's ssl bump/peak/splice... 
if not, then ATS will have to send a CONNECT request to your corp proxy(not sure if ATS supports issuing CONNECT to upstream servers).. 

try the below curl commands to test your corp proxy.

TLS termination..

curl -k -o /dev/null -v --resolve registry.npmjs.org:443:<CORP PROXY IP> https://registry.npmjs.org


curl -o /dev/null -v -x <CORP PROXY IP>:<PROXY PORT> https://registry.npmjs.org/

does your existing java based proxy support requests for  https://registry.npmjs.org ?

btw.. you can still inspect the TLS handshake within a pcap. you should also be able to
view the TLS alert issued by either side within the same pcap. 
you can also try placing ATS into debug mode to see whats going on.

On Wed, Mar 15, 2017 at 11:47 PM, Jim Groffen <jim.groffen@gmail.com> wrote:


I have written a simple proxy in Java to forward requests through the corporate proxy and access an external site using HTTPS. There is no internet access unless we go via this corporate proxy.

I'm now trying to replace my simple proxy with TrafficServer. My setup is as follows:

User -> TrafficServer -> Corporate Proxy -> External Site

I have configured two remap rules (using the NPMJS Registry for testing):

  regex_map http://(.*):8080/npm http://registry.npmjs.org:80/
  regex_map http://(.*):8080/snpm https://registry.npmjs.org:443/

I also have configured the corporate proxy as a parent proxy:

  dest_domain=. parent=[corporate proxy ip]:[port]

With this configuration I can hit the /npm path and traffic comes through correctly, but when I try the /snpm path I get a "Could Not Connect" error.

Monitoring the traffic shows that some tcp packets are sent between the TrafficServer and the corporate proxy, then the corporate proxy returns an HTTP 400 response.

Monitoring the Java based simple proxy for comparison, I see the TCP traffic, then a valid HTTP 200 response.

I've tried a few things around the SSL settings like relaxing the allowed ciphers, but it's hard to know what's wrong because the TCP traffic is likely the encrypted SSL traffic! One interesting point though, I can see registry.npmjs.org when using TrafficServer - I assume this is due to SNI (Server Name Indication) - but my Java based proxy doesn't do this. This made me wonder if SNI isn't supported on my corporate proxy.

Another possible wrinkle is I need to auth to the Corporate Proxy - I've enabled proxy.config.http.forward.proxy_auth_to_parent and am using the headers plugin to provide valid login details - which is all working for non-https traffic.

Thank you in advance, any help would be greatly appreciated!