Inconsistant dns resolution when using setDNSName

[abarre]

hello,

In our private instance using EC2 agents, we regularly see tests that are not performed on the right IP address.
To verify the performance on different targets, we use setDnsName instruction in a script block.
We currently have two kind of instruction :

Target the original address (and performa a DNS resolution)

setDnsName	www.go-sport.com	www.go-sport.com

This instruction may looks weird but it permit to remove the DNS resolution delay in the waterfall.

Target another location

setDnsName	www.go-sport.com	www.go-sport2.com

We often see that the test with the instruction setDnsName www.go-sport.com www.go-sport.com will resolve with the IP of www.go-sport2.com.
I suspect that in this case, a DNS cache is maintained in the WPT agent and a second test on the WPT agent will reuse a DNS resolution done on a previous test.

[pmeenan]

Can you reproduce it with the public instance? It shouldn't be a cache because setDnsName uses 2 methods to override the address (at the time of the test).

1 - For Chrome it adds a MAP host rule to the command line for the name, mapping it to the resolved IP address.

2 - For all browsers, it adds a hosts file entry for the name pointing to the resolved IP address (and then restores the hosts file change after the test).

The agent does an OS DNS lookup for the target name when it gets the job (before the hosts file change is made). I can add logging to have it log the results of those lookups if it would help for collecting debug logs.

If for some reason the hosts file doesn't get cleared correctly then I could see the agent DNS lookup returning the wrong address before the mapping happens.

[tkadlec]

@abarre Following up here to see if you were you able to reproduce on the public instance?

[abarre]

Hi, I didn't investigate this issue further. At Fasterize, we work around this issue by performing a DNS resolution in a wrapper and then directly call setDns with the IP via the API.

We are currently using the official webpagetest agents via an AMI so this issue is likely reproducable in the public instance.