Scott Forsyth's Blog

Postings on IIS, ASP.NET, SQL Server, Webfarms and general system admin.

Cloud Resources

IIS Resources

Windows Server 2008 R2 DNS Issues

I recently upgraded my home Windows Server 2008 Domain Controller to R2.  The upgrade process itself wasn’t too much work but was a bit more than ‘next, next, finish’ because the AD schema needed to be updated and the installer required that WSUS be uninstalled first.  But, those weren’t a big deal.

However, after the install, I got the strangest behavior.  Visiting some websites like www.microsoft.com, www.bing.com, www.windowsupdate.com and a number of other Microsoft websites didn’t work.  However, other websites worked perfectly.  In fact, www.google.com still worked.  It’s almost as if Microsoft decided they didn’t want to grow their search engine market share anymore and would start blocking their visitors. :)

What made it even more confusing was that if I viewed the errors in my browser, it timed out and gave a DNS error. However, if I pinged the DNS name, it worked. 

(feel free to skip to the bottom for the fix if you don’t want to read the details)

I did some searching and didn’t find an answer (although now that I know what search terms to look for, I see that others have run into this now).  I tried all the basic troubleshooting methods to no avail.

I skimmed some R2 release notes I found and I saw that there were EDns (EDNS0) changes with R2 but it was pretty vague.  EDns is a relatively new DNS protocol extension that is still coming of age.  Later I realized that I was on to something here.

I realized that I would need to fire up Network Monitor to get the story.  After running Network Monitor, an issue was immediately apparent as seen from the following screen shot snippet:

image

First, I wondered why my search for bing.com returned search.ms.com.edgesuite.net.  The answer to that wasn’t hard to find.  Those are the DNS names of the Akamai CDN which Microsoft uses for a lot of their sites.  The real issue there is the “Response – Format error”. 

I looked at the request and the results for a while and it seemed straight forward, so I did a network trace on a working server and found that R2 added some extra information.  Notice the bottom line of the following image with the “AdditionalRecord:  of type OPT on class Unknown DNSClass”.  The network trace on the working server didn’t have that.

image

So, I knew at this point that R2 was adding something that the Akamai DNS servers didn’t like.  I did a search for OPT and discovered that OPT is used in EDns.  I found a registry setting called EnableEDNSProbes which disables EDNS when set to 0.  After setting that and restarting the DNS Server service, everything worked perfectly.  I set it back again and it stopped working, so I knew I had narrowed it down.

While searching for information on EDns, I discovered that some DNS servers will attempt to make a EDNS probe, and if it fails then it will try again with a plain query.  That allows it to always work regardless of the support of the other DNS servers.  However, after testing I found that Microsoft DNS doesn’t do that.  EDNS can either be ‘on’ or ‘off’.  Bummer, I thought that was a good idea.

Testing further I discovered that it’s not enabled by default on Windows Server 2008 RTM.  I tried on another R2 server that wasn’t in production yet and confirmed that the issue appeared there too.  So, the issue wasn’t that something changed with EDns, it’s simply that it was enabled in R2 for the first time.

The reason that it failed in the web browser but worked with a ping is because the browser followed a redirect and failed on the redirected address and not the original address.  The ping didn’t follow the redirect so the failure never occurred.

It appears that the same issue occured when Windows Server 2003 was released: http://support.microsoft.com/kb/832223.  I don't remember that occuring and being a big deal so I suspect that Microsoft must have made changes to the default with later service packs or hot fixes.

Conclusion

It appears that the Internet isn’t fully up to date and ready to use EDns quite yet.  The solution for this is to disable EDns and wait another year or two until Akamai and other DNS servers catch up, or Microsoft releases a hot fix to support the failback option I mentioned above.

Note that this isn’t a problem for most Windows Server 2008 R2 member servers.  It’s only a problem for DNS *servers* that do recursive lookups.  i.e. likely only your domain controller will be affected if that is where your DNS Server role exists.

Fix

To disable EDns, you can do it from the command prompt, or by editing the registry.

From the command prompt, no restart of DNS is required.  If from the registry, make sure to restart the DNS Server service.

Command prompt: 

dnscmd /config /EnableEDNSProbes 0

No restart is needed.  It takes effect immediately.

or Registry: </>

Create a DWORD called EnableEDNSProbes and set to 0 in HKLM\SYSTEM\CurrentControlSet\services\DNS\Parameters

Restart the DNS Server service for it to take effect.

Comments

Mike said:

Thanks for this.  This has been driving me crazy with my 2008 R2 testing.  It's ironic that microsoft.com can't handle EDns.

# September 18, 2009 1:06 PM

Mike T. said:

Thank you for this, I had just started to go into the troubleshooting process in test my when I stumbled upon your blog. You probably saved me at least an hour of irritation.

# September 23, 2009 10:44 PM

Kenny Palmer said:

Thanks for this, i was pulling my hair out with this issue. good old MS. Very nice work getting to the bottom of this. My compliments.

# September 24, 2009 3:10 PM

James said:

Thanks for this! It has been driving me crazy for 2 weeks! R2 unable to hit Microsoft sites. how funny

# September 29, 2009 6:13 PM

Randy Baker said:

Thanks for this information...just got r2 up and running as a domain controller and dns server and this was driving me nuts.  You saved me a lot of time and headaches.  Thanx

# September 30, 2009 1:37 PM

Jeff said:

Wow thanks a lot. I have been going crazy with my testing at home. I upgraded, well rebuilt my entire home network from scratch and was having DNS issue like mad. I thought maybe it had something to do with Hyper V as I was running the AD controller virtualized. Finally in a bid to just get the thing to work. I installed AD, DNS, and DHCP on the host machine and left Hyper V uninstalled. Still didnt work so I put in a forwarder to 4.2.2.2 and that fixed the issue. Not what I wanted to do as I prefer root hints.

Ill try this out tonight and hope it fixes the issue. I also found it weird microsofts sites werent coming up. I figured something under the hood was changed in R2. Glad to find your site!!!!!

# October 5, 2009 2:21 PM

John said:

For the record it is Akamai that is non-compliant with EDNS, not W2K8 R2 DNS.

# October 8, 2009 3:32 PM

OWScott said:

Thanks John.  You're right, that's my intent.  Thanks for clarifying.

# October 8, 2009 4:24 PM

OWScott said:

Steve, good call.  You've pointed out the official specs for the fallback option.  It sounds like it's not just a 'nice have' to fall back, but it's a 'supposed to have'.  

So Akamai DNS needs to update to the latest, but Microsoft DNS Server needs to play nice with properly functioning old protocols.

# October 13, 2009 9:30 AM

Brian said:

Thanks for the info.  I've spent half a day trying to get DNS to work before I found your article!!!

# October 16, 2009 6:00 AM

AJ said:

I was just beating my head against the desktop. Until I saw this applied it and it WORKS!!! Why in the name of all thats holy was this not set to 0 as default.

# October 28, 2009 3:26 PM

Todd said:

Another area to check:  Cisco firewalls may have a feature called DNS Guard implemented.  Out of the box, it has a DNS message ceiling of 512 bytes.  So your firewall can also be trying to protect you, but blocking the ability to communicate through eDNS methods.  We were getting time outs with DNS Guard and eDNS implemented concurrently.

# October 30, 2009 11:36 AM

OWScott said:

Hi Todd, thanks for mentioning.  I considered that.  I have 2 enviornments that I tested in, one with Cisco equipment and one with home linksys equipment.  In both enviornments, the Akamai queries failed while other queries worked, so, based on your comments, it sounds like Cisco is something to watch for and may have come into play with some requests (I didn't test enough to see when), but the eDNS is the most significient player in my situation.

# November 1, 2009 6:19 PM

Patrick Stalteri said:

Thank you so much. This was absolutely driving me crazy. everything was working fine I finish upgrading and no internet although the server said it had internet connection. I put name forwarders in and got it up and running. now I can finish the project for the client.

Thanks again

# November 17, 2009 3:55 PM

Reuven said:

Thanks you save me time, i think after restart i need to run the command in order to disable EDNSP

# November 29, 2009 10:35 PM

Andrew said:

ABSOLUTE GENIUS.......We have had this problem for nearly 2 weeks. Couldn't understand why, when we changed our DNS Forwarders to OpenDNS IP's we could get to www.aol.com and www.microsoft.com, but when using the ISP's DNS Forwarders, we couldn't.

Very Happy Now :-)

# February 12, 2011 10:47 AM

Ivan said:

Our DNS issue was that root hints wouldn't work but forwarder approach is OK. Very confusing, the same thing after OS rebuilt. It turned out this was causing the problem. It is working now.

Thanks so much!

# February 13, 2011 12:34 AM

Frank said:

Thank You!

Spent a day struggling with this.  I don't understand how a product can be released like this.  MS never ceases to amaze me.

Did I say, Thank You!

# February 15, 2011 1:25 PM

Mark said:

Thank you. I too spent alot of time troubleshooting this issue. I am glad i found your article.

# February 22, 2011 7:32 AM

jeffery said:

I too am grateful for this post as my issue was with trouble access shaw websites and email.  Running the mentioned cmd code did the trick.  Unfortunately, it is not appearing to be stable on my system and I have to restart my dns service on the server every day or so.  Any thoughts?

Using external DNS bypasses the problem but would rather use internal and have it not glitch (had no issues with previous server running windows SBS 2003).

# March 3, 2011 5:21 PM

luspin.jackson@yahoo.com said:

Very nice post, Most new businesses make the mistake of thinking that hosting their own server will be a simple process that is easily manageable.

# March 25, 2011 6:49 AM

OWScott said:

Thanks Luspin.  I agree, at first glance it seems easy enough to host all aspects of a server and sites, but to do it properly there are a slew of big and little gotchas and technologies to consider.  

# March 28, 2011 11:02 AM

Stephen Shoup said:

YOU RAWK!! This elminated all of my issues.

# May 27, 2011 1:34 AM

Oli said:

Such a simple thing, took me hours... But this is where I found the solution!

So many thanks!!

# June 4, 2011 1:20 AM

Dan said:

This worked for us! Nice job man!

# June 15, 2011 1:04 PM

binu said:

Thank you very much for this article.  It was driving me insance.

# August 1, 2011 1:12 PM

Erik Finn Rasmussen said:

You saved my day. Thank You!

# August 12, 2011 3:40 PM

MCP said:

Thank you so much man.

# October 12, 2011 3:45 PM

ramzi said:

Ta awfully! That saved me really........

# January 11, 2012 7:12 AM

Robert Hines said:

Thank you very much for this.  It worked perfectly to configure my 2008 AD-DNS server to resolve addresses using the installed Root Hints.

# January 19, 2012 11:17 AM

Graham said:

Man this has been wrecking my head and as really put my work back.

Have made the registry changes many thanks for all your hard work

# February 13, 2012 9:55 AM

Janderson Mira said:

Thank you so much man, we were using scripts to restart DNS some times during the day, now it's working perfect. Saved us, thanks from Sao Paulo, Brazil.

# February 14, 2012 4:07 PM

Paulofonta said:

Thank you!

I was having this issue ever since I first tried ws2k8r2

Today I decided to make a search on it, and finally fixed it, after finding this thread ;)

# February 21, 2012 11:55 AM

George said:

thanks - solved a problem on a SBS 2011 from DELL. 4 days looking at this. not kidding on the 4 days.

# February 21, 2012 11:46 PM

Robert said:

Thanks for the solution!

# March 10, 2012 2:24 PM

Jason Merritt said:

You are an absolute legend,

I have been trying to fix a Windows Activiation issues for hours, narrowed it down to DNS but couldn't figure out why lookups of www.microsoft.com were failing.

Thanks this fixed my issues straight away.

# March 22, 2012 11:42 PM

OWScott said:

@Mich. Here's a quick read on EDNS: en.wikipedia.org/.../EDNS

# May 31, 2012 3:14 PM

DevL said:

...thanks, this fix solved part of the problem. I have 9 staff computers connected to server 2008 r2. On the staff computers the internet monitor tray icon indicates that internet is available but when the staff tries to access the internet there is no connection. Report: ERROR! DNS Server not available ...etc. The only fix I find is to reboot the router. How do I fix this...

# July 6, 2012 4:05 PM

OWScott said:

@DevL, sorry for the delay as I was traveling. That issue may be different than the DNS issue in this article.

What I suggest to troubleshoot it is to try to narrow down the issue when it occurs.  Some ideas:

- run an ip config and confirm the IP and gateway

- ping the gateway and ensure that it works

- assuming that it does, ping a variety of DNS names and see if they work (www.google.com, www.microsoft.com, www.zdnet.com, www.stackoverflow.com, etc) www.microsoft.com won't return a ping because of show it's setup, but it should resolve the IP address.

- when you have the issue try restarting the DNS Client service from services.msc.  That will give hints on whether the issue is related to the computer or the router.  

- also, when the issue occurs see how widespread the issue is to help narrow it down.

# July 16, 2012 6:52 PM

Alan B said:

Thanks, been driving me crazy for a few months now, glad to resolve.

Funny thing is, some of the articles I read said "use Hotfix xxxxx"... well duh, I can't download them as I can't get to microsoft.com LOL!

# October 2, 2012 7:15 PM

OWScott said:

Alan B, don't they call that a Catch 22? :)

# October 6, 2012 12:18 AM

Bower said:

Thanks for the info, I don't normally use my 2008 DC for external dns but I needed it to work for my new remote access system and your tip worked like a dream

# October 15, 2012 5:03 AM

Jeff Graves said:

Ran into this using Windows Home Server 2011 as well (which is based on 2008 R2). Same command fixed it for me (after I cleared the DNS cache).

# October 23, 2012 6:22 PM

1-Nul said:

Thanks thanks thanks!

Could not connect to domaincontroller on different site which was connected through VPN tunnel/firewall.

Resolving the intranet subdomain corp. kept failing by returning the IP for the parent internet domain.

Setting a conditional forwarder did not help while the target DNS server was behind that firewall.

Setting the parameter EnableEDNSProbes on all DNS servers (Win2008R2) on all sites solved the problem.

Note: DFS is also depending on DNS, not only internet will fail.

Best of luck!

# November 12, 2012 2:19 PM

Mark Ward said:

Yep you definitly are the man

# January 17, 2013 12:41 PM

Eki said:

I have a DNS related problem when i try to issue certificates:

DNS name does not exist.

Denied by policy module.

DNS server is on the domain controller where the certificate server is also.

Are you able to help? I would very much appreciate it.

# January 31, 2013 1:30 PM

OWScott said:

Hi Eki,

Are your server TCP/IP settings set to use 127.0.0.1 for the DNS server? It's possible for the record to be on the server but for the client role to not know about the server.

# February 4, 2013 10:53 AM

Pavan said:

Hi Scott Forsyth,

Gud Post and it is easy to solve the DNS problem i followed your solution and it gave me correct result.

And u save my time by googling for DNS Error.

Thks

# April 6, 2013 1:55 PM

BillD said:

Our Windows clients send request to our DNS servers (which are our DCs as well). DNS server gets request, has no record so forwards request up to the DNS server at the Forest level. It responds, our DNS server sends the reply back to the requesting Windows 7 client. This all takes place in 1 second, 2 seconds at the most, but typically the whole exchange is 1 second or under. The Windows 7 client logs a DNS error as such:

Name resolution for the name www.solarwinds.com timed out after the DNS server 1xx.2.19x.xxx:53 did not respond.

But the DNS server DID respond! This log entry was made 1 second after this computer sent the first request. So is it that it DOES get a response, but not the response it wanted??

# April 10, 2013 2:03 PM

OWScott said:

Hi BillD,

Does this happen every time or just sometimes? I wonder if the DNS timeout is set to 2 seconds, causing it to fail sometimes.

You may need to use WireShark to see what happens at the network traffic level on the client machine, to find out at which point the response is dropped.

# April 11, 2013 9:59 AM

Allan Wright said:

Top Man!   This saved our guys some head scratching too for some really obscure sites that just would not load, or some of our network traffic just being sluggish overall.  All fixed with one DWORD.  Happy Days!

# July 18, 2013 6:14 AM

Paul Cookson said:

That's a great help.  Thanks very much.

Paul

# September 23, 2013 9:24 AM

Fredrik Linnander said:

Did not solve my problem, turns out, my Dlink DSR-1000N firewall had a UDP flood feature, which gave the same problem/behaviour.

# November 14, 2013 5:01 AM

OWScott said:

Good to know Fedrik. Thanks for mentioning.

# November 19, 2013 7:25 PM
Leave a Comment

(required) 

(required) 

(optional)

(required)