Stork 2.0: Open Source DHCP Management Tool
ISC launched the Stork management application almost exactly five years ago, in November 2019, as an experimental tool.
Read post
We at ISC want to encourage networking people around the Internet to focus attention for a few minutes on an obscure topic, EDNS compliance.
The EDNS (Extension mechanisms for DNS) protocol allows us to add new features to DNS that were not envisioned when DNS was originally specified. RFC1035 fixed the size of the DNS message and its components back in 1987. Since then, there have been a number of very good reasons for extending the DNS protocol suite. Support for EDNS is technically optional, but as the applications relying on EDNS become more critical, widespread support for EDNS is increasingly important. That application designers and network operators cannot yet rely on EDNS working end to end is becoming a critical problem for the Internet.
How widespread is EDNS support today?
EDNS is currently supported on better than 90% of all DNS servers ISC surveyed recently (research report). EDNS was designed to allow further extensions to be easily deployed. If only one end of a connection is EDNS-aware, the extensions don’t work, but the basic DNS protocol should be unaffected. EDNS specifies how to do EDNS version negotiation and how to handle unknown EDNS options and unknown EDNS flags (by ignoring unknown EDNS options and flags). This has enabled us build support for EDNS across the Internet gradually without disrupting the DNS.
One of the most widely-deployed features that relies on this mechanism is support for the DNS Security, or DNSSEC extension. Systems that support DNSSEC validation can positively determine that the DNS data they are sending to the client was ‘signed’ by the owner of the zone. The demand for DNSSEC validation has made EDNS support much more important and probably helped drive adoption. However, the reliance on DNSSEC, particularly among those segments of the market that have adopted it broadly, including .gov and the country-code top level domains, also makes it even more important that EDNS works reliably.
This all sounds like success! What’s the problem?
The percentage of DNS servers on the Internet that support EDNS drops significantly from 90% with some support, to 60 - 85% when you look at full compliance. The percentage of systems with compliance problems varies among different types of domains. Top level domains, such as countries, and the DNS root, have the best compliance at approximately 85%. This is probably because they are required by ICANN to support DNSSEC. (The requirement is being phased in as cc-TLDs renew their agreements with ICANN, so it is not yet required of 100% of the TLDs.) The Alexa .gov servers, which also have a high DNSSEC deployment rate, have approximately 70% full EDNS compliance. The Alexa top and bottom 1000 servers scored the lowest, with only about 60% of servers passing our EDNS compliance tests.
Partial EDNS support can cause big problems
We have identified a number of different problems amongst the EDNS servers with partial support. For more details, please read the most recent compliance report. We saw a range of compliance problems, including:
- OPT only returned when DO=1 is present in the request
- BADVER not returned to EDNS (1)
- NOTIMP returned when a EDNS option is present
- FORMERR returned when a EDNS option is present
- BADVERS returned when a EDNS option is present
- NOTIMP returned when a EDNS Z flag is present
- FORMERR returned when a EDNS Z flag is present
- BADVERS returned when a EDNS Z flag is present
- EDNS option echoed back
- OPT not returned in truncated response
- EDNS (1) queries being dropped
- EDNS queries with a Z bit being dropped
- EDNS Z bits in queries echoed back
- TCP response size limited to EDNS UDP response size
- Truncated UDP response when send when response will not fit
- Fragmented responses being blocked
- DO=1 not returned by DNSSEC aware servers
Systems that nominally support EDNS sometimes do not recognize that EDNS is supposed to support multiple extensions to the DNS and fail when they see any new, unknown extensions. These ‘broken’ DNS servers do not handle unknown EDNS versions, flags and options correctly, returning unexpected result codes and option values. This triggers the systems querying them to retry by sending a plain (non-EDNS) message, effectively blocking the use of any EDNS application.
As we add more applications that rely on EDNS, partial compliance can end up resulting in failures with increasingly significant impact. Partial EDNS compliance is particularly a problem for systems that support the DNSSEC extension. If a server that has a signed zone is misclassified as not supporting EDNS the resolver will not get the DNSSEC records required to validate the answers from the secure zone. This will have the result that a client of a validating resolver may no longer be able to access some network resources. We don’t want to ’trigger’ a situation that causes DNSSEC-signed properties to become unreachable.
DNS Cookies present a new opportunity in the war against DDOS
Here we come to the current problem. We need to deploy another important EDNS application.
Recently, we have rediscovered another important application for EDNS, DNS cookies. DNS cookies, like web site cookies, enable two machines to recognize each other after an initial communication. Machines perpetrating abuse and attacks on the Internet will not exchange cookies and will not want to be identified and tracked. Using cookies to quickly identify non-abusive communications will enable DNS systems to identify suspected abuse traffic and defend against attacks much more efficiently. DNS cookies are not a new idea, the original proposal was made in 2006, but the IETF has only recently revived the draft and the current version is now approaching standardization. Because denial of service attacks are becoming an increasingly significant problem in the DNS, we want to enable this technique as soon as practically possible. ISC began shipping an experimental implementation of DNS cookies with BIND 9.10.0 in March of 2014.
The EDNS Cookies application is less forgiving of EDNS errors. Most current use of additional EDNS extensions is between consenting parties (NSID, EXPIRE). DNS COOKIES is a exception to the current usage patterns. A DNS COOKIES option is expected to be added to every QUERY providing additional DNS spoofing resistance to both the server and the client when the server supports DNS COOKIES.
What does this mean?
We cannot deploy DNS COOKIES today without anticipating a negative impact on DNS query success, because of the substantial proportion of DNS servers lacking full EDNS compliance. Based on ISC’s research we anticipate the failures will go beyond losing the value of the DNS cookies application, to include breakage of existing, working DNSSEC support. Those domains that have taken care to protect their DNS with DNSSEC will be most adversely impacted if their EDNS support is not fully compliant with the specifications for handling unknown options.
What can you do?
The first thing you can do is test the support for EDNS in your own DNS system. ISC is hosting an easy self-test tool on the web that queries your site and gives you a report in seconds. If you see a problem, it could be due to either your network infrastructure, or your DNS server itself.
Some firewall vendors and anti DoS vendors have default rules that see requests with any of these extension mechanism in use as a reason to drop the query defeating the intent of defining how to handle requests using these extension mechanisms. Timeouts are usually due to a misconfigured firewall though there is a chance that the DNS server. itself could be timing out. You may need to update or reconfigure your network infrastructure to allow EDNS to pass unobstructed.
All other errors (wrong rcodes, lack of OPT record, content incorrectly echoed) are problems with the DNS server. If your DNS server is deficient, contact your vendor for an upgrade. The EDNS protocol was standardized 16 years ago: full support should be available if you ask for it.
References
EDNS Compliance - presentation by Mark Andrews at IETF 92 in Dallas, March 24, 2015
EDNS self-test tool - enter your zone name and find out how good your EDNS compliance is
DNS Cookies current draft - pre-standard
