Codenomicon Testing for BIND

Our users value stability and security above anything else, when it comes to BIND. Every time we have to issue a security advisory we are inconveniencing thousands of network administrators. We also know they would rather be informed if there is some way to compromise or crash BIND. So, when we read that Codenomicon discovered the Heartbleed bug in OpenSSL, we decided to contact Codenomicon directly and learn if they would scan BIND as well. I was a bit worried that BIND might be next in the news, if we had a hidden serious defect; we decided to find out.

ISC takes full advantage of the Coverity Scan program for Open Source. We schedule our three major products (BIND, ISC DHCP, and Kea) for frequent analysis with Coverity, and try to stay on top of any issues that are flagged. We very much appreciate the test coverage we get from that. (You can see our Coverity status on the badges on the BIND and DHCP software pages on this website.)

Scanning with Codenomicon is complementary to Coverity. Codenomicon develops software that stress tests networking products by throwing unexpected or malformed packets at the product. It is a specialized form of negative testing that can be extremely successful at uncovering serious bugs that are latent in your software. It is less well-known in the open source community that Codenomicon also offers free scans to selected open source codebases through their Codenomicon Robust Open Source Software program (CROSS).

We wholeheartedly recommend the CROSS program to other open source developers. Our experience with the CROSS program has been completely positive. First of all, there was a minimum of fuss, delay, or administrative overhead. Within a couple of days after I contacted them, BIND was accepted into the program; a Codenomicon engineer downloaded the latest version of our source code, compiled it, and analyzed it to decide which protocol suites to run against it. They set up an account to monitor the progress of the scan. Ten days after I contacted them, BIND had already passed over 4 million test cases.

Unfortunately, after about 10 days, they hit a failure. First, they found a crash in dig. When we analyzed this, we realized the same code was also in BIND 9. Then, they found the same crash again in BIND. By then, we were already working on a patch. We posted CVE-2014-3859 on June 11th, “A query specially crafted to exploit a defect in EDNS option processing can cause named to terminate with an assertion failure” and issued BIND version 9.10.0-P2.

When I first contacted Codenomicon I was concerned, because of the publicity surrounding Heartbleed, that they might not notify ISC first if they found a defect, and allow us to manage the disclosure. I didn’t need to worry. Once the Codenomicon scan found a critical defect, they offered to help us with diagnosis, kept scanning, AND didn’t tell anyone else. What more could you ask for? We were able to follow an orderly phased disclosure process, issuing a security advisory and a patch, without any leaks or pressure from Codenomicon. Considering the PR value to them of publicizing a significant BIND vulnerability immediately following the Heartbleed bug, we were much relieved to learn their process and business culture is world-class. We highly recommend Codenomicon. If you are an open source developer, consider applying for the free CROSS program for open source.