It’s not been a good week for auDA, the .au policy and regulatory body, and their backend registry provider Afilias. First on Tuesday there was a security incident that auDA claims saw “a small number of domains” disappear for half an hour. Then today with the launch of second level (or .au direct) registrations, there has been another stuff up that sees all new second level/direct registrations having to be manually entered after registration with no timeframe given for a resolution.
In the security incident earlier this week, auDA went sadly on the back foot in an attempt to minimise what effectively was a total .au outage, claiming that only a “small number of domains”, 15,000 according to an article published in The Guardian, out of 3.4 million were impacted. This hardly seems like a transparent and mature approach to managing a critical infrastructure asset for the Australian government, business and people. Old habits do die hard!
The truth is that the root name server trust was lost for the entirety of the .au namespace, and this is a big deal.
The supposed 15,000 impacted domains quoted, may mistakenly suggest a representation of how many domains are DNSSEC-signed, which is not something to be proud of, because it also reflects the poor adoption of DNSSEC in Australia. What has auDA done to boost DNSSEC adoption, arguably the most important security features domain name registrants should take? Somewhat of an oversight, considering auDA’srecent appointment to theDNS Abuse Institute Advisory Council.
What if this outage had happened in the middle of a national emergency, such as the recent floods on Australia’s east coast or bushfires in the southwest, it could be catastrophic for those trying to not just gain information, but also those trying to disseminate information through television and radio.
So, what was the core of the actual outage? Well, explaining the issue in as simple as terms as possible, a DNS expert explained the problem to me as, “any fully DNSSEC validating recursor would have been unable to validate the chain of trust for any domain in the .au ccTLD and thus would have failed. A prominent example of this is the well-known and widely used Cloudflare public recursor ‘220.127.116.11’, which anyone using during this time would have been unable to access sites within the au ccTLD.”
The outage has also raised the ire of Adrian Kinderis, the founder of the first commercial .au registry AusRegistry, who tweeted “This has been horribly downplayed by auDA. No accountability. How can one bug have such an impact. A break down of protocols and process. How can we trust them?”
Kinderis was scathing of both the outage and today’s launch of second level registrations, which had its own debacle, as it was clear auDA and Afilias were not ready to go live. He says the launch should have been delayed until systems were working as they should have been expected.
“The lack of testing is apparent,” Kinderis told me. “Afilias’ .au operations are managed out of the U.S., so if anything critical happens outside of U.S. business hours it won’t be acted on immediately. We were very proud that all AusRegistry technical staff were based in Australia, and today under GoDaddy Registry they still are. The launch was a technical failure, but the PR was great.”
“A ccTLD is a piece of a nation’s critical infrastructure. auDA is a technical organisation responsible for ensuring that critical infrastructure works 100% of the time. We never had any DNS downtime when we managed the .au registry.”
“Today’s launch was the first chance Afilias have had to do anything apart from manage the .au ccTLD we handed over to them. Afilias were given a two-year extension on the understanding they could manage the introduction of second level registrations. With two years to plan their introduction, it’s embarrassing that Afilias hasn’t managed to pass their first test and they actually failed.”
“With Afilias having to manually enter new .au registrations, it’s taken .au back 20 years to the days of Robert Elz.” Kinderis spoke proudly of how AusRegistry were the first ccTLD registry globally to enable live updates. “Quite frankly, it’s embarrassing,” Kinderis continued. “Embarrassing for auDA, the auDA Board, Afilias and most importantly consumers.”
“Afilias were awarded the .au registry contract because, in the words of auDA, they were ‘technically superior’ to AusRegistry.” But how Kinderis asks “have they been technically superior to any registry, not just up against AusRegistry but any in the world?”
Looking forward Kinderis doesn’t expect any ramifications to either auDA or Afilias. He expects auDA to give Afilias a “hall pass” as it was auDA who gave Afilias a two-year extension to their registry contract, with the introduction of second level registrations the prime reason. “auDA should be calling for an explanation and making it public, people should lose their jobs over this. Afilias is failing to meet their SLAs (Service Level Agreement).”
But neither CEO Rosemary Sinclair, Chair Alan Cameron, or anyone else, is calling for accountability. Nobody at auDA or Afilias has put their name to any comment. The auDA Board is full of policy people with limited background in the domain name business. Only one director comes from the domain name industry.
One of the issues Kinderis is most concerned about is the security, operational stability and resiliency of Australia’s domain name system. “We’re starting to see how vulnerable the Australian DNS has become” he lamented.
Today’s snafu on the day the six-month Priority Allocation Process for second level/direct .au registrations commenced, which provides priority access to existing registrants for the first six months, originally came to light to me via registrar Synergy Wholesale. The registrar alerted customers of “some changes to the .au direct launch.” Synergy explained “due to issues implementing the .au direct TLD at the Afilias registry, people who purchase a .au direct domain name will not have their DNS record added to the .au zone until it is done manually by Afilias.”
The result is new .au domain name registrations at the second level “will not be live immediately.” Synergy explains registrants “will be able to edit the DNS records on the domain through Synergy Wholesale, however, the DNS will not be synced to the registry until Afilias adds them manually. Once it is manually added, your website will be able to resolve as normal.”
In an email to registrars, Afilias advised “during the initial period, the .au Registry will publish DNS updates for second level .au names manually” with updates occurring Tuesdays to Saturdays “until automated DNS updates resume.”
This has raised the ire of another, former auDA board member Simon Johnson, who asked “is the Australian Internet DNS manually updated by an overseas company, remotely offshore? I hear that Registry updates now manual! Who accepted risk on behalf of the Commonwealth? Critical infrastructure! Should have 🇦🇺 staff”
“The issue will also include changing Nameserver records for .au direct domains,” Synergy explained. “After a Nameserver change is made within Synergy Wholesale, it could take a few days for this change to take effect at the registry.”
It is expected a resolution of the issue will take at least one week to implement, however as Synergy note they “do not have a time frame for this to be resolved.”
To explain why this is happening Synergy informed their customers “when Afilias was implementing system changes to manage DNS for .au direct domains, the system was unable to perform the automatic DNS updates correctly.”
In registrar circles, it is understood this issue was widely known for up to a week before today’s launch of second level registrations. So it begs the question why wasn’t the issue uncovered earlier? And then why go ahead with the launch while this issue was unresolved? And why did auDA allow Afilias to miss the go-live date and manage the process manually, despite having at least two years to prepare?
auDA was contacted for comment on the issue with second level/direct registrations, but did not respond. However it appears the most likely explanation for today’s snafu is a lack of testing and communication, however auDA and Afilias have had at least two years to prepare for this launch. For Tuesday’s security breach, there has been a lack of transparency in what has happened and admitting mistakes and/or lessons learned. Same old (since 2016 at least) secretive auDA.