Blog Post

Encrypted Traffic Analysis: Encrypted DNS – Privacy, Security and the SOC (Part 3)

Effective Security Controls in an Encrypted DNS World

This is the third and final post in a series about encrypted DNS. Having discussed the privacy and security implications of encrypted DNS, we now focus on how security teams can continue to be effective as encrypted DNS gains popularity. I recognize that not every network is the same and every organization’s network stack is different. .My goal is to offer several possible solutions that can be applied depending on what your toolset is capable of.

Rather than break this post into sections based on protocol, I will organize it by types of controls, since there is a lot of overlap between the 3 protocols.

Firefox Canary Domain

Firefox has offered a solution to help control the usage of DNS over HTTPS (DoH) on your network. You can create an entry in your internal DNS server, a “canary domain”, that will signal to the browser not to use DoH.

Group Policy Management

If a user simply enables DoH manually in Firefox, then the browser will ignore the canary domain. This is by design, given the privacy concerns encrypted DNS is focused on. It prevents an ISP, government or whomever from using the canary domain to prevent DoH. To prevent this from happening on your network, you can disable the ability for users to alter Firefox settings using group policy management.

Disallowing Installation of Third Party Software

DoT and DNSCrypt require the installation of software on a device. DoH can be used that way, but it’s not a requirement. In any of these cases, as long as devices do not allow the installation of third party software, then users (or attackers) will not be able to use these protocols.

On the Wire

The last two mitigations require that you have complete control over all of the hosts on your network. I have never in my career monitored a network that were even aware of all of their assets let alone managed them all. In those cases, you’ll need to turn to blocking or detecting these protocols as they traverse your network.

The default port for DoT is 853, so it can potentially be blocked by blocking that port. However, it’s fairly trivial to change the default port to something else, like 443 where you expect to see TLS traffic, in order to circumvent that control – which doesn’t mean you shouldn’t block the port, just that you should be aware of that method’s shortcomings. Similarly, you could detect some DoT that way as well.

DNSCrypt is its own protocol, unique from TLS, which means that it can be identified and/or blocked if your toolset is able to parse it.

DoT may use TLS just like your browser. However, since the use of DoT requires the installation of a unique binary, the metadata present in the Client Hello portion of the TLS handshake will be different than that from a browser on the same device.

On the left is the metadata from a TLS session initiated by a DoT client, and on the right is metadata from a browser on the same device – as you can see they’re quite different. It’s this data that is used by fingerprinting methods such as JA3 to differentiate unique clients using TLS. If you have the ability to use a TLS fingerprinting method or identify differences in TLS metadata by parsing and running analysis on that data, then this is a viable method of detecting DoT.

By Destination

Since DoT and DoH can do a good job of blending into legitimate traffic, and you may not have the ability to implement many of the previously mentioned controls, there is one simple option that will likely go a long way: blocking based on destination. There is a publicly available list of DNS servers that support encrypted DNS. This list can be used to define blocklists and thus prevent these protocols from being used on your network.

In order to collect the destinations in a more reasonable and automatically updated format, you can periodically download them from here. You’ll also need to convert them from DNS Stamps to IPs and Domains, which I did with Awake’s internal language, but if you don’t have access to that then there are libraries that can help. You can download this jupyter notebook (that I’m submitting to infosecjupyterthon) that includes python code that will do this for you. Using this method you can be sure that your blocklists and IOCs are always up to date for detecting encrypted DNS.

The one place where this method falls short is when someone is using a custom server that is not publicly listed – for example, and excuse the FUD, if an attacker were to use their own server as a destination for DNS requests for malicious domains. This is a relatively easy thing to do; there are plenty of tutorials for installing software such as stubby, unbound, and others.

For these cases, it’s possible to use data science and the notion of characteristic artifacts to detect this behavior. : Why is TLS traffic periodically requesting this domain or IP that I’ve never seen on my network before? In fact, the detection of these types of interesting events is not really any different than attempting to detect general C2 in the same manner.

Keep Calm and Monitor On

The attacker’s use case is that they can now hide the DNS requests from those monitoring the network. This means that domain IOCs, DGA matching patterns, whois lookups, and reputation information won’t be available. However, they still have to send the encrypted DNS request somewhere. They can send it to an IP, but if they could/wanted to do that, then why use a domain for their C2 in the first place? Why not just use an IP address as the C2 destination? Said differently, this just makes the custom DNS server the IOC. Has this really benefited the attacker?? In short, I think the possibility that attackers might use encrypted DNS is much less of a concern for the security community, than the end-users inadvertently hiding threats for attackers by using encrypted DNS.

Either way, I hope that this series of posts has educated you on what is coming and equipped you with confidence that no matter what your situation is, there are many possible ways to deal with encrypted DNS on your network. And since none of them are perfect, perhaps you might consider a combination of the solutions depending on your unique requirements / situation.

Thanks for reading, and stay at home!



If you liked what you just read, subscribe to hear about our threat research and security analysis.

Tory Kei
Tory Kei

Threat Researcher