Ramblings of a Tampa engineer

In 2017 I installed a Pi-hole into my network and routed all my DNS traffic through it. Today is March 23, 2024 and I've been running it ever since. This will be the 8th post in the pi-hole tag.

The last year was a slow year for Pi-hole releases with all their focus going towards a new slightly rewritten v6 version. This promises to launch with the following large changes:

  • Removal of PHP for a REST API & web-server directly embedded into FTL binary.
  • Alignment of the list features between gravity & anti-gravity.
  • Unified config file for all components.
  • Server side pagination.
  • HTTPS support.

I applaud anyone beta-testing this software - much like my OPNsense software I don't wish to be the bleeding edge for my home network. The development team must really like c++ as they've chosen civetweb to be the replacement for lighttpd.

For a regular end user not tracking v6 - we only had a few upgrades that hit the web repository throughout the last year. We ended last year on 5.18 and now I sit on 5.21.

All in all - it's getting difficult to track just the web version as Pi-hole further expands its split into sub-modules. So as of this post I run:

  • Core: v5.17.3
  • FTL: v5.25.1
  • Web: v5.21

PADD continued to get updates and turned into quite a cool little tool for tracking the usage of Pi-hole.

Eagle-eyed users will notice a few differences in comparison to my last screenshot:

  • 56k queries vs 34k queries.
  • 319k blocked domains vs 196k blocked domains.
  • +3 clients on my network since last year.

This ended up being the large changes I spent work on in the past year - discovering new block-lists. This happened because I started seeing ads again and I was confused - my Pi-hole was still configured and working well. Outside of my original Xbox that was intentionally configured to use a different DNS - everything was still using my Pi-hole DNS.

I quickly realized that I hadn't really changed my lists in years and they had fallen out of date. I was basically using the same lists configured by default in 2017 and embarrassingly unchanged. So I sat down and did a solid of amount of testing and landed on a ton of different lists I prefer.

I ended up utilizing:

The world was moving away from single large lists as it was difficult to audit and update them. This was a list of lists that was the combination of a few different selections, including some old legacy large lists.

  • Malware hosts
  • Ad hosts
  • Privacy hosts
  • Tracker hosts
  • Smart TV hosts
  • Threat-intel hosts
  • Telemetry hosts

So I ended up with this list of block/ad lists below. They combined to give me roughly 320,000 blocked domains. However due to some lists working with regular expressions or wildcards that number can no longer be trusted as exact.

https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate versions Anti-Malware List/AntiMalwareHosts.txt

So this is the reason I gained roughly 123,000 new domains to block. This is where the enhancements to Pi-hole benefited majorly. Older style block lists that just block based on domain were starting to fail as randomly generated ad domains were being used. The ABP style of lists support an alternative syntax that is more efficient and captures more.

This list may not be perfect for you, but I enjoy blocking all ads, trackers, malware and getting my Sony TV under control. Especially since I run Wireguard on my OPNsense firewall my phone is always connected back home for mobile blocking when I'm on the go.

Now it was time to check on my internal "pi-stats" project that gets slower and slower as time goes on.

Total Queries: 58,120,152
First Query: 2019-06-17 00:55:35
Last Query: 2024-03-23 13:09:59
Time in days: 1741
Time taken: 102 minute(s).

It seemed to take about 12 minutes longer than last year with an additional 17 million requests to count.

Top 15 Allowed Domains

Domain Count
e7bf16b0-65ae-2f4e-0a6a-bcbe7b543c73.local 5,631,937
68c40e5d-4310-def5-a1c3-20640e1cd583.local 5,305,150
1d95ffae-4388-9fbc-1646-b2b637cecb64.local 4,898,205
localhost 4,461,024
806c4c48-1715-4220-054f-909f83563938.local 1,342,386
api-0.core.keybaseapi.com 1,160,317
b.canaryis.com 1,118,446
pistats.ibotpeaches.com 881,554 733,703
www.gstatic.com 713,477
ping2.ui.com 670,659
clients4.google.com 560,787
ping.ui.com 541,896
168.192.in-addr.arpa 531,626
www.google.com 457,837

A bit odd that Keybase broke into the top 6 of domains when it even wasn't in top 15 in the prior year. This meant it did roughly 1 million web requests since my last post. I vaguely remember Keybase dying due to a certificate expiration on Hacker News. It was such a bad expiration that you couldn't even auto-update the program anymore. You had to go obtain a fresh download to fix your copy. Too bad Keybase basically died once Zoom acquired it - you only see activity when it happens to be catastrophic failure.

I don't really use Keybase anymore so the fact that it was failing since December 2023 meant I had roughly 3 months of non-stop requests that were failing. I had just assumed my random upgrades to Kali would have done this upgrade - I forgot I tracked Keybase outside of it. So I jumped around between all computers that had Keybase and upgraded it manually. My hope being the amount of those network calls would calm down to normal levels.

Unpacking keybase (6.2.8-20240306193933.e38523abbe) over (6.0.2-20220610191041.a459abf326) ...
Setting up keybase (6.2.8-20240306193933.e38523abbe) ...

If I'm 1 person who forgot to update their Keybase and sent ~1 million requests between Dec 2023 and March 2024 - I can't imagine how their servers handled the rest of folks who never updated.

The random UUID domains are still here and might take years to be bounced out of the top 15. If you remember these were from the multicast behavior added in Big Sur, which caused tons of requests to be made to any .local domain. Why they wouldn't stop if the other device(s) didn't respond properly after the first 100,000 requests - I don't know.

All the other domains line up with my home network - we have

  • Ubiquiti
  • Canary
  • Reverse lookup of IP to find names
  • Google stuff

Nothing stands out except the odd Keybase spamming which I appear to have resolved. If we peek the blocked domains - perhaps the story is the same.

Top 15 Blocked Domains

Domain Count
806c4c48-1715-4220-054f-909f83563938.local 803,900
e7bf16b0-65ae-2f4e-0a6a-bcbe7b543c73.local 638,460
ssl.google-analytics.com 471,606
1d95ffae-4388-9fbc-1646-b2b637cecb64.local 432,008
app-measurement.com 336,953
68c40e5d-4310-def5-a1c3-20640e1cd583.local 247,892
watson.telemetry.microsoft.com 217,743
googleads.g.doubleclick.net 129,666
mobile.pipe.aria.microsoft.com 124,971
sb.scorecardresearch.com 82,818
settings-win.data.microsoft.com 71,150
www.googleadservices.com 66,580
sessions.bugsnag.com 59,678
pagead2.googlesyndication.com 58,744
secure-dcr.imrworldwide.com 52,384

This time we see the Microsoft reign of domains continued to be blocked at scale. I did jump to Windows 11 in the past year and every single upgrade continues to invade my privacy. I blogged about that a few months ago because it became insulting that a single OS patch upgrade would add some AI search-bar to my desktop packed full of analytical trackers.

I believe due to these values rising to insane numbers - I'll have to rethink my report and do values over the past year. Next year I'll adapt the report to top 10 all-time and top 5 in past year to see how things have changed.

Pi-hole continues to evolve and be a great addition to my local network. I'll pass another donation and see them again in another year with another yearly blog post.

You’ve successfully subscribed to Connor Tumbleson
Welcome back! You’ve successfully signed in.
Great! You’ve successfully signed up.
Success! Your email is updated.
Your link has expired
Success! Check your email for magic link to sign-in.