A healthy amount of paranoia

A few months ago I wrote a blog post at work after us experiencing someone losing an API key to an attacker while only working locally. Long story short they used one of those services that temporarily exposed your local site to the public network to test a webhook. That process caused a new SSL certificate to be minted and of course due to certificate transparency changes the creation of that SSL certificate put a record in public view.

An attacker monitored that list and scanned the website, identified the service from a request or two, fired targeted checks and within a few requests exfiltrated a secret. The blog post dives into how we discovered a tiny security flaw in Laravel which was hardened via Laravel Sentinel across a few packages.

The best non-malicious analogy I can align to that is when you acquire a home your mortgage/loan is semi public information. Companies will use that life event and loan information to urge you (the homeowner) for insurance or gifts and they work fast. You'll get these letters days after finalizing a loan for weeks to come. Thankfully in this case it isn't as malicious as a digital attacker working in seconds.

I started tinkering on some internal projects with GLM 5.3 and it was creepy how well this model targeted a system. Unlike existing non-AI solutions that just hammer every technique in existence - this model quickly identified its best guess of the underlying software (NGINX/PHP) from headers and source. Once it identified its target it worked quick to identify relevant vulnerabilities and get to work. It ended up finding a little gadget chain / serialization problem and I was impressed.

This was a model I found from a tweet and put a few dollars down and messed with it. This was entirely available for anyone in the public which means the real security based models in private or behind countries is probably way beyond that in terms of capabilities. So that induces a bit of paranoia, because take the security mindset of offense/defense. Defense has to be right every single time - one mistake and the attacker wins. In the era of AI attackers they will take no breaks and work until a success arrives or money runs out.

It reminds me that historically a website could defend itself pretty well by blocking countries at the IP address level. Depending on the service they may even block hosting providers or VPNs due to their known IP ranges in which they work. For most attacks that are nothing more than a sprayed generic attack across the web - you can't attack what you can't access sort of works.

Except in the era of AI agents consuming as much data as possible and attackers needing to get more clever. There is an enormous market of selling residential proxies to bypass rate limits or blend in among regular residential traffic. It's a bit insane to think your neighbor may be hosting a proxy and making money no idea they are proxying an infinite amount of nefarious things through their home.

It's like when Leaf was going offline because over 110,000 unique IPs were totaling 4~ million requests against my Halo Infinite stat site. Barely 2,000 people played Halo Infinite in the past 24 hours so it's pretty much impossible to consider a unique ~100,000 people going to a 3rd party stat site. While most of these bots were working inside IP ranges that resembled AWS, Google Cloud or Azure a solid chunk were not. Are these services getting clever and proxying traffic through huge sets of residential proxies?

At least in that case I could deploy Anubis to put a stop to the harvesting of data. It won't work forever, but much like blocking countries is only a barrier against the basic attacks Anubis works the same for AI bots. A good barrier to keep those AI agents away that don't want to do any extra work.

I'm blown away that models continue to improve on what is probably still under the hood just Markov chains, but a bit of paranoia grows that we are leading to something more dangerous than we realize.