Hi,

I had a couple of people mention they were getting an error trying to access a link I posted in c/fountainpens. One of the users (@gemelen@mammut.moe) created a traffic dump of trying to access the link using curl. He posted his results on this Opensuse paste bin.

He added this:

I guess Cloudflare prevents some web clients (like embedded viewer for a Mastodon mobile app) and some networks (like mine Uruguayan Internet provider) - I could open the feddit.online and see your post there in a desktop browser, but being forbidden to access from a mobile or via curl.

George

  • Jerry on PieFed@feddit.onlineMEnglish
    1·
    3 months ago

    No, the agent string is not enough.

    I just checked. In the past 24 hours, the firewall has dumped 98K requests from AI bots that tried really hard to look like normal traffic and not like AI bots. These bots did a lot more than just set a user agent to try to look like real users, but it didn’t help them.

    • Unattributed 𓂃✍︎@feddit.onlineOPEnglish
      1·
      3 months ago

      Not surprising… I’m sure CloudFlare is working overtime to block these ridiculous bots. I’ve seen my analytics provider is blocking tons of bots every day, even low volume sites like mine are getting hit with hundreds of bot requests.