home.social

#webadmin — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #webadmin, aggregated by home.social.

  1. Krishean Draconis @[email protected] ·

    fyi just caught meta-webindexer and meta-externalagent ignoring a Disallow: / rule in robots.txt

    the remote ip does resolve to fb as well, so it's not some random vps spoofing the user agent

    #webadmin #bots

    #webadmin #bots
  2. Krishean Draconis @[email protected] ·

    fyi just caught meta-webindexer and meta-externalagent ignoring a Disallow: / rule in robots.txt

    the remote ip does resolve to fb as well, so it's not some random vps spoofing the user agent

    #webadmin #bots

    #webadmin #bots
  3. Krishean Draconis @[email protected] ·

    fyi just caught meta-webindexer and meta-externalagent ignoring a Disallow: / rule in robots.txt

    the remote ip does resolve to fb as well, so it's not some random vps spoofing the user agent

    #webadmin #bots

    #webadmin #bots