#archiveteam — Public Fediverse posts

RE: https://social.treehouse.systems/@nicolas17/116269606648203703

This is a great outcome on #ArchiveTeam #ArchiveBot. Thank you, Nicolás!

Not the first time the #ArchiveTeam #ArchiveBot has been accused of excessive traffic.

https://utcc.utoronto.ca/~cks/space/blog/web/WebScrapingItsNotJustLoad?showcomments

I donʼt know whoʼs running the #ArchiveTeam #ArchiveBot, but you just donʼt load someoneʼs server like that. Loads multiples greater than Googlebot and Baidu. Not the first dodgy scraper found on Github, and I canʼt trust that people arenʼt using them as theftbots.

@jonny Nice. The torrents are seeding smoothly. https://sciop.net/tags/smithsonian

However I'm not getting any downloads from the webseed, at least in Trasmission 4.0.6 (38c164933e). It's looking for an URL like https://smithsonian-open-access.s3.amazonaws.com/media/NASM-2000-9391.tif . Do they really have all files in a single directory?

#digipres #ArchiveTeam

https://blog.gslin.org/archives/2025/08/18/12581/archive-team-%e6%8a%8a-goo-gl-%e6%8a%93%e5%ae%8c%e4%ba%86/

Archive Team 把 goo.gl 抓完了

#archive #archiveteam #goo.gl #google #service #short #team #url

Digital archival projects are crucial in the fight against fascism. I wrote about the why and the how.

And if you're reading this, that means you have a computer, so you too can contribute!

https://carefullmusings.bearblog.dev/the-urgency-of-digital-archiving/

#ArchiveTeam #SciOp #fascism #archive #resistance #DigitalPreservation

@raffaele just a heads up: if you wanna help save what’s there you can use a #ArchiveTeam warrior http://warrior.archiveteam.org and set it to work on the goo.gl project

@slashdot good, one less stupid link shortening service that breaks the web.

Luckily those good folks at #ArchiveTeam have been trying to archive them as much as possible.

You can help out save these short-sighted services by running your own VM that archives websites and uploads them to the @internetarchive

Check out ArchiveTeam and get the software here: https://wiki.archiveteam.org/

I still have a bunch of computers running ArchiveTeam Warrior and here are the totals for I've downloaded so far...

💿 Telegram 107 GiB
💿 Voice of America 94 GiB
💿 US Government 44 GiB
💿 Goo-gl 450 MiB
💿 Twitch 135 MiB

#ArchiveTeam #archive #data #web

@eloquence
Having posts or other indexed/indexable content refer to URL shorteners is dangerous for referrals/archiving/…:

#ArchiveTeam, the people behind e.g. the effort to archive US government websites in a hurry—before they were deleted/changed in an even greater hurry by the current administration, write about #URLShorteners:

"Such services are a ticking timebomb. If they go away, get hacked, or sell out, millions of links will be lost (see Wikipedia: Link Rot)."
https://wiki.archiveteam.org/index.php/URLTeam

@eloquence
Having posts or other indexed/indexable content refer to URL shorteners is dangerous for referrals/archiving/…:

#ArchiveTeam, the people behind e.g. the effort to archive US government websites in a hurry—before they were deleted/changed in an even greater hurry by the current administration, write about #URLShorteners:

"Such services are a ticking timebomb. If they go away, get hacked, or sell out, millions of links will be lost (see Wikipedia: Link Rot)."
https://wiki.archiveteam.org/index.php/URLTeam

@eloquence
Having posts or other indexed/indexable content refer to URL shorteners is dangerous for referrals/archiving/…:

#ArchiveTeam, the people behind e.g. the effort to archive US government websites in a hurry—before they were deleted/changed in an even greater hurry by the current administration, write about #URLShorteners:

"Such services are a ticking timebomb. If they go away, get hacked, or sell out, millions of links will be lost (see Wikipedia: Link Rot)."
https://wiki.archiveteam.org/index.php/URLTeam

@eloquence
Having posts or other indexed/indexable content refer to URL shorteners is dangerous for referrals/archiving/…:

#ArchiveTeam, the people behind e.g. the effort to archive US government websites in a hurry—before they were deleted/changed in an even greater hurry by the current administration, write about #URLShorteners:

"Such services are a ticking timebomb. If they go away, get hacked, or sell out, millions of links will be lost (see Wikipedia: Link Rot)."
https://wiki.archiveteam.org/index.php/URLTeam

@eloquence
Having posts or other indexed/indexable content refer to URL shorteners is dangerous for referrals/archiving/…:

#ArchiveTeam, the people behind e.g. the effort to archive US government websites in a hurry—before they were deleted/changed in an even greater hurry by the current administration, write about #URLShorteners:

"Such services are a ticking timebomb. If they go away, get hacked, or sell out, millions of links will be lost (see Wikipedia: Link Rot)."
https://wiki.archiveteam.org/index.php/URLTeam

The ArchiveTeam Warrior has been running intermittently on my laptop for ten days now.

It downloads stuff and puts it into the Internet Archive.

Everything's fine. It only runs while I use the laptop. I don't notice it. When the laptop goes into standby and wakes up again that doesn't seem to have any adverse effects.

I've downloaded and uploaded gigabytes so far. The top of the leader board for this project is half a petabyte.

Now I'm considering a installation where it could run around the clock. I don't want to increase our household's standby energy consumption too much, so I will see how that goes.

@internetarchive

#archiving
#internetArchive
#DataRescue
#dataPreservation
#digtitalPreservation
#archiveTeamWarrior
#archiveTeam

The ArchiveTeam Warrior has been running intermittently on my laptop for ten days now.

It downloads stuff and puts it into the Internet Archive.

Everything's fine. It only runs while I use the laptop. I don't notice it. When the laptop goes into standby and wakes up again that doesn't seem to have any adverse effects.

I've downloaded and uploaded gigabytes so far. The top of the leader board for this project is half a petabyte.

Now I'm considering a installation where it could run around the clock. I don't want to increase our household's standby energy consumption too much, so I will see how that goes.

@internetarchive

#archiving
#internetArchive
#DataRescue
#dataPreservation
#digtitalPreservation
#archiveTeamWarrior
#archiveTeam

The ArchiveTeam Warrior has been running intermittently on my laptop for ten days now.

It downloads stuff and puts it into the Internet Archive.

Everything's fine. It only runs while I use the laptop. I don't notice it. When the laptop goes into standby and wakes up again that doesn't seem to have any adverse effects.

I've downloaded and uploaded gigabytes so far. The top of the leader board for this project is half a petabyte.

Now I'm considering a installation where it could run around the clock. I don't want to increase our household's standby energy consumption too much, so I will see how that goes.

@internetarchive

#archiving
#internetArchive
#DataRescue
#dataPreservation
#digtitalPreservation
#archiveTeamWarrior
#archiveTeam

The ArchiveTeam Warrior has been running intermittently on my laptop for ten days now.

It downloads stuff and puts it into the Internet Archive.

Everything's fine. It only runs while I use the laptop. I don't notice it. When the laptop goes into standby and wakes up again that doesn't seem to have any adverse effects.

I've downloaded and uploaded gigabytes so far. The top of the leader board for this project is half a petabyte.

Now I'm considering a installation where it could run around the clock. I don't want to increase our household's standby energy consumption too much, so I will see how that goes.

@internetarchive

#archiving
#internetArchive
#DataRescue
#dataPreservation
#digtitalPreservation
#archiveTeamWarrior
#archiveTeam

The ArchiveTeam Warrior has been running intermittently on my laptop for ten days now.

It downloads stuff and puts it into the Internet Archive.

Everything's fine. It only runs while I use the laptop. I don't notice it. When the laptop goes into standby and wakes up again that doesn't seem to have any adverse effects.

I've downloaded and uploaded gigabytes so far. The top of the leader board for this project is half a petabyte.

Now I'm considering a installation where it could run around the clock. I don't want to increase our household's standby energy consumption too much, so I will see how that goes.

@internetarchive

#archiving
#internetArchive
#DataRescue
#dataPreservation
#digtitalPreservation
#archiveTeamWarrior
#archiveTeam

Installed and started the ArchiveTeam Warrior. Very smooth experience.

It downloads stuff and puts it into the Internet Archive.

I took the "ArchiveTeam’s Choice" project and it chose public telegram channels. It's not taking a lot of bandwidth or memory or space or computing, as far as I can tell. It might take too much of my time and focus if I continue staring at the dashboard to try and figure out what all that stuff is.

http://warrior.archiveteam.org/

@internetarchive

#archiving #internetArchive #DataRescue #dataPreservation #digtitalPreservation #archiveTeamWarrior #archiveTeam

Installed and started the ArchiveTeam Warrior. Very smooth experience.

It downloads stuff and puts it into the Internet Archive.

I took the "ArchiveTeam’s Choice" project and it chose public telegram channels. It's not taking a lot of bandwidth or memory or space or computing, as far as I can tell. It might take too much of my time and focus if I continue staring at the dashboard to try and figure out what all that stuff is.

http://warrior.archiveteam.org/

@internetarchive

#archiving #internetArchive #DataRescue #dataPreservation #digtitalPreservation #archiveTeamWarrior #archiveTeam

Installed and started the ArchiveTeam Warrior. Very smooth experience.

It downloads stuff and puts it into the Internet Archive.

I took the "ArchiveTeam’s Choice" project and it chose public telegram channels. It's not taking a lot of bandwidth or memory or space or computing, as far as I can tell. It might take too much of my time and focus if I continue staring at the dashboard to try and figure out what all that stuff is.

http://warrior.archiveteam.org/

@internetarchive

#archiving #internetArchive #DataRescue #dataPreservation #digtitalPreservation #archiveTeamWarrior #archiveTeam

Installed and started the ArchiveTeam Warrior. Very smooth experience.

It downloads stuff and puts it into the Internet Archive.

I took the "ArchiveTeam’s Choice" project and it chose public telegram channels. It's not taking a lot of bandwidth or memory or space or computing, as far as I can tell. It might take too much of my time and focus if I continue staring at the dashboard to try and figure out what all that stuff is.

http://warrior.archiveteam.org/

@internetarchive

#archiving #internetArchive #DataRescue #dataPreservation #digtitalPreservation #archiveTeamWarrior #archiveTeam

Installed and started the ArchiveTeam Warrior. Very smooth experience.

It downloads stuff and puts it into the Internet Archive.

I took the "ArchiveTeam’s Choice" project and it chose public telegram channels. It's not taking a lot of bandwidth or memory or space or computing, as far as I can tell. It might take too much of my time and focus if I continue staring at the dashboard to try and figure out what all that stuff is.

http://warrior.archiveteam.org/

@internetarchive

#archiving #internetArchive #DataRescue #dataPreservation #digtitalPreservation #archiveTeamWarrior #archiveTeam

There was a lack of a decent web based leaderboard so I wrote one in python and got up to speed on publishing to pypi properly. It's a nice minimal example of publishing a single python file to an installable command.

https://github.com/westonal/archive-warrior-leaderboard-cli

#archiveteam #archiveteamwarrior

Anyone else running an #ArchiveTeam instance on the usgov project? Mine was humming all week but not it's not getting any items. Is the archive... done?

I managed to help archive ~35GB of US Government web content with my #ArchiveTeam Warrior instance. At the moment there are no more available to-do items, but I'm keeping my warrior alive if any items re-enter the queue. Glad I was able to donate some CPU and bandwidth to the cause :)

#InternetArchive #USPol #Government

I love that there are so many great efforts out there preserving Geocities, and the internet as a whole. So much wholesome, personal stuff, imortalised.

To the #ArchiveTeam nerds, thank you for being awesome.

I only just now took a good look at the Archive Team Warrior logo 😹 10/10 no notes

#ArchiveTeam #ArchiveTeamWarrior

Started helping #archiveteam backup all of the US government websites last night. Up to 7Gb today! Simple docker setup in #Unraid for me but If you have bandwidth and a computer doing nothing you can help too: https://wiki.archiveteam.org/index.php/Main_Page

Really simple instructions on how to help archive the US Government's websites:

https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior#Installing_and_running_with_Docker

Although #ArchiveTeam are not affiliated with with the #InternetArchive, that is where the archived sites are stored.

#Archive #Backups #USPol

#US #USA #archiveteam #archive #usgovernment

YESSSS! ^.^ 💪 NLM website right now

If you have some bandwith to offer and want to help --> https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior

🚨 Technologists, archivists, and internet historians—this is your call to action! 🚨

As the authoritarian, censorship-happy Trump/Musk regime takes hold, federal government websites are getting wiped, altered, or disappeared entirely. Critical public records, scientific research, and historical data are at risk. We can’t let that happen.

🔹 Take action now: Use #ArchiveTeam’s tool to help preserve government websites before they vanish.

💾 How you can help:
1️⃣ Set up the Archive Team’s docker-compose tool.
2️⃣ Start archiving key sites you care about.
3️⃣ Share this message with other technologists.

The internet remembers only if we make it remember. Let’s keep public data public. 🛡️📜 #ArchivingMatters #InternetPreservation #SaveGovData

https://github.com/goudarziha/archive-team-docker-compose/tree/master

🚨 Technologists, archivists, and internet historians—this is your call to action! 🚨

As the authoritarian, censorship-happy Trump/Musk regime takes hold, federal government websites are getting wiped, altered, or disappeared entirely. Critical public records, scientific research, and historical data are at risk. We can’t let that happen.

🔹 Take action now: Use #ArchiveTeam’s tool to help preserve government websites before they vanish.

💾 How you can help:
1️⃣ Set up the Archive Team’s docker-compose tool.
2️⃣ Start archiving key sites you care about.
3️⃣ Share this message with other technologists.

The internet remembers only if we make it remember. Let’s keep public data public. 🛡️📜 #ArchivingMatters #InternetPreservation #SaveGovData

https://github.com/goudarziha/archive-team-docker-compose/tree/master

🚨 Technologists, archivists, and internet historians—this is your call to action! 🚨

As the authoritarian, censorship-happy Trump/Musk regime takes hold, federal government websites are getting wiped, altered, or disappeared entirely. Critical public records, scientific research, and historical data are at risk. We can’t let that happen.

🔹 Take action now: Use #ArchiveTeam’s tool to help preserve government websites before they vanish.

💾 How you can help:
1️⃣ Set up the Archive Team’s docker-compose tool.
2️⃣ Start archiving key sites you care about.
3️⃣ Share this message with other technologists.

The internet remembers only if we make it remember. Let’s keep public data public. 🛡️📜 #ArchivingMatters #InternetPreservation #SaveGovData

https://github.com/goudarziha/archive-team-docker-compose/tree/master

🚨 Technologists, archivists, and internet historians—this is your call to action! 🚨

As the authoritarian, censorship-happy Trump/Musk regime takes hold, federal government websites are getting wiped, altered, or disappeared entirely. Critical public records, scientific research, and historical data are at risk. We can’t let that happen.

🔹 Take action now: Use #ArchiveTeam’s tool to help preserve government websites before they vanish.

💾 How you can help:
1️⃣ Set up the Archive Team’s docker-compose tool.
2️⃣ Start archiving key sites you care about.
3️⃣ Share this message with other technologists.

The internet remembers only if we make it remember. Let’s keep public data public. 🛡️📜 #ArchivingMatters #InternetPreservation #SaveGovData

https://github.com/goudarziha/archive-team-docker-compose/tree/master

🚨 Technologists, archivists, and internet historians—this is your call to action! 🚨

As the authoritarian, censorship-happy Trump/Musk regime takes hold, federal government websites are getting wiped, altered, or disappeared entirely. Critical public records, scientific research, and historical data are at risk. We can’t let that happen.

🔹 Take action now: Use #ArchiveTeam’s tool to help preserve government websites before they vanish.

💾 How you can help:
1️⃣ Set up the Archive Team’s docker-compose tool.
2️⃣ Start archiving key sites you care about.
3️⃣ Share this message with other technologists.

The internet remembers only if we make it remember. Let’s keep public data public. 🛡️📜 #ArchivingMatters #InternetPreservation #SaveGovData

https://github.com/goudarziha/archive-team-docker-compose/tree/master

goo.gl 將在 2025/08/25 停止轉址

堆了一陣子的消息，Google 將在 2025/08/25 停止 goo.gl 轉址服務：「Google URL Shortener links will no longer be available」。

2019 就已經停止新的網址了：「Transitioning Google URL Shortener to Firebase Dynamic Links」，要維持這個服務對 G

https://blog.gslin.org/archives/2024/08/11/11930/goo-gl-%e5%b0%87%e5%9c%a8-2025-08-25-%e5%81%9c%e6%ad%a2%e8%bd%89%e5%9d%80/

#Computer #Murmuring #Network #Service #archive #archiveteam #google #internet #links #shortener #shutdown #sunset #url #urlteam

goo.gl 將在 2025/08/25 停止轉址

堆了一陣子的消息，Google 將在 2025/08/25 停止 goo.gl 轉址服務：「Google URL Shortener links will no longer be available」。

2019 就已經停止新的網址了：「Transitioning Google URL Shortener to Firebase Dynamic Links」，要維持這個服務對 G

https://blog.gslin.org/archives/2024/08/11/11930/goo-gl-%e5%b0%87%e5%9c%a8-2025-08-25-%e5%81%9c%e6%ad%a2%e8%bd%89%e5%9d%80/

#Computer #Murmuring #Network #Service #archive #archiveteam #google #internet #links #shortener #shutdown #sunset #url #urlteam

goo.gl 將在 2025/08/25 停止轉址

堆了一陣子的消息，Google 將在 2025/08/25 停止 goo.gl 轉址服務：「Google URL Shortener links will no longer be available」。

2019 就已經停止新的網址了：「Transitioning Google URL Shortener to Firebase Dynamic Links」，要維持這個服務對 G

https://blog.gslin.org/archives/2024/08/11/11930/goo-gl-%e5%b0%87%e5%9c%a8-2025-08-25-%e5%81%9c%e6%ad%a2%e8%bd%89%e5%9d%80/

#Computer #Murmuring #Network #Service #archive #archiveteam #google #internet #links #shortener #shutdown #sunset #url #urlteam

goo.gl 將在 2025/08/25 停止轉址

堆了一陣子的消息，Google 將在 2025/08/25 停止 goo.gl 轉址服務：「Google URL Shortener links will no longer be available」。

2019 就已經停止新的網址了：「Transitioning Google URL Shortener to Firebase Dynamic Links」，要維持這個服務對 G

https://blog.gslin.org/archives/2024/08/11/11930/goo-gl-%e5%b0%87%e5%9c%a8-2025-08-25-%e5%81%9c%e6%ad%a2%e8%bd%89%e5%9d%80/

#Computer #Murmuring #Network #Service #archive #archiveteam #google #internet #links #shortener #shutdown #sunset #url #urlteam

goo.gl 將在 2025/08/25 停止轉址

堆了一陣子的消息，Google 將在 2025/08/25 停止 goo.gl 轉址服務：「Google URL Shortener links will no longer be available」。

2019 就已經停止新的網址了：「Transitioning Google URL Shortener to Firebase Dynamic Links」，要維持這個服務對 G

https://blog.gslin.org/archives/2024/08/11/11930/goo-gl-%e5%b0%87%e5%9c%a8-2025-08-25-%e5%81%9c%e6%ad%a2%e8%bd%89%e5%9d%80/

#Computer #Murmuring #Network #Service #archive #archiveteam #google #internet #links #shortener #shutdown #sunset #url #urlteam

<<“So,” said Vincent, “you have your server. I remind you to be good internet citizens. Your actions are my responsibilities sitting on my shoulder.”>> #Posterous #ArchiveTeam #InternetArchive #ComputerHistory

From: @krinkle
https://fosstodon.org/@krinkle/112753232168570670

Just got out of a deep rabbit hole reading the epic tale of how the Archive Team volunteers preserved Posterous back in 2013. Woah!

https://wiki.archiveteam.org/index.php/Posterous/Story

via https://wiki.archiveteam.org/index.php/Frequently_Asked_Questions,
via https://wiki.archiveteam.org/index.php?title=ArchiveTeam_Warrior,
via https://legoktm.com/view/Free_bandwidth,
via @legoktm.

#Posterous #ArchiveTeam #InternetArchive #ComputerHistory

📬 Subscene geschlossen: Das Ende einer Ära für Film- und Serienfans
#Streaming #ArchiveTeam #Divxstation #FilmeundSerien #Subscene #Untertitel https://sc.tarnkappe.info/9ec9ba

@Danielmorgan @willa Welcome to the Fediverse! Tumult Hype looks amazing, and I am totally onboard with #archiveteam's mission.

I now have #archivewarrior running on my beefy desktop and in a VM on my Plex server (a retired Alienware gaming laptop which is pretty beefy in its own right :)

@darnell General Web Search is ... sort of its own thing. That's manageable through robots.txt or permissive / exclusive in-page tags.

(Those will generally prevent content from being presented, but may not prevent crawling, and in the case of on-page headers cannot by the mechanism through which they work (the spider has to crawl and read the header to determine what's being said).

There are groups such as the #ArchiveTeam who explicitly ignore robots.txt: https://wiki.archiveteam.org/index.php/Robots.txt

Then there's the somewhat newly recognised issue of AI LLM training data and derived works.

Other than those, what is your threat model here?

• What risks do you see?
• What are you trying to avoid?
• What would you specifically like to see?

My view is that online content is ... online. It's published, in the sense of public. If you want closed content you need to find some way of disclosing to a limited group. That has tremendous impacts on reach and influence.

That is contrasted with community and interaction, and a Fediverse which is crawled by Google is very different from one that is interfaced by Google and Facebook, parallel with their existing social networks (FB, Instagram, YouTube, Blogger, say).

@jwildeboer

#Meta #Metablock #DefederateMeta #ThreatModels #Risk #GeneralWebSearch #LLM #ArtificialIntelligence #TrainingData

@darnell General Web Search is ... sort of its own thing. That's manageable through robots.txt or permissive / exclusive in-page tags.

(Those will generally prevent content from being presented, but may not prevent crawling, and in the case of on-page headers cannot by the mechanism through which they work (the spider has to crawl and read the header to determine what's being said).

There are groups such as the #ArchiveTeam who explicitly ignore robots.txt: https://wiki.archiveteam.org/index.php/Robots.txt

Then there's the somewhat newly recognised issue of AI LLM training data and derived works.

Other than those, what is your threat model here?

• What risks do you see?
• What are you trying to avoid?
• What would you specifically like to see?

My view is that online content is ... online. It's published, in the sense of public. If you want closed content you need to find some way of disclosing to a limited group. That has tremendous impacts on reach and influence.

That is contrasted with community and interaction, and a Fediverse which is crawled by Google is very different from one that is interfaced by Google and Facebook, parallel with their existing social networks (FB, Instagram, YouTube, Blogger, say).

@jwildeboer

#Meta #Metablock #DefederateMeta #ThreatModels #Risk #GeneralWebSearch #LLM #ArtificialIntelligence #TrainingData

@darnell General Web Search is ... sort of its own thing. That's manageable through robots.txt or permissive / exclusive in-page tags.

(Those will generally prevent content from being presented, but may not prevent crawling, and in the case of on-page headers cannot by the mechanism through which they work (the spider has to crawl and read the header to determine what's being said).

There are groups such as the #ArchiveTeam who explicitly ignore robots.txt: https://wiki.archiveteam.org/index.php/Robots.txt

Then there's the somewhat newly recognised issue of AI LLM training data and derived works.

Other than those, what is your threat model here?

• What risks do you see?
• What are you trying to avoid?
• What would you specifically like to see?

My view is that online content is ... online. It's published, in the sense of public. If you want closed content you need to find some way of disclosing to a limited group. That has tremendous impacts on reach and influence.

That is contrasted with community and interaction, and a Fediverse which is crawled by Google is very different from one that is interfaced by Google and Facebook, parallel with their existing social networks (FB, Instagram, YouTube, Blogger, say).

@jwildeboer

#Meta #Metablock #DefederateMeta #ThreatModels #Risk #GeneralWebSearch #LLM #ArtificialIntelligence #TrainingData