#sciop — Public Fediverse posts

http://librarypunk.gay/e/160-sciopnet-feat-jonny-and-jez-part-1/

>And this is exactly why Linux distributions have been distributing ISOs using torrents for years. Aside from the sort of piracy thing, that's the other thing that BitTorrent gets used a lot for, and it means that the small organizations throwing up their own Linux distro can distribute it without it completely crashing their tiny servers.

You're not wrong :P Pic related. It annoys me that #tails @tails recently moved away from having torrents as a form of downloading.

>Yeah, it makes me, think about, and something I've thought about before but haven't really explored is you know how when you go on Archive.org and there's a file, and one of your options is just to download a torrent. Why don't we do that for institutional repositories and data repositories?

That's one of my favourite parts of the Internet Archive :D

>Because those are starting to get big. I mean, I know one of the reasons is because, like, we use proprietary software for our institutional repository and Clarabate and Elsevier don't want to support that.

Elsevier, the bane of my academic existence.

-----

Also, vis-a-vis webseeds... what's stopping a Tor onion service from being the HTTP(S) endpoint? Those are pretty damn hard to bring down (from a legal standpoint). I'd assume the main limitation is that BitTorrent clients don't speak "onion service," but that's a fixable problem. Hell, what's stopping the trackers themselves from being onion services?

#BitTorrent #SciOp #Tor #OnionService

RE: https://neuromatch.social/@jonny/116218229348059739

impossible to quote myself because i am always speaking in like a 10-layer run-on, but, still thinking about private/public social systems that support resilient digital social systems, around 33:45....

instead of it being one #sciop and that's a unitary thing, we want to do the same thing here where we have a number of these different trackers that can be online and talk to each other and share metadata back and forth ...

the more important role of a tracker is a site of social coordination, a site of giving organization and structure to a bundle of torrents. in particular giving a focus - in the same way that what cd focused labor towards archiving music, having sciop focusing labor as a place to put the public information torrents is the main thing that it actually does. So building those kind of social systems into the tracker, and the next steps are making those social systems extend across multiple trackers ...

A lot of the peer to peer space, especially post crypto boom can lean very libertarian in terms of its design and its goals. That its goal is to make the one big public archive of everything, and that doesn't exactly fit in this context and it's just a very particular arrangement of power. ... If you build your system around assuming that everything should be public and should always be immediately available, then you don't have the means of making these kinds of gray-area private negotiations and discussions that might need to happen for data that's a little more sensitive.

So we need a federation and sharing model that can scale from private, literally peer-to-peer as in "i want to know exactly who is involved in this swarm of peers" up to the global public index. So that's the next step as far as what we're working on this year.

We were on librarypunk again to talk about #sciop. True to form I talk too much and they had to split it into two parts... Again...

http://librarypunk.gay/e/160-sciopnet-feat-jonny-and-jez-part-1/

We spent a bit too much time talking about bittorrent and not enough time talking about the underlying fascism of it all, but hopefully the goal of a federated tracker for connected communities to self-archive comes thru

#sciop is now DMCA registered agent official thanks to a generous corporate sponsorship by Peertech Global Cyberindustrial Concern, LLC.

(This is a job to come work on #sciop)
https://digipres.club/@mickylindlar/116025881444877358

Have had to work on some other things lately, but returning to #sciop and wrote a blog post about current status and our plans for federation - it's about that time. Just need to do one more feature (commenting) and finish up some work for main job responsibilities, and it's off to implementing distributed activitypub where we decouple actors from instances
https://blog.sciop.net/2025-12-08/sfn-and-federation

@josh0 I have a VM consuming two entire copies of the bluesky firehose (okay not as big as twitters but) on a machine in my living room.

It's also the same machine that's processing all these json files.

Oh and running several hundred torrents comprising a few hundred TB of data for #sciop

No sweat, still has a couple dozen CPU threads doing nothing

It's full.

Learn more: https://lydie.cc/data.html

#resist #fascism #datahoarder #digitalpreservation #archive #sciop #torrent #publicdata

Here's another #FEP for representing torrents on activitypub :)

short, sweet, and with a reference implementation and tests!

towards a federated bittorrent tracker with #sciop !

PR: https://codeberg.org/fediverse/fep/pulls/714

Discussion: https://socialhub.activitypub.rocks/t/fep-d8c8-bittorrent-torrent-objects/8309 (or this thread)

#FEP_d8c8 #BitTorrentOverActivityPub #FederatedP2P #BitTorrent

This is extremely sad y'all. ZERO seeding going on from the SciOp / public data / antifa torrent server. Have a few spare gigs? Even that's something, please help preserve data the fascists are destroying!

#resist #fascism #datahoarder #digitalpreservation #archive #sciop #torrent #publicdata

@nyxmir If the CDC used to have it, it might still be on SciOp.

If you don't know SciOp, it's a community where people seed (through Torrent) data threatened or deleted by Trump and other fascists. The datasets are hefty, but the point is to share and keep them available for everyone:

https://sciop.net/datasets/?query=cdc&sort=-threat

Good luck!

#SciOp #Science #CDC #P2P

the haters all scoffed when i "embedded a whole set of bitmap fonts for a single use" but i knew i was following the light and the way. #sciop is in its "intelligibility" era, where we will do things like "tell people what is going on" and "comment on stuff" and "chat about moderation decisions" and whatnot.

#Sciop hit a Petabyte (actually a Pebibyte but nobody knows that word) of total proven capacity a week or two ago. That's all the seeders * the size of the things they are seeding. All volunteers, zero dollars in funding, piggybacking off existing resources wherever we can, run on a donated VPS. This is before we even get into federating archives and are still nailing down the basics of the site.

Peer to peer archives are real and they work, period. 216TiB of threatened cultural, climate, queer, and historical information held in common. That's a people powered archive, and you're welcome in it - to take from, to add to, and help sustain if you can.

Edit: if this is the first you're hearing of sciop, it's at https://sciop.net

First post on the #sciop blog - on adding webseeds from the web interface and why this is cool for bridging archives and bringing bigger systems into the swarm. If you can't run a bittorrent client but still want to seed, this post is for you!

with art from @aud and @lina

https://blog.sciop.net/2025-08-29/webseeds

Im making a blog for #sciop because we keep doing dope shit and not writing it down. Putting a call out for guest artists who want to contribute fake GeoCities era banner ads

Worried about #Smithsonian data and collections? We are too. Our friends over at #SafeguardingResearchAndCulture have been hard at work helping with #DataRescue and adding Smithsonian information to #SciOp. Check out their available datasets and please spread the word: sciop.net/datasets/

Datasets - SciOp

if you are seeding anything on #sciop (or anywhere else too) using qbittorrent (and probably other clients too), you should increase your max torrent size to something like 2GB - that's what's causing the recurring problem that many people have flagged to us where their torrents seem to disappear from their client after restarting:
https://github.com/arvidn/libtorrent/issues/8012

tools > options > advanced, set both torrent file size limit and bdecode token limit very high

v2 torrents are very very good for archives, but they are more rarely used in piracy, so there is comparatively less optimization pressure for them. so this explains why our seed stats are so spiky, because we encourage hybrid/v2, and by default any v2 torrent larger than a few dozen GB will just go poof on restart.

edit: this was actually fixed in qbt 5.1.2, so you can also just update

Last week trump announced plans to "review" 8 Smithsonian museums. Today he doubled down, very explicit about the intent to revise history to reflect the ethno-nationalist fantasy of US history.

You can do something about that! We are backing up the digital archives of those museums on sciop: https://sciop.net/tags/smithsonian

You can take direct action to preserve the historical artifacts the right wants to destroy:

1) you can download a copy and seed it, every seeder counts. Subscribe to the smithsonian RSS feed to auto-download torrents as they are scraped.

2) we have also written a crawler connected to sciop that distributes the scraping work, and automatically creates and uploads a validated torrent that piggybacks off the s3 bucket as a webseed source while it lasts (instructions in reply).

The data from the 8 threatened museums is on the order of ~10 TB, and we have split it up by jpg/tif so people without much spare storage can join in on the jpg's at least. The full contents of the public smithsonian bucket is ~700TB, so if we want to have a full independent copy we'll need lots more seeders.

All this code is being written flat out, on the run, as it's needed by volunteers with exactly zero resources, so it's not polished or well documented, and if you're interested in helping damp the flames of the book burning by contributing to any of the code or docs, we'd love to have you.

#Smithsonian #Sciop

The slides of my talk at #WHY2025 "Safeguarding Research & Culture: Save public data from the digital bookburnings!" are now online:
https://hu.berlin/SRC-WHY2025

Recording here (27min):
https://media.ccc.de/v/why2025-238-safeguarding-research-culture-save-public-data-from-the-digital-bookburnings
(Wow, awesome work by @c3voc 💜)

More context: https://program.why2025.org/why2025/talk/B8DANE/

#SafeguardingResearch #SciOp @SafeguardingResearch

Some of you may have seen the news re National #Climate Assessment Reports
https://www.cnn.com/2025/08/07/climate/wright-national-climate-assessments-updating

A friendly reminder:
They are all accessible here in this archive from November https://globalchange.govarchive.us/

As well as on #SciOp (from April)
https://sciop.net/datasets/globalchange-gov-webrip/pdf

Digital archival projects are crucial in the fight against fascism. I wrote about the why and the how.

And if you're reading this, that means you have a computer, so you too can contribute!

https://carefullmusings.bearblog.dev/the-urgency-of-digital-archiving/

#ArchiveTeam #SciOp #fascism #archive #resistance #DigitalPreservation

I just downloaded every single #torrent from #SciOp and am ensuring that all of them are now in the #antifa torrent server.

btw does anyone want the complete 1.2 gigs of torrents from SciOp in one ZIP package?

#resist #fascism #datahoarder #digitalpreservation #archive

man i just had a series of extremely good ideas* that are very simple and very implementable for #sciop that i think will cause an absolutely disgusting amount of (good, intrinsically deduplicating, actually decrease server load by creating a supporting swarm of peers) public data scraping to happen and basically lower the barrier to scouting endangered datasets to zero

*if you received the message flood of me having them you are not allowed to tell people if they are actually bad

@ai6yr open call for radio people to team up with @SafeguardingResearch and #sciop to snatch the data from the satellites, decode, and re-upload as torrents to replace critical infrastructure with something even better

@ww super cool! Tagging this for more reach: #getfedihired #academicjobs #sciop #safeguardingresearch #scienceundersiege

:crt_w_green_lines: Hackathon: Data Under Threat / Data Rescueing (Aug 7) in #München

The LMU Open Science Center (@lmu_osc) runs a hackathon to support the #SciOp #SafeguardingResearch initiative: Rescuing research data that is deleted by the Trump administration.

Bonus: @lavaeolus will give an ignition talk!

📅 Thursday, 2025-08-07, 16 – 19 (only in-person)
👉 Details and signup: https://github.com/lmu-osc/safeguar.de-hackathon

Become a data rescuer by turning your own laptop into a Research Data Rescue Node, scraping at-risk data sets, and breathing new life into your old HDD as part of a global, decentralised network.

#LMUMünchen #OpenScience #OpenData #DataRescue
CC @SafeguardingResearch @bitsUndBaeumeAuxMuc

I revived an old HDD with a #RaspberryPi Zero W 2 for #DataRescue:ing:

It runs ...

(a) a Bittorrent client that seeds at-risk data sets from the #SciOp database
(b) the `sciop-scraper` script to get new datasets into the swarm

Setup instructions for the Pi Zero: https://codeberg.org/nicebread/HiveSeed/src/branch/main/L1-RDRN_RPi.md

Setup instructions for `sciop-scrape` (on macOS & RPi): https://codeberg.org/nicebread/HiveSeed/src/branch/main/L1-sciop-scrape.md

Let me know if the instructions work for you; happy to collaborate on the manual.

#safeguardingresearch

Added a 10 Terabyte seeding node to the #SciOp #SafeguardingResearch swarm; focusing on large (> 1 TB) data sets with 0 or 1 seeders.

TIL about <https://sciop.net>, a wonderful science dataset archvial project that is getting more important by the day due to US happenings. Feel free to join and seed!

#safeguardingresearch #sciop

About US research data under threat and how everyone can contribute to saving it - @lavaeolus and me were interviewed in the TU Delft paper Delta: https://delta.tudelft.nl/en/article/saving-academic-data-is-easier-than-you-think-and-you-can-do-it-too mostly regarding our parts in the Safeguarding Research and Culture initiative at https://safeguar.de
#TUDelft #SafeguardingResearch #SciOp

@jonny With the updated commands I got it to run now (with minor modifications) on macOS. On RPi I will try again tomorrow (currently no access to the machine).

I am currently scraping „rp_enchanter_ver02“ with 24 GB and counting. Three questions:

(1) Can I know how large the download will be?
(2) Can I stop the scraping, or will the download then be corrupted?
(3) I assume that after downloading it automatically starts seeding?

Should we keep this conversation on (a) Mastodon, (b) safeguar.de forum or (c) Codeberg issues? Where can most people profit from it?

#safeguardingresearch #SciOp

@jonny this entire thread is amazing, top-notch tool development for a noble cause.
@ #academia : if you feel desperate about the wholesale breakdown of science under the current US administration, consider helping out with #SciOp: Decentralized backups of datasets under threat, in a torrent swarm.

Have a disused laptop or Raspi? Make it part of the swarm and take the data outside the US (or any) administration's grasp!

#scienceunderattack #bittorrent #decentralizedbackup #libraryofcongress

𝒹𝑜𝓃𝓀𝓈

(we have more of them)

#Sciop #WorldPremiere #MusicVideo #LiveEvent #AttendanceInTheBillions #UncountableAttendance #AQuantityOfDocsThatShamesTheHeavens #IfIcarusHadActuallyMadeItToSpaceInsteadOfMeltingThosePunkAssWingsItWouldBeLikeTheseDocs #ActuallyMostlyStillUnfinished #NotThatBigOfAChange #DontClickAnyOtherLinks #IfYouCantTakeMeAtMyPartiallyDocumentedYouDontDeserveMeAtMyFullyDocumented

hehe #sciop #has #achieved #bittorrent #sentience