#cosocialtechops — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #cosocialtechops, aggregated by home.social.
-
I’ll be hosting a #CoSocialTechOps Working Group meeting on Tuesday, January 20 at 9pm EST.
We’ll discuss the remaining small items that need to be tackled to rid our operations of US companies, and future plans.
This is open to anyone who wants to learn about our ops or has an interest in helping out!
Sign up here: https://luma.com/arjbo97a
-
Config file said: “Generated with mastodon:setup on 2022-12-11 06:31:40 UTC” 🥲
Onward!
-
Deleting S3 buckets is a great way to ring in the new year.
Not taking this stuff with us into 2026.
-
R.I.P. CoSocial Digital Ocean DB Cluster
(Aug 27, 2023 - Dec 22, 2025)You were very reliable, quite fast, and far too expensive.
-
Friday night. Time to sysadmin.
So now that we're on Fedihost and the database on Digital Ocean is no longer of any use, it is just an expensive collection of bits, and we should do away with it.
There is something deep in me that *hates* deleting data of any sort, even when I have duplicate and triplicate backups.
I tell myself this paranoia is useful, but I might just have a problem. 😂
-
Hey #CoSocialCa -
I'll host a TechOps gathering tomorrow evening.
Anyone who can make it is welcome to attend to discuss:
- Our #ElbowsUp plans to move to Canadian (or not-US) hosting
- Mastodon 4.5 upgrade planning
- Future plans
- Etc.Details here: https://luma.com/idn9g2b9
-
Whoooboy that gets the blood flowing.
Digital Ocean DB just completely crapped out while I was taking a backup, and for a moment appeared to be quite badly screwed up before (thankfully) fixing itself.
-
As we prepare for the #CoSocialCa AGM, the #CoSocialTechOps working group is meeting tomorrow evening (8:30-9:00 EST) to discuss our strategy for the coming year.
We’ll review the current state of our operations and our #ElbowsUp initiative - making plans to move our stack from Digital Ocean and AWS to non-US service providers.
New members welcome!
If you’re a #Canadian technologist with an interest in learning about or helping to run #Fediverse services please RSVP: https://luma.com/je2m3phf
-
Migration from 4.3.7 -> 4.4.5 underway.
DB backup taking a bit. The “statuses" table is chonky.
-
Running out of space to store all these posts. Going to upgrade our DB now. If this works as intended, no one will notice a thing.
If it doesn’t… well I’d rather not think about that.
-
No. Unplanned. Downtime.
-
-
-
Time for some more DB space!
Our #CosocialCa Mastodon instance has been steadily growing as it fills up with posts from our members and the broader #Fediverse, and we're getting near the limit of our DB cluster at Digital Ocean.
I'm gonna push some buttons and embiggen our cluster shortly.
If DO does things correctly, no one will notice.
Because I am a responsible (paranoid) sysadmin, I am taking a backup right now and will proceed when it is done.
-
Hey #CoSocialCa:
If you're at all curious to learn how Mastodon works, learning about the tools and tech used by #CoSocialTechops, or how you can help to keep the lights on around here, please join me for a meet-and-greet on Thursday this week.
We'll be live-streaming the upgrade of our Mastodon instance from v4.2.10 to v4.2.12, doing some show-and-tell and Q&A.
Any non-member #MastoAdmin who are interested in helping out are welcome, as well!
-
Our Mastodon DB is growing at a rate of about 1 GB per week. I’ve added another 10 GB to our existing cluster (for the low low price of $2/mo.)
In another couple of months we’ll be forced to move to the next tier of DB hardware - and its associated costs - as the current cluster can’t be increased in size any further.
-
Made some small tweaks to our Sidekiq queues to prepare for your spicy #WWDC takes. Bring it on.
-
-
For the first time the #CoSocialCa Mastodon server has started to struggle just a little bit to keep up with the flow of the Fediverse.
We’ve usually been “push” heavy but we’ve started to see some spikes in “pull” queue latency. The worst of these spikes was today, where we fell behind by at least a couple minutes for most of the afternoon.
1/?
-
@Matt_Noyes right now we’re committed to hosting in Canada.
In practice that means AWS & Digital Ocean Canadian data centres, and OVH bare metal server running Cloudron in Quebec data center.
We could experiment with something on our Cloudron box like Castopod BUT my challenge is we have zero paid staff, so it would be best effort.
Let’s do a dedicated call to kick around some ideas. I’ll start a public thread in our forum.