#fedbox — Public Fediverse posts

Another set of robustness improvements for #GoActivityPub and #FedBOX based on the new integration test suite.

Today I've added proper support for checking blocked actors at all points where that's relevant for the processing and fetching pipelines.

Also I've added "I do not federate" responses for actors created without Inboxes, and similarly for actors without Outboxes.

There's still a couple of bugs remaining for these two topics, but tomorrow is another day...

I'm again amazed how much easier it is to find and fix issues with the new integration test suite for #FedBOX. :goose_hacker:

I'm surprised at how many issues my new integration tests are finding.

I already had an integration testsuite, but the way the #fedbox application was run was not in full isolation, and I expect some of the setup steps I had included don't really exist on a fresh install.

We now build a fresh container image, start it up, provision it with mock data, and then run tests, but only through mechanisms that are available to a prospective server operator: CLI commands executed through SSH, and ActivityPub client to server operations.

#fedidev #activityPubDev #activitypub

Made a lot of progress on the #FedBOX test containers setup.

We now can have moderately complicated testing setups that will soonishly replace the old integration tests.

#ActivityPub

@stefan I'm sorry, but it's the same. You're blinded by the use case that mastodon enshrined in the fediverse.

Instances do not need to be ActivityPub servers, but clients.

If a low resource server handles only the ActivityPub logic and everyone can set one up, and then Mastodon, Pixelfed, Bookwyrm, etc are only applications that the same user can log in to, and uses to generate activities it'll work fully like a PDS.

This is, in fact, what I'm working towards with #GoActivityPub and its reference server implementation #FedBOX.

Well, I've just fixed an annoying CI failure for the new integration tests for #FedBOX.

This one had been driving me mad for at least two months, but it was never a super high priority, as they're not a load bearing part of the test suite yet.

It was all due to a too small timeout on the container initialization. Sheesh...

Resurrected the nodeinfo helper service for the #FedBOX generic #ActivityPub service.

Hopefully this will lead to better integration with other federated services, but I find very little practical use for these hamfisted .well-known APIs integrated into the fediverse...

@evan @deadsuperhero there are also a number of frameworks that have already signalled interest or intent in supporting the API:
#Bonfire #FedBox #Fedify #Emissary

Due to a little impulse from @steve I've improved the documentation for #FedBOX and its adjacent #OAuth2 authorization server.

Currently it lives here: https://mariusor.srht.site/apps/fedbox/authorize/

RE: https://metalhead.club/@mariusor/116058273246682831

Thanks Marius! It was awesome to work with you on the logo! #FedBOX #ActivityPub

@andycarolan did it again and now #FedBOX, the generic #ActivityPub server, has a fresh of the press logo. :) Thank you Andy! 🥳

https://mariusor.srht.site/fedbox/

✅ proxyUrl for #FedBOX on the server side and for #BOX on the client side.

Now I have to port the server bits which (like I said previously) are quite simple to #ONI, where it's more important to have them for me personally.

@steve in #FedBOX I'm using the instance actor's regular inbox as a sharedInbox for the others.

I'm not sure in which way you're thinking it can help with federated vs. local timeline though, as the simplest logic to do that (IMHO) is to verify the activities' actors if they belong to the same instance as the actor that operates the "client". And for that it does not matter where they've been received.

And, in my experience, building timelines in clients can't really be done at viewing time due to inherent slowness in the AP fetches required. The clients need to buffer the collections locally and then they can apply whatever logic on top.

@hongminhee in #FedBOX and #GoActivityPub there's a cascading of behaviour for each ActivityPub object type, so you can call code on a Person/Group/etc, making the assumption that it has the more limited shape of a regular Object.

Since Go doesn't have more fancy types of polymorphism, I had to use some very ugly work arounds to be able to make this work. :( Sometimes I wish I was still working on a more powerful typed language, or a dynamic typed one.

Well, fixing this took way more than it should.

And it's another of those issues where I optimized too early in the life of the #GoActivityPub project and it now bit me in the behind. I split the library into too many modules, which granted are pretty independent functionally, but they create a web of dependencies that is hard to navigate sometimes.

The problem was that one of the modules that I forgot I modified in order make tests pass was not pushed upstream, and all the storage backends depended on it to be fixed.

#go #activitypub #FedBOX

Oh man, why did I ever think that implementing multiple storage backends for my #activitypub service was a good idea?

Having to backport new features across 4 versions is becoming a bit of a pain in the butt I have to say.

#FedBOX #GoActivityPub

It's a painful realization that I come to that no matter how much effort I put into making my #ActivityPub server be fast it's still going to suck if in order to build a meaningful page for a user the client needs to do many requests.

So the #brutalinks link aggregator now makes use of asynchronous collection fetching and then content rendering is being done from local storage.

This decreased the loading times to probably less than half of what they were before.

However I still need to find a good model for aggregating and balancing all the sequential loads with an eventual asynchronous sequential sending of activities.

#FedBOX #GoActivityPub

@naturzukunft I'm actually making heavy use of the filtering capabilities in #FedBOX: https://federated.id/inbox?maxItems=100&object.type=Person&type=Create

@julian that's an acceptable compromise, in fact it's #FedBOX uses both methods for threading. :D

@drahardja @dgoldsmith

@naturzukunft the server (the one I'm most focused on) is called #FedBOX: https://github.com/go-ap/fedbox

As a client I'm working on a link aggregator called #brutalinks: https://github.com/mariusor/brutalinks

I'm sure I made plenty of mistakes, but I have to admit I find it surprisingly satisfying to be able to operate on a data type that I can overlay on top of the existing #FedBOX storage engines and get native and *fast* querying for them.

The indexes are quite chunky despite being built on top of roaring bitmaps because there's so many "indexable" elements in an #ActivityPub object. (Currently I'm indexing the type, the content, summary, name, preferredUsername, the recipients, the actor and the object)

As I explore some more, I hope I streamline some of these issues, and make the whole thing more robust.

Due to the long time I neglected brutalinks.tech, tech debt overwhelmed the server and I had to put it down.

It's currently in maintenance mode for the near-medium future, until I manage to get back to it and bring it in working order.

It's likely that the only people disturbed by this development are random spammers, but still, apologies.

#GoActivityPub #brutalinks #FedBOX

So a quick recap for what #FedBOX and #GoActivityPub adjacent projects I started in the meanwhile:

* a single instance fediverse server meant to simplify how FedBOX works (called ONI). https://git.sr.ht/~mariusor/oni

* a TUI interface for operating admin actions on FedBOX/ONI instances. It's meant to serve as a base for a C2S client for them also. https://git.sr.ht/~mariusor/motley

3/

In #FedBOX land, I have finally made progress in excising the OAuth2 routing to its own individual service.

Deploying has increased complexity, but it paves the path for replacing our spit and duct-tape user management to something more robust, like #ory #kratos.

Leftovers in FedBOX are the OAuth2 client CRUD operations, soon to be replaced with automated client creation.

#activitypub #GoActivityPub

Made the export/import functionality of a #FedBOX server work better, with support for renaming the instance.

This was never a feature meant for production use, but more and more I realize how using json as a format is on the high end of terrible. :D

Before taking a break from ActivityPub at the start of the year I was working on bringing all storage types for #FedBOX (the generic #ActivityPub service) to better support persistence of collections.

Finally over the past weekend I managed to bring the sqlite backend (the last remaining) from ~180/458 failing integration tests to just 10/458.

It was definitely a journey.

The best part of this is that it paves the way for any #Golang sql.DB compatible database.

@laxla that's what I'm trying to do with #FedBOX

@linos my project #FedBOX supports multiple values for the attributedTo property on ActivityPub objects.

The use case this was required for, even though it's not implemented at the moment, was when mods would edit an object that came under scrutiny on the main client that uses FedBOX as a backend, the #brutalinks link aggregator.

Collaborative editing would also work, but there are no clients capable of that at the moment.

@jenniferplusplus In #FedBOX I use the ActivityStreams context property to point to the ID of the top level object, which is nebulously defined in the spec, and if you squint at it in the proper light, it might actually fit the purpose. :D

@helge