#mailpile — Public Fediverse posts

One of the things #Mailpile never did well, was processing bounces and other delivery status notifications (DSNs).

I am currently being side-tracked from what I was working on (sending mail), to add the features #moggie needs to do better.

As a start, I just want to recognize the DSN as part of the thread, the error is a reply to the message. Once the two are linked in this way, I can iterate on how they are presented in the UI.

Mutt doesn't appear to do even this much. Do other mail clients?

Watching #moggie copy e-mails out of my old encrypted #Mailpile storage, into a shiny new Maildir-in-a-ZIP is really very satisfying.

It would be nice if it were faster, but in practice this will be such a rare operation that it's probably not worth optimizing. At least not yet!

If I create a relatively simple command that replicates the tags from #Mailpile into a new #moggie setup, then my migration toolchain will be complete!

I moved back to Iceland from the UK a couple of months ago. Am in the UK for a few days, visiting, in our old flat.

There is a parcel! With my name on it! An EE WiFi hub and some other gadget as well? I didn't order this. Called EE - they have no record of sending this, no account in my name. They told me to bin it.

If this were 2014, when #Mailpile was hot shit and #Snowden's revelations all the news and there were [maybe] spies at our community cons, I'd be VERY suspicious right about now...

In #Mailpile v1, I one of the bits of code I spent the most time on, was exactly that - the TLS validation system.

At the time (it has improved since), Python's standard libraries made it almost impossible to make secure outgoing connections.

And TLS issues were hard to troubleshot. And because we had Opinions(tm), we wanted to sometimes use a TOFU policy for certs, and CA-based security at others. And sometimes we wanted to connect over Tor. But not always.

Thus: the Connection Broker. (2/n)

It is very easy to write code that *usually* works with an mbox file.

It is surprisingly tricky to write code that *reliably* works with an #mbox file.

Almost impossible, actually, since there are multiple incompatible variants of the format.

I keep coming back to this code and finding new problems with it. That is a good thing, I guess? Making progress...

#email #moggie #mailpile

@HerraBRE Really cool! Very excited maybe one day hosting mails with https://github.com/mjl-/mox and reading them with #mailpile <3

I just published a new blog post on the #Mailpile website, discussing the grant we are getting from #NLNet and what we are going to do with the money: https://www.mailpile.is/blog/2023-05-01_A_Mail_Client_in_Six_Steps.html

The tl;dr is that we are bringing #moggie from "a thing Bjarni hacks on as often as he can", to "a new terminal mode e-mail client which people should be able to use."

I'm pretty excited about this!

Installing the old #mailpile v1 has become more and more difficult. This is kinda OK, as development there has halted and I don't want new users on that code-base...

But it's also really not OK for those who want to examine Mailpile v1 or worse, have old Mailpile data lying around which they now have trouble accessing.

So today I built and published #Docker images which make it easy to run Mailpile v1 on a modern distro: https://github.com/mailpile/Mailpile-v1-Docker

Feedback is welcome!

@deadsuperhero

Did you give #Mailpile a try?
https://www.mailpile.is/

It has been a good, geeky work-week so far. Not much tootin' been busy.

I did some #devops twiddling for #pagekite. Yay SSL certs?

Made more progress on grant applications for #Mailpile.

Studied #Matter (https://en.wikipedia.org/wiki/Matter_(standard) ) a bit, within the context of #esp32 and #micropython development.

I also had multiple naps and stared out the window a lot going HMM.

Now it's time for another window gazing session, before I dive back into #moggie related #OpenPGP work...

Feeling vaguely proud of the old #Mailpile v1 bayesian #spam filter, which successfully showed me my legit #email from DHL last week, while still sending all the DHL-themed #phishing straight to spam.

This is without using any DNS based blacklists or anything of that nature, just pure content analysis.

Generating keywords that represent message structure and other such things, to better train the spam filter, is one of the more interesting parts of this project. It's fun to see it work!

Having idle thoughts about replication and non-email data (contacts, calendar events) for #moggie.

#Mailpile v1 couldn't replicate. It was a significant limitation which I should probably address this time around.

So many things to think about...

I wrote a long e-mail to #NLNet yesterday, answering questions they have about my #moggie / #mailpile v2 development plans.

How did I come up with X months? It's just how much time I have available! Do I have a roadmap? Not really, but it's in my head? Do I plan collaboration with other projects? Not just yet, I'm being introverted and stealthy still?

Not sure it's what they wanted to hear, but honesty is the only policy. I gave lots of details.

But maybe I *should* publish a better roadmap?

moggie email \
[email protected] \
--subject='Bank stuff' \
--message='Sorry there is so much!' \
--forwarding=inline,html \
--forward='from:bank date:2022' \
|sendmail -t

Dunno if I'm ever going to want to run such a scary command, but I'm very pleased that it works!

Still on the task list: teach it to sign / encrypt, teach it to speak SMTP, teach it to save sent messages locally...

(This becomes an API for writing #email clients)

#moggie #mailpile

... and no, ignoring the HTML part entirely is not actually a viable strategy on today's Internet.

There is quite often content in the HTML part which the user needs to read. It's important. And if it looks wrong and wonky, that's just bad UX.

Trying to stick with a plain-text-only strategy that was one of the worse mistakes we made in #Mailpile, it really hurt our usability and made the app look like an unpolished toy.

asymmetry?

What if I also made it convenient to send (and receive) AES encrypted ZIP files?

What if I made it easy (yes, this would require running helper services) for individuals to use their own keys to DKIM sign their messages? And made #Mailpile aware enough of DKIM to classify messages as "unsigned", "signed by organization" or "signed by author"?

What if I made it easy for users to self-host their own TLS-secured web-based chat-rooms, polls or file shares, and send out links?

What if?

As part of my #Mailpile and #moggie work, I am rethinking a little bit what it really means to offer a "secure e-mail client" to the masses.

I think fundamentally, it has to mean making it easier to use the privacy preserving tech we already have. I'm too small to move markets and create new protocols.

So yes, maybe OpenPGP / Autocrypt. Maybe S/MIME. Maybe some clever Tor side-channels...

But these suffer the problem that all participants need compatible clients.

How about some (...)

... by making #moggie index e-mail message-IDs in a way that is backwards compatible with #Mailpile v1, I now have a very basic tag export/import workflow that works!

I need to refine it a bit, to copy over metadata about the tags themselves, but the basics are in place!

I'm not sure there are enough #Mailpile v1 users out there to justify making this more user-friendly than a HOWTO and some CLI commands. But I'm very pleased with this progress.

... now all I have to do is write code that imports all the tags and e-mails from an old #Mailpile into a new #moggie!

Looking at my #Mailpile metadata, it knows about about 200k e-mail addresses and about 1.1 million e-mails... feels a bit daunting.

One at a time, I guess? Computers are good at that sort of thing.

This week I have been teaching #moggie to decrypt and parse the data stored by #Mailpile v1.

I started trying to port over the old Mailpile encryption code, but got frustrated and really didn't want to keep all that complexity.

So I rewrote just the decryption part yesterday, as cleanly and simply as I could. About 180 lines of new (faster!) code (317 with tests), replacing over 1000 lines before.

Negative LOC counts are the most productive LOC counts (ssh, don't tell Elon).

Woohoo, I updated the #Mailpile website so it reflects the current reality!

Mailpile v1 is dead, long live Mailpile v2?

Next task is to give some guidance to people who still need to run the Mailpile v1 code for some reason.

I think it's finally time to publish an official #docker image - it's the cleanest way I know to get clunky #Python 2 stuff running on modern systems.

That can be the subject of the first how-to blog post...

I need to update #Mailpile's website (https://www.mailpile.is/) and start writing blog posts about #moggie.

I'm thinking this time I will do fewer thinking-about-things posts, and try and mostly post tutorials on how to do things with the new code.

What kind of content do you prefer?

One of the things I worry about for #moggie (#Mailpile 2) is data portability.

Mailpile's mission is still to encourage people to download their e-mail from their cloudy mail accounts and store and process locally. But Mailpile 1 stored e-mail in a custom encrypted format, which means migrating off of Mailpile to another client was a major Pain In The Ankle.

Trying to do better this time around.

I got this done and filed, today!

Here's hoping @NGIZero agree that #Mailpile and #moggie are a good idea. :-D

The deadline hasn't passed (2 days left!), if you are working on something neat, you should also consider asking them for help: https://nlnet.nl/news/2022/20221001-call.html

Time for an #introduction I guess.

I am:

- #Eindhoven (Netherlands) native
- Physicist by training
- Software developer by profession (industrial and medical equipment, mainly #C++, but also #C and #Python)
- Electronics tinkerer (#ESP8266, #D1Mini, #mechanicalkeyboards, #platformio, #kicad)
- Smart home hobbyist (#HomeAssistant, #MQTT, #Tasmota)
- Into #selfhosting (#docker, #mailpile, #pihole)
- 3D printing early adopter (since 2014, #prusa, #marlin, #fusion360)

Time for an #introduction I guess.

I am:

- #Eindhoven native
- Physicist by training
- Software developer by profession (industrial and medical equipment, mainly #C++, but also #C and #Python)
- Electronics tinkerer (#ESP8266, #D1Mini, #mechanicalkeyboards, #platformio, #kicad)
- Smart home hobbyist (#HomeAssistant, #MQTT, #Tasmota)
- Into #selfhosting (#docker, #mailpile, #pihole)
- 3D printing early adopter (since 2014, #prusa, #marlin, #fusion360)

So... the project I am working on the most these days, is called #moggie. Moggie is (will be) #Mailpile v2, an almost-total rewrite in Python 3.

A moggie is a mixed-breed cat; the name is homage to #mutt, which even after all these years is still one of the best #email clients there is. IMHO.

The name is also a challenge and reminder to myself: Throughout Mailpile's development, I kept reaching for mutt when Mailpile fell short, which it inevitably did. I want #moggie to be more like #mutt.

Hello world! An #introduction.

I'm Bjarni, an Icelandic coder who mainly writes Python programs and gives them away as open source. My focus is on autonomy and privacy online; mostly relating to email (#mailpile) and the web (#pagekite).

I'm fluent in security (#passcrow), but I'm more excited by creating neat tools and empowering people, than I am in exploits and encryption algos.

I plan to use this account to discuss my work & hopefully connect with folks working on related things. Say hi!