Search
1000 results for “xmpp_providers”
-
Ever since the Invisible Salamanders paper was published, there has been a quiet renaissance within my friends and colleagues in applied cryptography for studying systems that use Authenticated Encryption with Associated Data (AEAD) constructions, understanding what implicit assumptions these systems make about the guarantees of the AEAD mode they chose to build upon, and the consequences of those assumptions being false.
I’ve discussed Invisible Salamanders several times throughout this blog, from my criticisms of AES-GCM and XMPP + OMEMO to my vulnerability disclosures in Threema.
Five years after Invisible Salamanders, it’s become clear to me that many software developers do not fully appreciate the underlying problem discussed in the Invisible Salamanders paper, even when I share trivial proof-of-concept exploits.
Background
Fast AEAD constructions based on polynomial MACs, such as AES-GCM and ChaCha20-Poly1305, were designed to provide confidentiality and integrity for the plaintext data, and optionally integrity for some additional associated data, in systems where both parties already negotiated one shared symmetric key.
The integrity goals of the systems that adopted these AEAD constructions were often accompanied by performance goals–usually to prevent Denial of Service (DoS) attacks in networking protocols. Verification needed to be very fast and consume minimal resources.
In this sense, AEAD constructions were an incredible success. So successful, in fact, that most cryptographers urge application developers to use one of the fast AEAD modes as the default suggestion without looking deeper at the problem being solved. This is a good thing, because most developers will choose something stupid like ECB mode in the absence of guidance from cryptographers, and AEAD modes are much, much safer than any hand-rolled block cipher modes.
The problem is, that one tiny little assumption that both parties (sender, recipient) for a communication have agreed on exactly one symmetric key for use in the protocol.
Fast MACs Are Not Key-Committing
Cryptographers have concluded that AEAD constructions based on polynomial MACs–while great for performance and rejection of malformed packets without creating DoS risks–tend to make the same assumption. This is even true of misuse-resistant modes like AES-GCM-SIV and extended-nonce constructions like XSalsa20-Poly1305.
When discussing this implicit assumption of only one valid key in the systems that use these AEAD modes, we say that the modes are not key-committing. This terminology is based on what happens when this assumption is false.
Consequently, you can take a single, specially crafted ciphertext (with an authentication tag) and decrypt it under multiple different keys. The authentication tags will be valid for all keys, and the plaintext will be different.
Art: SwizzWhat does this look like in practice?
Consider my GCM exploit, which was written to generate puzzle ciphertexts for the DEFCON Furs badge challenge a few years ago. How it works is conceptually simple (although the actual mechanics behind step 4 is a bit technical):
- Generate two keys.
There’s nothing special about these keys, or their relationship to each other, and can be totally random. They just can’t be identical or the exploit is kind of pointless.
- Encrypt some blocks of plaintext with key1.
- Encrypt some more blocks of plaintext with key2.
- Calculate a collision block from the ciphertext in the previous two steps–which is just a bit of polynomial arithmetic in GF(2^128)
- Return the ciphertext (steps 2, 3, 4) and authentication tag calculated over them (which will collide for both keys).
A system that decrypts the output of this exploit under key1 will see some plaintext, followed by some garbage, followed by 1 final block of garbage.
If the same system decrypts under key2, it will see some garbage, followed by some plaintext, followed by 1 final block of garbage.
For many file formats, this garbage isn’t really a problem. Additionally, a bit more precomputation allows you to choose garbage that will be more advantageous to ensuring both outputs are accepted as “valid” by the target system.
For example, choosing two keys and a targeted nonce may allow both the valid plaintext and garbage blocks to begin with a PDF file header.
If you’re familiar with the file polyglot work of Ange Albertini, you can use this to turn the invisible salamanders problem into an artform.
Why is it called Invisible Salamanders?
The proof-of-concept used in the paper involved sending one picture (of a salamander) over an end-to-end encrypted messaging app, but when the recipient flagged it as abusive, the moderator saw a different picture.
https://www.youtube.com/watch?v=3M1jIO-jLHI
Thus, the salamander was invisible to the moderators of the encrypted messaging app.
As for the choice of a “salamander”, I’ve been told by friends familiar with the research that was inspired by the original name of the Signal Protocol being “Axolotl”.
But, like, who cares about these details besides me? It’s a cute and memorable name.
What are the consequences of violating the “one key” assumption?
That depends entirely on what your system does!
In Database Cryptography Fur the Rest of Us, I discussed the use of AEAD modes to prevent confused deputy attacks. This works great, but if you’re building an application that supports multi-tenancy, you suddenly have to care about this issue again.
An earlier design for OPAQUE, a password authenticated key exchange algorithm, was broken by a partitioning oracle attack due to building atop AEAD modes that are not key-committing. This let an attacker recover passwords from Shadowsocks proxy servers with a complexity similar to a binary search algorithm.
These are two very different impacts from the same weakness, which I believe is a significant factor for why the Invisible Salamanders issue isn’t more widely understood.
Sometimes violating the “one key” assumption that went into fast AEAD modes based on Polynomial MACs completely destroys the security of your system.
Other times, it opens the door for a high-complexity but low-impact behavior that simply violates the principle of least astonishment but doesn’t buy the attacker anything useful.
They Just Don’t Get It
The Invisible Salamanders issue is relevant in any system that uses symmetric-key encryption where more than one key can be valid.
This includes, but is not limited to:
- Multi-tenant data warehouses
- Group messaging protocols
- Envelope encryption schemes with multiple wrapping keys
- Bearer tokens (such as JSON Web Tokens) in systems that utilize Key IDs
Systems can mitigate this issue by introducing an explicit key commitment scheme (based on a cryptographic hash rather than a polynomial MAC) or by using a committing cipher mode (such as AES + HMAC, if done carefully).
However, most of the time, this advice falls on deaf ears whenever this concern is brought up by a cryptography engineer who’s more aware of this issue.
“Abuse reporting? We don’t have no stinking abuse reporting!”
The most common misunderstanding is, “We don’t have a report abuse feature, so this issue doesn’t affect us.”
This is because the Invisible Salamanders talk and paper focused on how it could be leveraged to defeat abuse reporting tools and bypass content moderation.
In my experience, many security teams would read the paper and conclude that it only impacts abuse reporting features and not potentially all systems that allow multiple symmetric keys in a given context.
Another Exploit Scenario
Imagine you’re building a Data Loss Prevention product that integrates with corporate file-sharing and collaboration software (e.g. ownCloud) for small and medium businesses.
One day, someone decides to ship an end-to-end encryption feature to the file-sharing software that uses AES-GCM to encrypt files, and then encrypts the keys to each recipient’s public key. This is basically the envelope encryption use-case above.
In order to update your integration to act as another “user”, whose public key must be included in all E2EE transfers, and will block download of ciphertexts it cannot decrypt OR contains sensitive information.
And this works, until an insider threat clever enough to abuse the Invisible Salamanders issue comes along.
In order for said insider threat (e.g., a senior business analyst) to leak sensitive data (e.g., anything that would be useful for illegal insider trading) to another person that shouldn’t have access to it (e.g., a store clerk that’s talking to the press), they just have to do this:
- Encrypt the data they want to exfiltrate using key1.
- Encrypt some innocuous data that won’t trigger your DLP product, using key2.
- Ensure that both messages encrypt to the same ciphertext and authentication tag.
- Give their recipient key1, give everyone else (including your DLP software) key2.
Bam! File leaked, and everyone’s none the wiser, until it’s too late. Let’s actually imagine what happens next:
A random store clerk has leaked sensitive data to the press that only a few analysts had access to.
The only communication between the analyst and the store clerk is a file that was shared to all employees, using the E2EE protocol. No emails or anything else were identified.
Your DLP product didn’t identify any other communications between these two, but somehow the store clerk has the data on their desktop.
A detailed forensics analysis may eventually figure out what happened, but by then, the damage is done and your product’s reputation is irrecoverably damaged.
All because the hypothetical E2EE protocol didn’t include a key-commitment mechanism, and nobody identified this deficit in their designs.
This isn’t to endorse DLP solutions at all, but rather, to highlight one of the many ways that the Invisible Salamander issue can be used creatively by clever attackers.
Art: AJThe Lesson to Learn
If you’re building a network protocol that uses AEAD to encrypt data over an insecure network (e.g., WireGuard), keep up the good work.
If you’re doing anything more involved than that, at the application layer, pause for a moment and consider whether your system will ever need multiple valid symmetric keys at once.
And, if the answer is “yes”, then you should always explicitly add a key-commitment mechanism to your system design.
(Hire a cryptographer if you’re not sure how to proceed.)
In my opinion, hemming and hawing over whether there’s a significant impact to the Invisible Salamanders issue is a worse use of your time than just solving it directly.
Eventually, I expect a new generation of AEAD modes will be standardized that explicitly provide key-commitment.
When these new designs are standardized, widely supported, and sufficiently trusted by experts, feel free to update my advice to “prefer using those modes” instead.
Header art: Harubaki, CMYKat, and Brian Gratwicke
https://soatok.blog/2024/09/10/invisible-salamanders-are-not-what-you-think/
#AEAD #AESGCM #InvisibleSalamanders #randomKeyRobustness #symmetricCryptography
- Generate two keys.
-
Ever since the Invisible Salamanders paper was published, there has been a quiet renaissance within my friends and colleagues in applied cryptography for studying systems that use Authenticated Encryption with Associated Data (AEAD) constructions, understanding what implicit assumptions these systems make about the guarantees of the AEAD mode they chose to build upon, and the consequences of those assumptions being false.
I’ve discussed Invisible Salamanders several times throughout this blog, from my criticisms of AES-GCM and XMPP + OMEMO to my vulnerability disclosures in Threema.
Five years after Invisible Salamanders, it’s become clear to me that many software developers do not fully appreciate the underlying problem discussed in the Invisible Salamanders paper, even when I share trivial proof-of-concept exploits.
Background
Fast AEAD constructions based on polynomial MACs, such as AES-GCM and ChaCha20-Poly1305, were designed to provide confidentiality and integrity for the plaintext data, and optionally integrity for some additional associated data, in systems where both parties already negotiated one shared symmetric key.
The integrity goals of the systems that adopted these AEAD constructions were often accompanied by performance goals–usually to prevent Denial of Service (DoS) attacks in networking protocols. Verification needed to be very fast and consume minimal resources.
In this sense, AEAD constructions were an incredible success. So successful, in fact, that most cryptographers urge application developers to use one of the fast AEAD modes as the default suggestion without looking deeper at the problem being solved. This is a good thing, because most developers will choose something stupid like ECB mode in the absence of guidance from cryptographers, and AEAD modes are much, much safer than any hand-rolled block cipher modes.
The problem is, that one tiny little assumption that both parties (sender, recipient) for a communication have agreed on exactly one symmetric key for use in the protocol.
Fast MACs Are Not Key-Committing
Cryptographers have concluded that AEAD constructions based on polynomial MACs–while great for performance and rejection of malformed packets without creating DoS risks–tend to make the same assumption. This is even true of misuse-resistant modes like AES-GCM-SIV and extended-nonce constructions like XSalsa20-Poly1305.
When discussing this implicit assumption of only one valid key in the systems that use these AEAD modes, we say that the modes are not key-committing. This terminology is based on what happens when this assumption is false.
Consequently, you can take a single, specially crafted ciphertext (with an authentication tag) and decrypt it under multiple different keys. The authentication tags will be valid for all keys, and the plaintext will be different.
Art: SwizzWhat does this look like in practice?
Consider my GCM exploit, which was written to generate puzzle ciphertexts for the DEFCON Furs badge challenge a few years ago. How it works is conceptually simple (although the actual mechanics behind step 4 is a bit technical):
- Generate two keys.
There’s nothing special about these keys, or their relationship to each other, and can be totally random. They just can’t be identical or the exploit is kind of pointless.
- Encrypt some blocks of plaintext with key1.
- Encrypt some more blocks of plaintext with key2.
- Calculate a collision block from the ciphertext in the previous two steps–which is just a bit of polynomial arithmetic in GF(2^128)
- Return the ciphertext (steps 2, 3, 4) and authentication tag calculated over them (which will collide for both keys).
A system that decrypts the output of this exploit under key1 will see some plaintext, followed by some garbage, followed by 1 final block of garbage.
If the same system decrypts under key2, it will see some garbage, followed by some plaintext, followed by 1 final block of garbage.
For many file formats, this garbage isn’t really a problem. Additionally, a bit more precomputation allows you to choose garbage that will be more advantageous to ensuring both outputs are accepted as “valid” by the target system.
For example, choosing two keys and a targeted nonce may allow both the valid plaintext and garbage blocks to begin with a PDF file header.
If you’re familiar with the file polyglot work of Ange Albertini, you can use this to turn the Invisible Salamanders problem into an artform.
And this is just the simple attack!
The Invisible Salamanders paper outlined a more advanced variant (with a proof of concept) in Section 3.2, which doesn’t suffer from nearly as much garbage data as the simple attack.
As Bruce Schneier often says, “Attacks only get better, they never get worse.”
Why is it called Invisible Salamanders?
The proof-of-concept used in the paper involved sending one picture (of a salamander) over an end-to-end encrypted messaging app, but when the recipient flagged it as abusive, the moderator saw a different picture.
https://www.youtube.com/watch?v=3M1jIO-jLHI
Thus, the salamander was invisible to the moderators of the encrypted messaging app.
As for the choice of a “salamander”, I’ve been told by friends familiar with the research that was inspired by the original name of the Signal Protocol being “Axolotl”.
But, like, who cares about these details besides me? It’s a cute and memorable name.
What are the consequences of violating the “one key” assumption?
That depends entirely on what your system does!
In Database Cryptography Fur the Rest of Us, I discussed the use of AEAD modes to prevent confused deputy attacks. This works great, but if you’re building an application that supports multi-tenancy, you suddenly have to care about this issue again.
An earlier design for OPAQUE, a password authenticated key exchange algorithm, was broken by a partitioning oracle attack due to building atop AEAD modes that are not key-committing. This let an attacker recover passwords from Shadowsocks proxy servers with a complexity similar to a binary search algorithm.
These are two very different impacts from the same weakness, which I believe is a significant factor for why the Invisible Salamanders issue isn’t more widely understood.
Sometimes violating the “one key” assumption that went into fast AEAD modes based on Polynomial MACs completely destroys the security of your system.
Other times, it opens the door for a high-complexity but low-impact behavior that simply violates the principle of least astonishment but doesn’t buy the attacker anything useful.
They Just Don’t Get It
The Invisible Salamanders issue is relevant in any system that uses symmetric-key encryption where more than one key can be valid.
This includes, but is not limited to:
- Multi-tenant data warehouses
- Group messaging protocols
- It’s sometimes tempting to discount group messaging as a relevant consideration if your experience is “emulated groups atop 1-to-1 messaging”, but there are protocols that establish a Group Key (i.e., RFC 9420) and then use that for all group messages.
- Envelope encryption schemes with multiple wrapping keys
- Bearer tokens (such as JSON Web Tokens) in systems that utilize Key IDs
Systems can mitigate this issue by introducing an explicit key commitment scheme (based on a cryptographic hash rather than a polynomial MAC) or by using a committing cipher mode (such as AES + HMAC, if done carefully).
However, most of the time, this advice falls on deaf ears whenever this concern is brought up by a cryptography engineer who’s more aware of this issue.
“Abuse reporting? We don’t have no stinking abuse reporting!”
The most common misunderstanding is, “We don’t have a report abuse feature, so this issue doesn’t affect us.”
This is because the Invisible Salamanders talk and paper focused on how it could be leveraged to defeat abuse reporting tools and bypass content moderation.
In my experience, many security teams would read the paper and conclude that it only impacts abuse reporting features and not potentially all systems that allow multiple symmetric keys in a given context.
Another Exploit Scenario
Imagine you’re building a Data Loss Prevention product that integrates with corporate file-sharing and collaboration software (e.g. ownCloud) for small and medium businesses.
One day, someone decides to ship an end-to-end encryption feature to the file-sharing software that uses AES-GCM to encrypt files, and then encrypts the keys to each recipient’s public key. This is basically the envelope encryption use-case above.
So, you dutifully update your integration to act as another “user”, whose public key must be included in all E2EE transfers, and will block download of ciphertexts it cannot decrypt OR contains sensitive information.
And this works, until an insider threat clever enough to abuse the Invisible Salamanders issue comes along.
In order for said insider threat (e.g., a senior business analyst) to leak sensitive data (e.g., anything that would be useful for illegal insider trading) to another person that shouldn’t have access to it (e.g., a store clerk that’s talking to the press), they just have to do this:
- Encrypt the data they want to exfiltrate using key1.
- Encrypt some innocuous data that won’t trigger your DLP product, using key2.
- Ensure that both messages encrypt to the same ciphertext and authentication tag.
- Give their recipient key1, give everyone else (including your DLP software) key2.
Bam! File leaked, and everyone’s none the wiser, until it’s too late. Let’s actually imagine what happens next:
A random store clerk has leaked sensitive data to the press that only a few analysts had access to.
The only communication between the analyst and the store clerk is a file that was shared to all employees, using the E2EE protocol. No emails or anything else were identified.
Your DLP product didn’t identify any other communications between these two, but somehow the store clerk has the data on their desktop.
A detailed forensics analysis may eventually figure out what happened, but by then, the damage is done and your product’s reputation is irrecoverably damaged.
All because the hypothetical E2EE protocol didn’t include a key-commitment mechanism, and nobody identified this deficit in their designs.
This isn’t to endorse DLP solutions at all, but rather, to highlight one of the many ways that the Invisible Salamander issue can be used creatively by clever attackers.
Art: AJ“Couldn’t you do the same with steganography?”
No, the attack is very different from stego.
Stego is about hiding a message in plain sight, so that only the person that knows where/how to look can find it.
The Invisible Salamanders attack lets you send one ciphertext through a network then selectively decrypt it to one of two plaintexts, depending on which key you reveal to each participant.
In the Invisible Salamanders paper and talk, they used this to send “abusive” messages to a recipient that the moderator would not see. Thus, invisible.
In one, the message is always emitted to anyone who knows how to find it. In the other, the attacker selects which you see, even if you have mechanisms to ensure you’re seeing the same ciphertext. It’s not a subtle difference.
Mitigation Techniques
There are multiple ways to mitigate the risk of Invisible Salamanders in a cryptosystem.
- Use HMAC, or (failing that) something built atop cryptographic hash functions, rather than a Polynomial MAC.
- Use an AEAD cipher designed with multi-recipient integrity as a security goal.
- Compute a non-invertible, one-way commitment of the encryption key.
A trivial mitigation looks like this:
class SoatokExampleEncryptor { const NEW_ENCRYPT_KEY = 'myProtocol$encryptKey'; const NEW_COMMITMENT = 'myProtocol$commitment'; public function __construct(#[SensitiveParameter] private string $key) {} /** * Let's assume we're starting with a simple AES-GCM wrapper */ public function legacyEncrypt(string $plaintext, string $assocData = ''): string { $nonce = random_bytes(12); $tag = ''; $ciphertext = openssl_encrypt( $plaintext, 'aes-256-gcm', $this->key, OPENSSL_RAW_DATA, $nonce, $tag, $assocData ); return $nonce . $ciphertext . $tag; } /** * An improved function looks something like this */ public function newEncrypt(string $plaintext, string $assocData = ''): string { // Avoid birthday bound issues with 256-bits of randomness $longerNonce = random_bytes(32); // Derive a subkey and synthetic nonce $tmp = hash_hkdf('sha512', $this->key, 44, self::NEW_ENCRYPT_KEY . $longerNonce); $encKey = substr($tmp, 0, 32); $nonce = substr($tmp, 32); // New: Key commitment $commitment = hash_hkdf('sha512', $this->key, 32, self::NEW_COMMITMENT . $longerNonce); // Most of this is unchanged $tag = ''; $ciphertext = openssl_encrypt( $plaintext, 'aes-256-gcm', $encKey, OPENSSL_RAW_DATA, $nonce, $tag, $assocData ); return $longerNonce . $commitment . $ciphertext . $tag; }}And then the decryption logic would recalculate the commitment, and compare it with the stored value, in constant-time.
It’s important that the commitment be stored with the ciphertext, rather than bundling it with the key.
(It may be worthwhile to also include the commitment in the associated data, to add a mechanism against downgrade attacks.)
The Lesson to Learn
If you’re building a network protocol that uses AEAD to encrypt data over an insecure network (e.g., WireGuard), keep up the good work.
If you’re doing anything more involved than that, at the application layer, pause for a moment and consider whether your system will ever need multiple valid symmetric keys at once.
And, if the answer is “yes”, then you should always explicitly add a key-commitment mechanism to your system design.
(Hire a cryptographer if you’re not sure how to proceed.)
In my opinion, hemming and hawing over whether there’s a significant impact to the Invisible Salamanders issue is a worse use of your time than just solving it directly.
Eventually, I expect a new generation of AEAD modes will be standardized that explicitly provide key-commitment.
When these new designs are standardized, widely supported, and sufficiently trusted by experts, feel free to update my advice to “prefer using those modes” instead.
Header art: Harubaki, CMYKat, and a photo by Brian Gratwicke. Poorly photoshopped by myself.
https://soatok.blog/2024/09/10/invisible-salamanders-are-not-what-you-think/
#AEAD #AESGCM #InvisibleSalamanders #randomKeyRobustness #symmetricCryptography
- Generate two keys.
-
Ever since the Invisible Salamanders paper was published, there has been a quiet renaissance within my friends and colleagues in applied cryptography for studying systems that use Authenticated Encryption with Associated Data (AEAD) constructions, understanding what implicit assumptions these systems make about the guarantees of the AEAD mode they chose to build upon, and the consequences of those assumptions being false.
I’ve discussed Invisible Salamanders several times throughout this blog, from my criticisms of AES-GCM and XMPP + OMEMO to my vulnerability disclosures in Threema.
Five years after Invisible Salamanders, it’s become clear to me that many software developers do not fully appreciate the underlying problem discussed in the Invisible Salamanders paper, even when I share trivial proof-of-concept exploits.
Background
Fast AEAD constructions based on polynomial MACs, such as AES-GCM and ChaCha20-Poly1305, were designed to provide confidentiality and integrity for the plaintext data, and optionally integrity for some additional associated data, in systems where both parties already negotiated one shared symmetric key.
The integrity goals of the systems that adopted these AEAD constructions were often accompanied by performance goals–usually to prevent Denial of Service (DoS) attacks in networking protocols. Verification needed to be very fast and consume minimal resources.
In this sense, AEAD constructions were an incredible success. So successful, in fact, that most cryptographers urge application developers to use one of the fast AEAD modes as the default suggestion without looking deeper at the problem being solved. This is a good thing, because most developers will choose something stupid like ECB mode in the absence of guidance from cryptographers, and AEAD modes are much, much safer than any hand-rolled block cipher modes.
The problem is, that one tiny little assumption that both parties (sender, recipient) for a communication have agreed on exactly one symmetric key for use in the protocol.
Fast MACs Are Not Key-Committing
Cryptographers have concluded that AEAD constructions based on polynomial MACs–while great for performance and rejection of malformed packets without creating DoS risks–tend to make the same assumption. This is even true of misuse-resistant modes like AES-GCM-SIV and extended-nonce constructions like XSalsa20-Poly1305.
When discussing this implicit assumption of only one valid key in the systems that use these AEAD modes, we say that the modes are not key-committing. This terminology is based on what happens when this assumption is false.
Consequently, you can take a single, specially crafted ciphertext (with an authentication tag) and decrypt it under multiple different keys. The authentication tags will be valid for all keys, and the plaintext will be different.
Art: SwizzWhat does this look like in practice?
Consider my GCM exploit, which was written to generate puzzle ciphertexts for the DEFCON Furs badge challenge a few years ago. How it works is conceptually simple (although the actual mechanics behind step 4 is a bit technical):
- Generate two keys.
There’s nothing special about these keys, or their relationship to each other, and can be totally random. They just can’t be identical or the exploit is kind of pointless.
- Encrypt some blocks of plaintext with key1.
- Encrypt some more blocks of plaintext with key2.
- Calculate a collision block from the ciphertext in the previous two steps–which is just a bit of polynomial arithmetic in GF(2^128)
- Return the ciphertext (steps 2, 3, 4) and authentication tag calculated over them (which will collide for both keys).
A system that decrypts the output of this exploit under key1 will see some plaintext, followed by some garbage, followed by 1 final block of garbage.
If the same system decrypts under key2, it will see some garbage, followed by some plaintext, followed by 1 final block of garbage.
For many file formats, this garbage isn’t really a problem. Additionally, a bit more precomputation allows you to choose garbage that will be more advantageous to ensuring both outputs are accepted as “valid” by the target system.
For example, choosing two keys and a targeted nonce may allow both the valid plaintext and garbage blocks to begin with a PDF file header.
If you’re familiar with the file polyglot work of Ange Albertini, you can use this to turn the Invisible Salamanders problem into an artform.
And this is just the simple attack!
The Invisible Salamanders paper outlined a more advanced variant (with a proof of concept) in Section 3.2, which doesn’t suffer from nearly as much garbage data as the simple attack.
As Bruce Schneier often says, “Attacks only get better, they never get worse.”
Why is it called Invisible Salamanders?
The proof-of-concept used in the paper involved sending one picture (of a salamander) over an end-to-end encrypted messaging app, but when the recipient flagged it as abusive, the moderator saw a different picture.
https://www.youtube.com/watch?v=3M1jIO-jLHI
Thus, the salamander was invisible to the moderators of the encrypted messaging app.
As for the choice of a “salamander”, I’ve been told by friends familiar with the research that was inspired by the original name of the Signal Protocol being “Axolotl”.
But, like, who cares about these details besides me? It’s a cute and memorable name.
What are the consequences of violating the “one key” assumption?
That depends entirely on what your system does!
In Database Cryptography Fur the Rest of Us, I discussed the use of AEAD modes to prevent confused deputy attacks. This works great, but if you’re building an application that supports multi-tenancy, you suddenly have to care about this issue again.
An earlier design for OPAQUE, a password authenticated key exchange algorithm, was broken by a partitioning oracle attack due to building atop AEAD modes that are not key-committing. This let an attacker recover passwords from Shadowsocks proxy servers with a complexity similar to a binary search algorithm.
These are two very different impacts from the same weakness, which I believe is a significant factor for why the Invisible Salamanders issue isn’t more widely understood.
Sometimes violating the “one key” assumption that went into fast AEAD modes based on Polynomial MACs completely destroys the security of your system.
Other times, it opens the door for a high-complexity but low-impact behavior that simply violates the principle of least astonishment but doesn’t buy the attacker anything useful.
They Just Don’t Get It
The Invisible Salamanders issue is relevant in any system that uses symmetric-key encryption where more than one key can be valid.
This includes, but is not limited to:
- Multi-tenant data warehouses
- Group messaging protocols
- It’s sometimes tempting to discount group messaging as a relevant consideration if your experience is “emulated groups atop 1-to-1 messaging”, but there are protocols that establish a Group Key (i.e., RFC 9420) and then use that for all group messages.
- Envelope encryption schemes with multiple wrapping keys
- Bearer tokens (such as JSON Web Tokens) in systems that utilize Key IDs
Systems can mitigate this issue by introducing an explicit key commitment scheme (based on a cryptographic hash rather than a polynomial MAC) or by using a committing cipher mode (such as AES + HMAC, if done carefully).
However, most of the time, this advice falls on deaf ears whenever this concern is brought up by a cryptography engineer who’s more aware of this issue.
“Abuse reporting? We don’t have no stinking abuse reporting!”
The most common misunderstanding is, “We don’t have a report abuse feature, so this issue doesn’t affect us.”
This is because the Invisible Salamanders talk and paper focused on how it could be leveraged to defeat abuse reporting tools and bypass content moderation.
In my experience, many security teams would read the paper and conclude that it only impacts abuse reporting features and not potentially all systems that allow multiple symmetric keys in a given context.
Another Exploit Scenario
Imagine you’re building a Data Loss Prevention product that integrates with corporate file-sharing and collaboration software (e.g. ownCloud) for small and medium businesses.
One day, someone decides to ship an end-to-end encryption feature to the file-sharing software that uses AES-GCM to encrypt files, and then encrypts the keys to each recipient’s public key. This is basically the envelope encryption use-case above.
So, you dutifully update your integration to act as another “user”, whose public key must be included in all E2EE transfers, and will block download of ciphertexts it cannot decrypt OR contains sensitive information.
And this works, until an insider threat clever enough to abuse the Invisible Salamanders issue comes along.
In order for said insider threat (e.g., a senior business analyst) to leak sensitive data (e.g., anything that would be useful for illegal insider trading) to another person that shouldn’t have access to it (e.g., a store clerk that’s talking to the press), they just have to do this:
- Encrypt the data they want to exfiltrate using key1.
- Encrypt some innocuous data that won’t trigger your DLP product, using key2.
- Ensure that both messages encrypt to the same ciphertext and authentication tag.
- Give their recipient key1, give everyone else (including your DLP software) key2.
Bam! File leaked, and everyone’s none the wiser, until it’s too late. Let’s actually imagine what happens next:
A random store clerk has leaked sensitive data to the press that only a few analysts had access to.
The only communication between the analyst and the store clerk is a file that was shared to all employees, using the E2EE protocol. No emails or anything else were identified.
Your DLP product didn’t identify any other communications between these two, but somehow the store clerk has the data on their desktop.
A detailed forensics analysis may eventually figure out what happened, but by then, the damage is done and your product’s reputation is irrecoverably damaged.
All because the hypothetical E2EE protocol didn’t include a key-commitment mechanism, and nobody identified this deficit in their designs.
This isn’t to endorse DLP solutions at all, but rather, to highlight one of the many ways that the Invisible Salamander issue can be used creatively by clever attackers.
Art: AJ“Couldn’t you do the same with steganography?”
No, the attack is very different from stego.
Stego is about hiding a message in plain sight, so that only the person that knows where/how to look can find it.
The Invisible Salamanders attack lets you send one ciphertext through a network then selectively decrypt it to one of two plaintexts, depending on which key you reveal to each participant.
In the Invisible Salamanders paper and talk, they used this to send “abusive” messages to a recipient that the moderator would not see. Thus, invisible.
In one, the message is always emitted to anyone who knows how to find it. In the other, the attacker selects which you see, even if you have mechanisms to ensure you’re seeing the same ciphertext. It’s not a subtle difference.
Mitigation Techniques
There are multiple ways to mitigate the risk of Invisible Salamanders in a cryptosystem.
- Use HMAC, or (failing that) something built atop cryptographic hash functions, rather than a Polynomial MAC.
- Use an AEAD cipher designed with multi-recipient integrity as a security goal.
- Compute a non-invertible, one-way commitment of the encryption key.
A trivial mitigation looks like this:
class SoatokExampleEncryptor { const NEW_ENCRYPT_KEY = 'myProtocol$encryptKey'; const NEW_COMMITMENT = 'myProtocol$commitment'; public function __construct(#[SensitiveParameter] private string $key) {} /** * Let's assume we're starting with a simple AES-GCM wrapper */ public function legacyEncrypt(string $plaintext, string $assocData = ''): string { $nonce = random_bytes(12); $tag = ''; $ciphertext = openssl_encrypt( $plaintext, 'aes-256-gcm', $this->key, OPENSSL_RAW_DATA, $nonce, $tag, $assocData ); return $nonce . $ciphertext . $tag; } /** * An improved function looks something like this */ public function newEncrypt(string $plaintext, string $assocData = ''): string { // Avoid birthday bound issues with 256-bits of randomness $longerNonce = random_bytes(32); // Derive a subkey and synthetic nonce $tmp = hash_hkdf('sha512', $this->key, 44, self::NEW_ENCRYPT_KEY . $longerNonce); $encKey = substr($tmp, 0, 32); $nonce = substr($tmp, 32); // New: Key commitment $commitment = hash_hkdf('sha512', $this->key, 32, self::NEW_COMMITMENT . $longerNonce); // Most of this is unchanged $tag = ''; $ciphertext = openssl_encrypt( $plaintext, 'aes-256-gcm', $encKey, OPENSSL_RAW_DATA, $nonce, $tag, $assocData ); return $longerNonce . $commitment . $ciphertext . $tag; }}And then the decryption logic would recalculate the commitment, and compare it with the stored value, in constant-time.
It’s important that the commitment be stored with the ciphertext, rather than bundling it with the key.
(It may be worthwhile to also include the commitment in the associated data, to add a mechanism against downgrade attacks.)
The Lesson to Learn
If you’re building a network protocol that uses AEAD to encrypt data over an insecure network (e.g., WireGuard), keep up the good work.
If you’re doing anything more involved than that, at the application layer, pause for a moment and consider whether your system will ever need multiple valid symmetric keys at once.
And, if the answer is “yes”, then you should always explicitly add a key-commitment mechanism to your system design.
(Hire a cryptographer if you’re not sure how to proceed.)
In my opinion, hemming and hawing over whether there’s a significant impact to the Invisible Salamanders issue is a worse use of your time than just solving it directly.
Eventually, I expect a new generation of AEAD modes will be standardized that explicitly provide key-commitment.
When these new designs are standardized, widely supported, and sufficiently trusted by experts, feel free to update my advice to “prefer using those modes” instead.
Header art: Harubaki, CMYKat, and a photo by Brian Gratwicke. Poorly photoshopped by myself.
https://soatok.blog/2024/09/10/invisible-salamanders-are-not-what-you-think/
#AEAD #AESGCM #InvisibleSalamanders #randomKeyRobustness #symmetricCryptography
- Generate two keys.
-
Ever since the Invisible Salamanders paper was published, there has been a quiet renaissance within my friends and colleagues in applied cryptography for studying systems that use Authenticated Encryption with Associated Data (AEAD) constructions, understanding what implicit assumptions these systems make about the guarantees of the AEAD mode they chose to build upon, and the consequences of those assumptions being false.
I’ve discussed Invisible Salamanders several times throughout this blog, from my criticisms of AES-GCM and XMPP + OMEMO to my vulnerability disclosures in Threema.
Five years after Invisible Salamanders, it’s become clear to me that many software developers do not fully appreciate the underlying problem discussed in the Invisible Salamanders paper, even when I share trivial proof-of-concept exploits.
Background
Fast AEAD constructions based on polynomial MACs, such as AES-GCM and ChaCha20-Poly1305, were designed to provide confidentiality and integrity for the plaintext data, and optionally integrity for some additional associated data, in systems where both parties already negotiated one shared symmetric key.
The integrity goals of the systems that adopted these AEAD constructions were often accompanied by performance goals–usually to prevent Denial of Service (DoS) attacks in networking protocols. Verification needed to be very fast and consume minimal resources.
In this sense, AEAD constructions were an incredible success. So successful, in fact, that most cryptographers urge application developers to use one of the fast AEAD modes as the default suggestion without looking deeper at the problem being solved. This is a good thing, because most developers will choose something stupid like ECB mode in the absence of guidance from cryptographers, and AEAD modes are much, much safer than any hand-rolled block cipher modes.
The problem is, that one tiny little assumption that both parties (sender, recipient) for a communication have agreed on exactly one symmetric key for use in the protocol.
Fast MACs Are Not Key-Committing
Cryptographers have concluded that AEAD constructions based on polynomial MACs–while great for performance and rejection of malformed packets without creating DoS risks–tend to make the same assumption. This is even true of misuse-resistant modes like AES-GCM-SIV and extended-nonce constructions like XSalsa20-Poly1305.
When discussing this implicit assumption of only one valid key in the systems that use these AEAD modes, we say that the modes are not key-committing. This terminology is based on what happens when this assumption is false.
Consequently, you can take a single, specially crafted ciphertext (with an authentication tag) and decrypt it under multiple different keys. The authentication tags will be valid for all keys, and the plaintext will be different.
Art: SwizzWhat does this look like in practice?
Consider my GCM exploit, which was written to generate puzzle ciphertexts for the DEFCON Furs badge challenge a few years ago. How it works is conceptually simple (although the actual mechanics behind step 4 is a bit technical):
- Generate two keys.
There’s nothing special about these keys, or their relationship to each other, and can be totally random. They just can’t be identical or the exploit is kind of pointless.
- Encrypt some blocks of plaintext with key1.
- Encrypt some more blocks of plaintext with key2.
- Calculate a collision block from the ciphertext in the previous two steps–which is just a bit of polynomial arithmetic in GF(2^128)
- Return the ciphertext (steps 2, 3, 4) and authentication tag calculated over them (which will collide for both keys).
A system that decrypts the output of this exploit under key1 will see some plaintext, followed by some garbage, followed by 1 final block of garbage.
If the same system decrypts under key2, it will see some garbage, followed by some plaintext, followed by 1 final block of garbage.
For many file formats, this garbage isn’t really a problem. Additionally, a bit more precomputation allows you to choose garbage that will be more advantageous to ensuring both outputs are accepted as “valid” by the target system.
For example, choosing two keys and a targeted nonce may allow both the valid plaintext and garbage blocks to begin with a PDF file header.
If you’re familiar with the file polyglot work of Ange Albertini, you can use this to turn the invisible salamanders problem into an artform.
Why is it called Invisible Salamanders?
The proof-of-concept used in the paper involved sending one picture (of a salamander) over an end-to-end encrypted messaging app, but when the recipient flagged it as abusive, the moderator saw a different picture.
https://www.youtube.com/watch?v=3M1jIO-jLHI
Thus, the salamander was invisible to the moderators of the encrypted messaging app.
As for the choice of a “salamander”, I’ve been told by friends familiar with the research that was inspired by the original name of the Signal Protocol being “Axolotl”.
But, like, who cares about these details besides me? It’s a cute and memorable name.
What are the consequences of violating the “one key” assumption?
That depends entirely on what your system does!
In Database Cryptography Fur the Rest of Us, I discussed the use of AEAD modes to prevent confused deputy attacks. This works great, but if you’re building an application that supports multi-tenancy, you suddenly have to care about this issue again.
An earlier design for OPAQUE, a password authenticated key exchange algorithm, was broken by a partitioning oracle attack due to building atop AEAD modes that are not key-committing. This let an attacker recover passwords from Shadowsocks proxy servers with a complexity similar to a binary search algorithm.
These are two very different impacts from the same weakness, which I believe is a significant factor for why the Invisible Salamanders issue isn’t more widely understood.
Sometimes violating the “one key” assumption that went into fast AEAD modes based on Polynomial MACs completely destroys the security of your system.
Other times, it opens the door for a high-complexity but low-impact behavior that simply violates the principle of least astonishment but doesn’t buy the attacker anything useful.
They Just Don’t Get It
The Invisible Salamanders issue is relevant in any system that uses symmetric-key encryption where more than one key can be valid.
This includes, but is not limited to:
- Multi-tenant data warehouses
- Group messaging protocols
- Envelope encryption schemes with multiple wrapping keys
- Bearer tokens (such as JSON Web Tokens) in systems that utilize Key IDs
Systems can mitigate this issue by introducing an explicit key commitment scheme (based on a cryptographic hash rather than a polynomial MAC) or by using a committing cipher mode (such as AES + HMAC, if done carefully).
However, most of the time, this advice falls on deaf ears whenever this concern is brought up by a cryptography engineer who’s more aware of this issue.
“Abuse reporting? We don’t have no stinking abuse reporting!”
The most common misunderstanding is, “We don’t have a report abuse feature, so this issue doesn’t affect us.”
This is because the Invisible Salamanders talk and paper focused on how it could be leveraged to defeat abuse reporting tools and bypass content moderation.
In my experience, many security teams would read the paper and conclude that it only impacts abuse reporting features and not potentially all systems that allow multiple symmetric keys in a given context.
Another Exploit Scenario
Imagine you’re building a Data Loss Prevention product that integrates with corporate file-sharing and collaboration software (e.g. ownCloud) for small and medium businesses.
One day, someone decides to ship an end-to-end encryption feature to the file-sharing software that uses AES-GCM to encrypt files, and then encrypts the keys to each recipient’s public key. This is basically the envelope encryption use-case above.
In order to update your integration to act as another “user”, whose public key must be included in all E2EE transfers, and will block download of ciphertexts it cannot decrypt OR contains sensitive information.
And this works, until an insider threat clever enough to abuse the Invisible Salamanders issue comes along.
In order for said insider threat (e.g., a senior business analyst) to leak sensitive data (e.g., anything that would be useful for illegal insider trading) to another person that shouldn’t have access to it (e.g., a store clerk that’s talking to the press), they just have to do this:
- Encrypt the data they want to exfiltrate using key1.
- Encrypt some innocuous data that won’t trigger your DLP product, using key2.
- Ensure that both messages encrypt to the same ciphertext and authentication tag.
- Give their recipient key1, give everyone else (including your DLP software) key2.
Bam! File leaked, and everyone’s none the wiser, until it’s too late. Let’s actually imagine what happens next:
A random store clerk has leaked sensitive data to the press that only a few analysts had access to.
The only communication between the analyst and the store clerk is a file that was shared to all employees, using the E2EE protocol. No emails or anything else were identified.
Your DLP product didn’t identify any other communications between these two, but somehow the store clerk has the data on their desktop.
A detailed forensics analysis may eventually figure out what happened, but by then, the damage is done and your product’s reputation is irrecoverably damaged.
All because the hypothetical E2EE protocol didn’t include a key-commitment mechanism, and nobody identified this deficit in their designs.
This isn’t to endorse DLP solutions at all, but rather, to highlight one of the many ways that the Invisible Salamander issue can be used creatively by clever attackers.
Art: AJThe Lesson to Learn
If you’re building a network protocol that uses AEAD to encrypt data over an insecure network (e.g., WireGuard), keep up the good work.
If you’re doing anything more involved than that, at the application layer, pause for a moment and consider whether your system will ever need multiple valid symmetric keys at once.
And, if the answer is “yes”, then you should always explicitly add a key-commitment mechanism to your system design.
(Hire a cryptographer if you’re not sure how to proceed.)
In my opinion, hemming and hawing over whether there’s a significant impact to the Invisible Salamanders issue is a worse use of your time than just solving it directly.
Eventually, I expect a new generation of AEAD modes will be standardized that explicitly provide key-commitment.
When these new designs are standardized, widely supported, and sufficiently trusted by experts, feel free to update my advice to “prefer using those modes” instead.
Header art: Harubaki, CMYKat, and Brian Gratwicke
https://soatok.blog/2024/09/10/invisible-salamanders-are-not-what-you-think/
#AEAD #AESGCM #InvisibleSalamanders #randomKeyRobustness #symmetricCryptography
- Generate two keys.
-
@dnkrupinski @prav since Prav is part of the #XMPP network, people who don't want to provide a phone number can choose any XMPP app/service to talk to #Prav users. In that sense, phone number is optional. It makes a few things easier for people who are ok to use phone number like in other mainstream messengers. But unlike those other mainstream options like WhatsApp, Telegram or Signal, we don't lock people to Prav. For example, you can use Monocles Chat and talk to Prav users.
-
Happy Birthday to #InstantMessaging and #RealTimeCommunication with #XMPP!
Emerged from the #Jabber #community in 1999 and originally designed to provide an open and #decentralised alternative to #ICQ and #MSN, XMPP evolved to an independent alternative for todays #chat apps.
#rtc #whatsapp #signal -
https://blog.fabiomanganiello.com/reply/the-local-tech-neighbourhood
A corollary to the concept of small tech neighbourhood is the idea of local tech communities.
It's easy for folks like me, who have been self-hosting on computers in our closets since high school, to lose track with the common non-tech person.
It's easy for us to say "just self-host your Nextcloud, Matrix, XMPP, Mastodon, Searxng, Wallabag etc.", adding a "it's just a
docker runcommand".I've also seen "just run your f*cking website" articles get to the top of HackerNews.
But then I thought: imagine if it was, say, the bakery industry rather than the software industry to go through a process of #enshittification.
Imagine that buying your bread or croissant tomorrow requires a $100/month Bread+ subscription with plenty of upsells, an age verification process that requires you to send your passport or retina scan to the Big Bread consortium, a tracking system that measures exactly how much bread you eat at home, and that keeps selling your eating habits to whoever wants to purchase them - all while your bread quality gets progressively worse over time, and the oligopoly that holds it, after acquiring or putting out of business any credible competitors, starts experimenting also with selling expired products, wrapped in paper littered with ads that you have to spend at least 30 seconds reading before you can make your sandwich.
Of course people would be outraged.
But many, no matter how outraged they are, will just swallow the bitter pill and stick with the Bread+ subscription (perhaps because they don't care enough yet to trigger a change in their habits, perhaps because they have other priorities and battles to fight), hoping that the next change to the T&C won't make it even worse.
And some will instead start looking for alternatives.
Would it make sense for a professional baker who's been in the business for decades to tell those folks "just bake your own f*cking bread" - and then complain that not enough are moving out of the Bread+ walled garden?
Or perhaps would it make more sense for that baker to volunteer for a competitive price to sell their bread to its friends, family and local community?
We all know that running your own stuff on your own machines is the best way to avoid enshittification. Just like wisely selecting and mixing your wheat and butter to bake your bread and cakes is the best way to make those products the way you like them. But not everyone will do it, nor they are supposed to do it.
People live different lives, have different priorities and choose different battles. Any pragmatic human must at some point delegate some tasks - that's how modern societies were born. We don't judge those outside of our area of competencies for not being able or willing to perform a certain task, no matter how trivial that looks to us.
An average neighbourhood has someone good enough at baking that they can sell their products to their local community.
And nowadays perhaps it also has someone good enough with computers that they can probably run on an old laptop or unused VPS stuff like a small Matrix server, a GoToSocial instance, a Nextcloud instance or an Immich server.
And they could probably serve that to small local groups of 10-20 people for something like $10/month (and at low volumes those services usually don't even require much maintenance).
It can be $100/200 month that can abundantly cover the cost of a small VPS, let alone the cost of running a miniPC at home.
It'd be a bargain for the subscribers too - nowadays for $10/month you don't even get a Netflix subscription, let alone a complete cloud alternative with no trackers and no ads.
It can even be a reliable source of income for IT folks in areas that would otherwise not provide other viable employment solutions.
In a healthy society with a high level of immunity against enshittification, your local self-hoster should probably become the digital equivalent of your local baker or your local pub.
-
Users Are Too Dependent on Centralized Techno-Fascist Corporate Structure to Ever Leave Discord
I’m watching people scatter into countless real-time chat alternatives to Discord after Discord started pulling the age-verification and age-gating card.
It’s very frustrating because people are entirely missing the point of a community and how social networks work. Real-time platforms and social media networks only work well when a large number of people share the same space at the same time. If everyone creates separate servers or competing apps, the result is fragmentation that makes it unviable.
One reason why Bluesky became so successful is the invitation and starter-pack move. It essentially allowed people to move collectively as cliques. Bluesky used invitations and starter packs to move groups of friends together. This kept communities intact. Moving as cliques preserves network structure, whereas random scattering does not. People aren’t do not seem to intend to move as cliques or subgraphs of networks off of Discord. And the whole reason people were on Discord was to host their communities, so an alternative becomes pointless if your community doesn’t remain intact.
Instead of an active, strongly connected, possibly distributed network, you get dozens of small pockets. I am referring to a potential distributed network rather than a single centralized platform, because Matrix is an example of a decentralized chat protocol. Not all alternatives have to be centralized like Discord. Technically, many older chat protocols, such as XMPP and IRC, are examples of federated real-time synchronous messaging. They allowed communication between users on different, independently operated servers. Federation means that multiple servers can interconnect so that users from separate networks can exchange messages with one another seamlessly.
Decentralized alternatives would not be a problem if people moved to the same distributed network as cohesive groups. However, what I am seeing is that people move in disconnected and stochastic ways to entirely separate distributed networks, so communities are not kept intact. For example, when people move to XMPP servers or Matrix servers, it bifurcates and disconnects social networks. Notice I said XMPP or Matrix, which logically means people are on Matrix but not XMPP, or they are on XMPP but not Matrix. That implies a person would need to be on both Matrix and XMPP to speak to their original community from Discord if it split down the middle. To synchronize conversations in chats, there would need to be a bridge. It’s a pretty complicated solution.
The likely outcome is that people will remain on the dominant platform because of its scale and structure. The deeper irony is that while people may want independence from corporate platforms, they often struggle to organize effectively without the centralized structure those platforms provide. They’ve become so dependent on corporate structures to support their communities that they have no clue how to organize their own social networks in a sustainable way.
I’ve always been an internet nerd, but most of my social life has been offline. I view my interactions with the social app layer of the internet as a game, so losing that domain of the Internet is not devastating to me.
I’ll give you an example. This is a WordPress site. You hear this insincere nostalgia from Millennials and Gen X for a simulacrum that never was, especially concerning forums. Check this out: when you go into the plugin installation section of WordPress, this is on the second row you see:
That means any WordPress site has the capability to host a forum. They’re nostalgic for a setup where you can use a simple install script on any hosting service to install WordPress. After that, you can then just add a plugin to turn it into a forum. Hell, they can do this on WordPress.com if they don’t want to self-host.
You can make a forum, but no one will use it because they’d rather use a centralized platform like Reddit. Users have become so dependent on corporations to structure and organize communities that they can’t do it themselves. It’s sort of like the cognitive debt that accrues when people outsource their thinking to AI.
The issue is not that forums are hard to host or create; rather, the issue is that people have become so dependent on centralized corporate structures that they can’t maintain or organize their own communities, which is why everyone ends up on Reddit or Discord. A reason I keep hearing for why people don’t want to leave Discord is that it’s hard to recreate the community structure that Discord’s features provide. They claim that they want independence from corporate platforms, but rely on the centralized structure those platforms provide to function socially.
People say they want decentralized freedom, but in practice they depend on centralized platforms to maintain social cohesion. Stochastically scattering to the digital winds of the noosphere destroys the very communities they’re trying to preserve.
-
The Programmer’s Fulcrum: 27 February, 2026Welcome to this week’s The Programmer’s Fulcrum.
It’s your weekly review of the essential news in the Open Media Network and Fediverse development communities with a focus on devastating big tech via Techno Anarchism. We aim to provide actionable content you can use to destroy Techno Feudalism each week. It has the additional benefit of weakening authoritarianism.
IMHO, the best way to do […]
https://newsletter.mobileatom.net/the-programmers-fulcrum-27-february-2026/ #ActivityPub #AI #ATProto #bluesky #Buttondown #Codeberg #CSS #Emissary #FAIR #fediverse #Ghost #GoToSocial #Gutenberg #Holos #HTML #HTMX #javascript #Linux #Mastodon #Nextcloud #OMN #OrganicMaps #PixelFed #Tuta #WordPress #XMPP #xWiki -
🇫🇷 Bridge XMPP ↔️ FÉDIVERSE
Un service développé spécialement pour vous, le bridge XMPP/AP permet de dialoguer entre les applications du fédiverse et la messagerie instantanée XMPP. Intuitif et facile d'utilisation, à partir de votre application et de votre compte habituel, il vous suffit de contacter @xmpp_bridge (depuis le fédiverse) ou xmpp:[email protected] (depuis XMPP).
Si vous administrez un serveur, vous pourrez également l'installer vous-même, le code est disponible en source ouverte.
Toute la documentation pour l'utilisation, l'administration ou l'installation est ici: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
____________________
🇬🇧 Bridge XMPP ↔️ FEDIVERSE
A service developed especially for you, the XMPP/AP Bridge allows you to communicate between #Fediverse applications and #XMPP instant messaging. Intuitive and easy to use, from your usual application and account, simply contact @xmpp_bridge (from the Fediverse) or xmpp:[email protected] (from XMPP).
If you administer a server, you can also install it yourself; the code is provided as open source.
All documentation for use, administration, or installation is available here: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
-
🇫🇷 Bridge XMPP ↔️ FÉDIVERSE
Un service développé spécialement pour vous, le bridge XMPP/AP permet de dialoguer entre les applications du fédiverse et la messagerie instantanée XMPP. Intuitif et facile d'utilisation, à partir de votre application et de votre compte habituel, il vous suffit de contacter @xmpp_bridge (depuis le fédiverse) ou xmpp:[email protected] (depuis XMPP).
Si vous administrez un serveur, vous pourrez également l'installer vous-même, le code est disponible en source ouverte.
Toute la documentation pour l'utilisation, l'administration ou l'installation est ici: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
____________________
🇬🇧 Bridge XMPP ↔️ FEDIVERSE
A service developed especially for you, the XMPP/AP Bridge allows you to communicate between #Fediverse applications and #XMPP instant messaging. Intuitive and easy to use, from your usual application and account, simply contact @xmpp_bridge (from the Fediverse) or xmpp:[email protected] (from XMPP).
If you administer a server, you can also install it yourself; the code is provided as open source.
All documentation for use, administration, or installation is available here: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
-
🇫🇷 Bridge XMPP ↔️ FÉDIVERSE
Un service développé spécialement pour vous, le bridge XMPP/AP permet de dialoguer entre les applications du fédiverse et la messagerie instantanée XMPP. Intuitif et facile d'utilisation, à partir de votre application et de votre compte habituel, il vous suffit de contacter @xmpp_bridge (depuis le fédiverse) ou xmpp:[email protected] (depuis XMPP).
Si vous administrez un serveur, vous pourrez également l'installer vous-même, le code est disponible en source ouverte.
Toute la documentation pour l'utilisation, l'administration ou l'installation est ici: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
____________________
🇬🇧 Bridge XMPP ↔️ FEDIVERSE
A service developed especially for you, the XMPP/AP Bridge allows you to communicate between #Fediverse applications and #XMPP instant messaging. Intuitive and easy to use, from your usual application and account, simply contact @xmpp_bridge (from the Fediverse) or xmpp:[email protected] (from XMPP).
If you administer a server, you can also install it yourself; the code is provided as open source.
All documentation for use, administration, or installation is available here: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
-
🇫🇷 Bridge XMPP ↔️ FÉDIVERSE
Un service développé spécialement pour vous, le bridge XMPP/AP permet de dialoguer entre les applications du fédiverse et la messagerie instantanée XMPP. Intuitif et facile d'utilisation, à partir de votre application et de votre compte habituel, il vous suffit de contacter @xmpp_bridge (depuis le fédiverse) ou xmpp:[email protected] (depuis XMPP).
Si vous administrez un serveur, vous pourrez également l'installer vous-même, le code est disponible en source ouverte.
Toute la documentation pour l'utilisation, l'administration ou l'installation est ici: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
____________________
🇬🇧 Bridge XMPP ↔️ FEDIVERSE
A service developed especially for you, the XMPP/AP Bridge allows you to communicate between #Fediverse applications and #XMPP instant messaging. Intuitive and easy to use, from your usual application and account, simply contact @xmpp_bridge (from the Fediverse) or xmpp:[email protected] (from XMPP).
If you administer a server, you can also install it yourself; the code is provided as open source.
All documentation for use, administration, or installation is available here: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
-
🇫🇷 Bridge XMPP ↔️ FÉDIVERSE
Un service développé spécialement pour vous, le bridge XMPP/AP permet de dialoguer entre les applications du fédiverse et la messagerie instantanée XMPP. Intuitif et facile d'utilisation, à partir de votre application et de votre compte habituel, il vous suffit de contacter @xmpp_bridge (depuis le fédiverse) ou xmpp:[email protected] (depuis XMPP).
Si vous administrez un serveur, vous pourrez également l'installer vous-même, le code est disponible en source ouverte.
Toute la documentation pour l'utilisation, l'administration ou l'installation est ici: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
____________________
🇬🇧 Bridge XMPP ↔️ FEDIVERSE
A service developed especially for you, the XMPP/AP Bridge allows you to communicate between #Fediverse applications and #XMPP instant messaging. Intuitive and easy to use, from your usual application and account, simply contact @xmpp_bridge (from the Fediverse) or xmpp:[email protected] (from XMPP).
If you administer a server, you can also install it yourself; the code is provided as open source.
All documentation for use, administration, or installation is available here: https://chat.gayfr.online/blog/ap_bridge%40gayfr.live/xmpp-activitypub-bridge-documentation-KdpNJ9
-
The Fulcrum 10 April, 2026Welcome to this week’s The Programmer’s Fulcrum.
It’s your weekly review of the essential news in the Open Media Network and Fediverse development communities with a focus on devastating big tech via Techno Anarchism. We aim to provide actionable content you can use to destroy Techno Feudalism each week. It has the additional benefit of weakening authoritarianism.
IMHO, the best way to do […]
https://newsletter.mobileatom.net/the-fulcrum-10-april-2026/ #ActivityPub #astro #ATProto #BackdropCMS #Bear #Bonfire #Codefloe #CSS #Decidim #DWeb #EmDash #FDroid #FediLab #fediverse #Ghost #Holos #HTML #HTMX #JavasScript #Kdenlive #LAUTI #Linux #OMN #PWAs #RSS #WebAwesome #WordPress #WriteFreely #XMPP #xWiki -
Users Are Too Dependent on Centralized Techno-Fascist Corporate Structure to Ever Leave Discord
I’m watching people scatter into countless real-time chat alternatives to Discord after Discord started pulling the age-verification and age-gating card.
It’s very frustrating because people are entirely missing the point of a community and how social networks work. Real-time platforms and social media networks only work well when a large number of people share the same space at the same time. If everyone creates separate servers or competing apps, the result is fragmentation that makes it unviable.
One reason why Bluesky became so successful is the invitation and starter-pack move. It essentially allowed people to move collectively as cliques. Bluesky used invitations and starter packs to move groups of friends together. This kept communities intact. Moving as cliques preserves network structure, whereas random scattering does not. People aren’t do not seem to intend to move as cliques or subgraphs of networks off of Discord. And the whole reason people were on Discord was to host their communities, so an alternative becomes pointless if your community doesn’t remain intact.
Instead of an active, strongly connected, possibly distributed network, you get dozens of small pockets. I am referring to a potential distributed network rather than a single centralized platform, because Matrix is an example of a decentralized chat protocol. Not all alternatives have to be centralized like Discord. Technically, many older chat protocols, such as XMPP and IRC, are examples of federated real-time synchronous messaging. They allowed communication between users on different, independently operated servers. Federation means that multiple servers can interconnect so that users from separate networks can exchange messages with one another seamlessly.
Decentralized alternatives would not be a problem if people moved to the same distributed network as cohesive groups. However, what I am seeing is that people move in disconnected and stochastic ways to entirely separate distributed networks, so communities are not kept intact. For example, when people move to XMPP servers or Matrix servers, it bifurcates and disconnects social networks. Notice I said XMPP or Matrix, which logically means people are on Matrix but not XMPP, or they are on XMPP but not Matrix. That implies a person would need to be on both Matrix and XMPP to speak to their original community from Discord if it split down the middle. To synchronize conversations in chats, there would need to be a bridge. It’s a pretty complicated solution.
The likely outcome is that people will remain on the dominant platform because of its scale and structure. The deeper irony is that while people may want independence from corporate platforms, they often struggle to organize effectively without the centralized structure those platforms provide. They’ve become so dependent on corporate structures to support their communities that they have no clue how to organize their own social networks in a sustainable way.
I’ve always been an internet nerd, but most of my social life has been offline. I view my interactions with the social app layer of the internet as a game, so losing that domain of the Internet is not devastating to me.
I’ll give you an example. This is a WordPress site. You hear this insincere nostalgia from Millennials and Gen X for a simulacrum that never was, especially concerning forums. Check this out: when you go into the plugin installation section of WordPress, this is on the second row you see:
That means any WordPress site has the capability to host a forum. They’re nostalgic for a setup where you can use a simple install script on any hosting service to install WordPress. After that, you can then just add a plugin to turn it into a forum. Hell, they can do this on WordPress.com if they don’t want to self-host.
You can make a forum, but no one will use it because they’d rather use a centralized platform like Reddit. Users have become so dependent on corporations to structure and organize communities that they can’t do it themselves. It’s sort of like the cognitive debt that accrues when people outsource their thinking to AI.
The issue is not that forums are hard to host or create; rather, the issue is that people have become so dependent on centralized corporate structures that they can’t maintain or organize their own communities, which is why everyone ends up on Reddit or Discord. A reason I keep hearing for why people don’t want to leave Discord is that it’s hard to recreate the community structure that Discord’s features provide. They claim that they want independence from corporate platforms, but rely on the centralized structure those platforms provide to function socially.
People say they want decentralized freedom, but in practice they depend on centralized platforms to maintain social cohesion. Stochastically scattering to the digital winds of the noosphere destroys the very communities they’re trying to preserve.
-
Users Are Too Dependent on Centralized Techno-Fascist Corporate Structure to Ever Leave Discord
I’m watching people scatter into countless real-time chat alternatives to Discord after Discord started pulling the age-verification and age-gating card.
It’s very frustrating because people are entirely missing the point of a community and how social networks work. Real-time platforms and social media networks only work well when a large number of people share the same space at the same time. If everyone creates separate servers or competing apps, the result is fragmentation that makes it unviable.
One reason why Bluesky became so successful is the invitation and starter-pack move. It essentially allowed people to move collectively as cliques. Bluesky used invitations and starter packs to move groups of friends together. This kept communities intact. Moving as cliques preserves network structure, whereas random scattering does not. People aren’t do not seem to intend to move as cliques or subgraphs of networks off of Discord. And the whole reason people were on Discord was to host their communities, so an alternative becomes pointless if your community doesn’t remain intact.
Instead of an active, strongly connected, possibly distributed network, you get dozens of small pockets. I am referring to a potential distributed network rather than a single centralized platform, because Matrix is an example of a decentralized chat protocol. Not all alternatives have to be centralized like Discord. Technically, many older chat protocols, such as XMPP and IRC, are examples of federated real-time synchronous messaging. They allowed communication between users on different, independently operated servers. Federation means that multiple servers can interconnect so that users from separate networks can exchange messages with one another seamlessly.
Decentralized alternatives would not be a problem if people moved to the same distributed network as cohesive groups. However, what I am seeing is that people move in disconnected and stochastic ways to entirely separate distributed networks, so communities are not kept intact. For example, when people move to XMPP servers or Matrix servers, it bifurcates and disconnects social networks. Notice I said XMPP or Matrix, which logically means people are on Matrix but not XMPP, or they are on XMPP but not Matrix. That implies a person would need to be on both Matrix and XMPP to speak to their original community from Discord if it split down the middle. To synchronize conversations in chats, there would need to be a bridge. It’s a pretty complicated solution.
The likely outcome is that people will remain on the dominant platform because of its scale and structure. The deeper irony is that while people may want independence from corporate platforms, they often struggle to organize effectively without the centralized structure those platforms provide. They’ve become so dependent on corporate structures to support their communities that they have no clue how to organize their own social networks in a sustainable way.
I’ve always been an internet nerd, but most of my social life has been offline. I view my interactions with the social app layer of the internet as a game, so losing that domain of the Internet is not devastating to me.
I’ll give you an example. This is a WordPress site. You hear this insincere nostalgia from Millennials and Gen X for a simulacrum that never was, especially concerning forums. Check this out: when you go into the plugin installation section of WordPress, this is on the second row you see:
That means any WordPress site has the capability to host a forum. They’re nostalgic for a setup where you can use a simple install script on any hosting service to install WordPress. After that, you can then just add a plugin to turn it into a forum. Hell, they can do this on WordPress.com if they don’t want to self-host.
You can make a forum, but no one will use it because they’d rather use a centralized platform like Reddit. Users have become so dependent on corporations to structure and organize communities that they can’t do it themselves. It’s sort of like the cognitive debt that accrues when people outsource their thinking to AI.
The issue is not that forums are hard to host or create; rather, the issue is that people have become so dependent on centralized corporate structures that they can’t maintain or organize their own communities, which is why everyone ends up on Reddit or Discord. A reason I keep hearing for why people don’t want to leave Discord is that it’s hard to recreate the community structure that Discord’s features provide. They claim that they want independence from corporate platforms, but rely on the centralized structure those platforms provide to function socially.
People say they want decentralized freedom, but in practice they depend on centralized platforms to maintain social cohesion. Stochastically scattering to the digital winds of the noosphere destroys the very communities they’re trying to preserve.
-
Users Are Too Dependent on Centralized Techno-Fascist Corporate Structure to Ever Leave Discord
I’m watching people scatter into countless real-time chat alternatives to Discord after Discord started pulling the age-verification and age-gating card.
It’s very frustrating because people are entirely missing the point of a community and how social networks work. Real-time platforms and social media networks only work well when a large number of people share the same space at the same time. If everyone creates separate servers or competing apps, the result is fragmentation that makes it unviable.
One reason why Bluesky became so successful is the invitation and starter-pack move. It essentially allowed people to move collectively as cliques. Bluesky used invitations and starter packs to move groups of friends together. This kept communities intact. Moving as cliques preserves network structure, whereas random scattering does not. People aren’t do not seem to intend to move as cliques or subgraphs of networks off of Discord. And the whole reason people were on Discord was to host their communities, so an alternative becomes pointless if your community doesn’t remain intact.
Instead of an active, strongly connected, possibly distributed network, you get dozens of small pockets. I am referring to a potential distributed network rather than a single centralized platform, because Matrix is an example of a decentralized chat protocol. Not all alternatives have to be centralized like Discord. Technically, many older chat protocols, such as XMPP and IRC, are examples of federated real-time synchronous messaging. They allowed communication between users on different, independently operated servers. Federation means that multiple servers can interconnect so that users from separate networks can exchange messages with one another seamlessly.
Decentralized alternatives would not be a problem if people moved to the same distributed network as cohesive groups. However, what I am seeing is that people move in disconnected and stochastic ways to entirely separate distributed networks, so communities are not kept intact. For example, when people move to XMPP servers or Matrix servers, it bifurcates and disconnects social networks. Notice I said XMPP or Matrix, which logically means people are on Matrix but not XMPP, or they are on XMPP but not Matrix. That implies a person would need to be on both Matrix and XMPP to speak to their original community from Discord if it split down the middle. To synchronize conversations in chats, there would need to be a bridge. It’s a pretty complicated solution.
The likely outcome is that people will remain on the dominant platform because of its scale and structure. The deeper irony is that while people may want independence from corporate platforms, they often struggle to organize effectively without the centralized structure those platforms provide. They’ve become so dependent on corporate structures to support their communities that they have no clue how to organize their own social networks in a sustainable way.
I’ve always been an internet nerd, but most of my social life has been offline. I view my interactions with the social app layer of the internet as a game, so losing that domain of the Internet is not devastating to me.
I’ll give you an example. This is a WordPress site. You hear this insincere nostalgia from Millennials and Gen X for a simulacrum that never was, especially concerning forums. Check this out: when you go into the plugin installation section of WordPress, this is on the second row you see:
That means any WordPress site has the capability to host a forum. They’re nostalgic for a setup where you can use a simple install script on any hosting service to install WordPress. After that, you can then just add a plugin to turn it into a forum. Hell, they can do this on WordPress.com if they don’t want to self-host.
You can make a forum, but no one will use it because they’d rather use a centralized platform like Reddit. Users have become so dependent on corporations to structure and organize communities that they can’t do it themselves. It’s sort of like the cognitive debt that accrues when people outsource their thinking to AI.
The issue is not that forums are hard to host or create; rather, the issue is that people have become so dependent on centralized corporate structures that they can’t maintain or organize their own communities, which is why everyone ends up on Reddit or Discord. A reason I keep hearing for why people don’t want to leave Discord is that it’s hard to recreate the community structure that Discord’s features provide. They claim that they want independence from corporate platforms, but rely on the centralized structure those platforms provide to function socially.
People say they want decentralized freedom, but in practice they depend on centralized platforms to maintain social cohesion. Stochastically scattering to the digital winds of the noosphere destroys the very communities they’re trying to preserve.
-
In the newest edition of The Newsletter Leaf Journal, I recap the week that was at The New Leaf Journal, provide links from around the web, and discuss what looks like a very interesting FOSS project called sms4you, which allows for forwarding SMS messages to XMPP or email.
https://buttondown.email/newsletterleafjournal/archive/the-newsletter-leaf-journal-lxi-billy-the-pet svdesc/
#newsletter #aroundtheweb #links #owls #birds #foss #sms #xmpp #jmp #sms4you
-
We are looking to raise funds to provide bursaries to people promoting #Prav / #xmpp in different conferences / events. Unlike Operating System or Office Suite, messaging apps is not only personal choice.
We have setup a dedicated fund for this:
https://opencollective.com/pravapp/projects/event-bursariesWe have some funds from #PravConf sponsorship, but more would help us support people going to many more events like #BiharFOSS or #LucknowFOSS where we don't have many local community members.
-
The FreeBSD-native-ish home lab and network
For many years my setup was pretty simple: A FreeBSD home server running on my old laptop. It runs everything I need to be present on the internet, an email server, a web server (like the one you’ve accessed right now to see this blog post) and a public chat server (XMPP/Jabber) so I can be in touch with friends.
For my home network, I had a basic Access Point and a basic Router.
Lately, my setup has become more… intense. I have IPv6 thanks to Hurricane Electric, the network is passed to my home network (which we’ll talk about in a bit), a home network with multiple VLANs, since friends who come home also need WiFi.
I decided to blog about the details, hoping it would help someone in the future.
I’ll start with the simplest one.
The Home Server
I’ve been running home servers for a long time. I believe that every person/family needs a home server. Forget about buying your kids iPads and Smartphones. Their first devices should be a real computer (sorry Apple, iOS devices are still just a toy) like a desktop/laptop and a home server. The home server doesn’t need to be on the public internet, but mine is, for variety of reasons. This blog being one of them.
I get a static IP address from my ISP, Ucom. After the management change that happened couple of years ago, Ucom has become a very typical ISP (think shitty), but they are the only ones that provide a static IP address, instead of setting it on your router, where you have to do port forwarding.
My home server, hostnamed pingvinashen (meaning the town of the penguins, named after the Armenian cartoon) run FreeBSD. Historically this machine has run Debian, Funtoo, Gentoo and finally FreeBSD.
Hardware wise, here’s what it is:
root@pingvinashen:~ # dmidecode -s system-product-nameLatitude E5470root@pingvinashen:~ # sysctl hw.modelhw.model: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHzroot@pingvinashen:~ # sysctl hw.physmemhw.physmem: 17016950784root@pingvinashen:~ # zpool listNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTzroot 420G 178G 242G - - 64% 42% 1.00x ONLINE -
While most homelabbers use hardware virtualization, I think that resources are a tight thing, and should be managed properly. Any company that markets itself as “green/eco-friendly” and uses hardware virtualization should do calculations using a pen and paper and prove if going native would save power/resources or not. (sometimes it doesn’t, usually it does)
I use containers, the old-school ones, Jails to be more specific.
I manage jails using Jailer, my own tool, that tries to stay out of your way when working with Jails.
Here are my current jails:
root@pingvinashen:~ # jailer listNAME STATE JID HOSTNAME IPv4 GWantranig Active 1 antranig.bsd.am 192.168.10.42/24 192.168.10.1antranigv Active 2 antranigv.bsd.am 192.168.10.52/24 192.168.10.1git Stoppedhuginn0 Active 4 huginn0.bsd.am 192.168.10.34/24 192.168.10.1ifconfig Active 5 ifconfig.bsd.am 192.168.10.33/24 192.168.10.1lucy Active 6 lucy.vartanian.am 192.168.10.37/24 192.168.10.1mysql Active 7 mysql.antranigv.am 192.168.10.50/24 192.168.10.1newsletter Active 8 newsletter.bsd.am 192.168.10.65/24 192.168.10.1oragir Active 9 oragir.am 192.168.10.30/24 192.168.10.1psql Active 10 psql.pingvinashen.am 192.168.10.3/24 192.168.10.1rss Active 11 rss.bsd.am 192.168.10.5/24 192.168.10.1sarian Active 12 sarian.am 192.168.10.53/24 192.168.10.1syuneci Active 13 syuneci.am 192.168.10.60/24 192.168.10.1znc Active 14 znc.bsd.am 192.168.10.152/24 192.168.10.1
You already get a basic idea of how things are. Each of my blogs (Armenian and English) has its own Jail. Since I’m using WordPress, I need a database, so I have a MySQL jail (which ironically runs MariaDB) inside of it.
I also have a Git server, running gitea, which is down at the moment as I’m doing maintanence. The Git server (and many other services) requires PostgreSQL, hence the existence of a PostgreSQL jail. I run huginn for automation (RSS to Telegram, RSS to XMPP). My sister has her own blog, using WordPress, so that’s a Jail of its own. Same goes about my fiancée.
Other Jails are Newsletter using Listmonk, Sarian (the Armenian instance of lobste.rs) and a personal ZNC server.
As an avid RSS advocate, I also have a RSS Jail, which runs Miniflux. Many of my friends use this service.
Oragir is an instance of WriteFreely, as I advocate public blogging and ActivityPub. Our community uses that too.
The web server that forwards all this traffic from the public to the Jails is nginx. All it does is
proxy_passas needed. It runs on the host.Other services that run on the host are DNS (BIND9), an email service running
OpenSMTPd(which will be moved to a Jail soon), the chat service runningprosody(which will be moved to a Jail soon) and finally, WireGuard, because I love VPNs.Finally, there’s a IPv6-over-IPv4 tunnel that I use to obtain IPv6 thanks to Hurricane Electric.
Yes, I have a firewall, I use
pf(4).For the techies in the room, here’s what my
rc.conflooks like.# cat /etc/rc.conf# Defaultsclear_tmp_enable="YES"syslogd_flags="-ss"sendmail_enable="NONE"#local_unbound_enable="YES"sshd_enable="YES"moused_enable="YES"ntpd_enable="YES"# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disabledumpdev="AUTO"zfs_enable="YES"hostname="pingvinashen.am"# Networkingdefaultrouter="37.157.221.1"gateway_enable="YES"ifconfig_em0="up"vlans_em0="37 1000" # 1000 -> WAN; 37 -> Home Routerifconfig_em0_1000="inet 37.157.221.130 netmask 255.255.255.0"ifconfig_em0_37="inet 192.168.255.2 netmask 255.255.255.0"static_routes="home"route_home="-net 172.16.100.0/24 -gateway 192.168.255.1"cloned_interfaces="bridge0 bridge6 bridge10"ifconfig_bridge10="inet 192.168.10.1 netmask 255.255.255.0"## IPv6ipv6_gateway_enable="YES"gif_interfaces="gif0"gifconfig_gif0="37.157.221.130 216.66.84.46"ifconfig_gif0="inet6 2001:470:1f14:ef::2 2001:470:1f14:ef::1 prefixlen 128"ipv6_defaultrouter="2001:470:1f14:ef::1"ifconfig_em0_37_ipv6="inet6 2001:470:7914:7065::2 prefixlen 64"ipv6_static_routes="home guest"ipv6_route_home="-net 2001:470:7914:6a76::/64 -gateway 2001:470:7914:7065::1"ipv6_route_guest="-net 2001:470:7914:6969::/64 -gateway 2001:470:7914:7065::1"ifconfig_bridge6_ipv6="inet6 2001:470:1f15:e4::1 prefixlen 64"ifconfig_bridge6_aliases="inet6 2001:470:1f15:e4::25 prefixlen 64 \inet6 2001:470:1f15:e4::80 prefixlen 64 \inet6 2001:470:1f15:e4::5222 prefixlen 64 \inet6 2001:470:1f15:e4:c0fe::53 prefixlen 64 \"# VPNwireguard_enable="YES"wireguard_interfaces="wg0"# Firewallpf_enable="YES"# Jailsjail_enable="YES"jailer_dir="zfs:zroot/jails"# DNSnamed_enable="YES"# Mailsmtpd_enable="YES"smtpd_config="/usr/local/etc/smtpd.conf"# XMPPprosody_enable="YES"turnserver_enable="YES"# Webnginx_enable="YES"tor_enable="YES"
The
gif0interface is a IPv6-over-IPv4 tunnel. I have static routes to my home network, so I don’t go to my server over the ISP every time. This also gives me the ability to get IPv6 in my home network that is routed via my home server.As you have guessed from this config file, I do have VLANs setup. So let’s get into that.
The Home Network
First of all, here’s a very cheap diagram
I have the following VLANs setup on the switch.
VLAN IDPurpose1Switch Management1000pingvinashen (home server) WAN1001evn0 (home router) WAN37pingvinashen ↔ evn042Internal Management100Home LAN69Home GuestHere are the active ports
PortVLANsPurpose24untagged: 1Switch management, connects to Port 222untagged: 1000pingvinashen WAN, from ISP21untagged: 1001Home WAN, from ISP20tagged: 1000, 37To pingvinashen, portem019untagged: 1001To home router, portigb118tagged: 42, 100, 69, 99To home router, portigb217untagged: 37To home router, portigb016tagged: 42, 100, 69To Lenovo T480s15untagged: 100To Raspberri Pi 42untagged: 99From Port 24, for switch management1untagged: 42; tagged: 100, 69; PoETo UAP AC ProThe home router, hostnamed
evn0(named after the IATA code of Yerevan’s Zvartnots International Airport) runs FreeBSD as well, the hardware is the followingroot@evn0:~ # dmidecode -s system-product-nameAPU2root@evn0:~ # sysctl hw.modelhw.model: AMD GX-412TC SOC root@evn0:~ # sysctl hw.physmemhw.physmem: 4234399744root@evn0:~ # zpool listNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTzroot 12.5G 9.47G 3.03G - - 67% 75% 1.00x ONLINE -
The home router does… well, routing. It also does DHCP, DNS, SLAAC, and can act as a syslog server.
Here’s what the
rc.conflooks likeclear_tmp_enable="YES"sendmail_enable="NONE"syslogd_flags="-a '172.16.100.0/24:*' -H"zfs_enable="YES"dumpdev="AUTO"hostname="evn0.illuriasecurity.com"pf_enable="YES"gateway_enable="YES"ipv6_gateway_enable="YES"sshd_enable="YES"# Get an IP address from the ISP's GPONifconfig_igb1="DHCP"# Internal routes with pingvinashenifconfig_igb0="inet 192.168.255.1 netmask 255.255.255.0"ifconfig_igb0_ipv6="inet6 2001:470:7914:7065::1 prefixlen 64"static_routes="pingvinashen"route_pingvinashen="-net 37.157.221.130/32 -gateway 192.168.255.2"ipv6_defaultrouter="2001:470:7914:7065::2"# Home Mgmt, Switch Mgmt, Home LAN, Home Guestifconfig_igb2="up"vlans_igb2="42 99 100 69"ifconfig_igb2_42="inet 172.31.42.1 netmask 255.255.255.0"ifconfig_igb2_99="inet 172.16.99.1 netmask 255.255.255.0"ifconfig_igb2_100="inet 172.16.100.1 netmask 255.255.255.0"ifconfig_igb2_100_ipv6="inet6 2001:470:7914:6a76::1 prefixlen 64"ifconfig_igb2_69="inet 192.168.69.1 netmask 255.255.255.0"ifconfig_igb2_69_ipv6="inet6 2001:470:7914:6969::1 prefixlen 64"# DNS and DHCPnamed_enable="YES"dhcpd_enable="YES"named_flags=""# NTPntpd_enable="YES"# Router Advertisement and LLDPrtadvd_enable="YES"lldpd_enable="YES"lldpd_flags=""
Here’s
pf.conf, because security is important.ext_if="igb1"bsd_if="igb0"int_if="igb2.100"guest_if="igb2.69"mgmt_if="igb2.42"sw_if="igb2.99"ill_net="172.16.0.0/16"nat pass on $ext_if from $int_if:network to any -> ($ext_if)nat pass on $ext_if from $mgmt_if:network to any -> ($ext_if)nat pass on $ext_if from $guest_if:network to any -> ($ext_if)set skip on { lo0 }block in allpass on $int_if from $int_if:network to anypass on $mgmt_if from $mgmt_if:network to anypass on $sw_if from $sw_if:network to anypass on $guest_if from $guest_if:network to anyblock quick on $guest_if from any to { $int_if:network, $mgmt_if:network, $ill_net, $sw_if:network }pass in on illuria0 from $ill_net to { $ill_net, $mgmt_if:network }pass inet proto icmppass inet6 proto icmp6pass out all keep stateI’m sure there are places to improve, but it gets the job done and keeps the guest network isolated.
Here’s
rtadvd.conf, for my IPv6 folksigb2.100:\ :addr="2001:470:7914:6a76::":prefixlen#64:\ :rdnss="2001:470:7914:6a76::1":\ :dnssl="evn0.loc.illuriasecurity.com,loc.illuriasecurity.com":igb2.69:\ :addr="2001:470:7914:6969::":prefixlen#64:\ :rdnss="2001:470:7914:6969::1":
For DNS, I’m running BIND, here’s the important parts
listen-on { 127.0.0.1; 172.16.100.1; 172.16.99.1; 172.31.42.1; 192.168.69.1; };listen-on-v6 { 2001:470:7914:6a76::1; 2001:470:7914:6969::1; };allow-query { 127.0.0.1; 172.16.100.0/24; 172.31.42.0/24; 192.168.69.0/24; 2001:470:7914:6a76::/64; 2001:470:7914:6969::/64;};And for DHCP, here’s what it looks like
subnet 172.16.100.0 netmask 255.255.255.0 { range 172.16.100.100 172.16.100.150; option domain-name-servers 172.16.100.1; option subnet-mask 255.255.255.0; option routers 172.16.100.1; option domain-name "evn0.loc.illuriasecurity.com"; option domain-search "loc.illuriasecurity.com evn0.loc.illuriasecurity.com";}host zvartnots { hardware ethernet d4:57:63:f1:5a:36; fixed-address 172.16.100.7;}host unifi0 { hardware ethernet 58:9c:fc:93:d1:0b; fixed-address 172.31.42.42;}
[…]subnet 172.31.42.0 netmask 255.255.255.0 { range 172.31.42.100 172.31.42.150; option domain-name-servers 172.31.42.1; option subnet-mask 255.255.255.0; option routers 172.31.42.1;}subnet 192.168.69.0 netmask 255.255.255.0 { range 192.168.69.100 192.168.69.150; option domain-name-servers 192.168.69.1; option subnet-mask 255.255.255.0; option routers 192.168.69.1;}So you’re wondering, what’s this
unifi0? Well, that brings us toT480s
This laptop has been gifted to me by [REDACTED] for my contributions to the Armenian government (which means when a server goes down and no one knows how to fix it, they called me and I showed up)
Here’s the hardware
root@t480s:~ # dmidecode -s system-versionThinkPad T480sroot@t480s:~ # sysctl hw.modelhw.model: Intel(R) Core(TM) i5-8350U CPU @ 1.70GHzroot@t480s:~ # sysctl hw.physmemhw.physmem: 25602347008root@t480s:~ # zpool listNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTzroot 224G 109G 115G - - 44% 48% 1.00x ONLINE -
The T480s has access to VLAN 100, 42, 69, but the host itself has access only to VLAN 100 (LAN), while the jails can exist on other VLANs.
So I have a Jail named
unifi0that runs the Unifi Management thingie.Here’s what
rc.confof the host looks likeclear_tmp_enable="YES"syslogd_flags="-ss"sendmail_enable="NONE"sshd_enable="YES"ntpd_enable="YES"# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disabledumpdev="AUTO"zfs_enable="YES"hostname="t480s.evn0.loc.illuriasecurity.com"ifconfig_em0="up -rxcsum -txcsum"vlans_em0="100 42 69"ifconfig_em0_100="up"ifconfig_em0_42="up"ifconfig_em0_69="up"cloned_interfaces="bridge0 bridge100 bridge42 bridge69"create_args_bridge100="ether 8c:16:45:82:b4:10"ifconfig_bridge100="addm em0.100 SYNCDHCP"ifconfig_bridge100_ipv6="inet6 auto_linklocal"rtsold_flags="-i -F -m bridge100"rtsold_enable="YES"create_args_bridge42=" ether 8c:16:45:82:b4:42"create_args_bridge69=" ether 8c:16:45:82:b4:69"ifconfig_bridge42="addm em0.42"ifconfig_bridge69="addm em0.69"jail_enable="YES"jailer_dir="zfs:zroot/jailer"ifconfig_bridge0="inet 10.1.0.1/24 up"ngbuddy_enable="YES"ngbuddy_private_if="nghost0"dhcpd_enable="YES"lldpd_enable="YES"
I used Jailer to create the
unifi0jail, here’s what thejail.conflooks like# vim: set syntax=sh:exec.clean;allow.raw_sockets;mount.devfs;unifi0 { $id = "6"; devfs_ruleset = 10; $bridge = "bridge42"; $domain = "evn0.loc.illuriasecurity.com"; vnet; vnet.interface = "epair${id}b"; exec.prestart = "ifconfig epair${id} create up"; exec.prestart += "ifconfig epair${id}a up descr vnet-${name}"; exec.prestart += "ifconfig ${bridge} addm epair${id}a up"; exec.start = "/sbin/ifconfig lo0 127.0.0.1 up"; exec.start += "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown jail"; exec.poststop = "ifconfig ${bridge} deletem epair${id}a"; exec.poststop += "ifconfig epair${id}a destroy"; host.hostname = "${name}.${domain}"; path = "/usr/local/jailer/unifi0"; exec.consolelog = "/var/log/jail/${name}.log"; persist; mount.fdescfs; mount.procfs;}Here are the important parts inside the jail
root@t480s:~ # cat /usr/local/jailer/unifi0/etc/rc.confifconfig_epair6b="SYNCDHCP"sendmail_enable="NONE"syslogd_flags="-ss"mongod_enable="YES"unifi_enable="YES"root@t480s:~ # cat /usr/local/jailer/unifi0/etc/start_if.epair6b ifconfig epair6b ether 58:9c:fc:93:d1:0b
Don’t you love it that you can see what’s inside the jail from the host? God I love FreeBSD!
Did I miss anything? I hope not.
Oh, for the homelabbers out there, the T480s is the one that runs things like Jellyfin if needed.
Finally, the tiny
Raspberry Pi 4, Model B
I found this in a closed, so I decided to run it for TimeMachine.
I guess all you care about is
rc.confhostname="tm0.evn0.loc.illuriasecurity.com"ifconfig_DEFAULT="DHCP inet6 accept_rtadv"sshd_enable="YES"sendmail_enable="NONE"sendmail_submit_enable="NO"sendmail_outbound_enable="NO"sendmail_msp_queue_enable="NO"growfs_enable="YES"powerd_enable="YES"# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disabledumpdev="AUTO"zfs_enable="YES"rtsold_enable="YES"samba_server_enable="YES"
And the Samba Configuration
[global]# Network settingsworkgroup = WORKGROUPserver string = Samba Server %vnetbios name = RPi4# Logginglog file = /var/log/samba4/log.%mmax log size = 50log level = 0# Authenticationsecurity = userencrypt passwords = yespassdb backend = tdbsammap to guest = Bad Usermin protocol = SMB2max protocol = SMB3# Apple Time Machine settingsvfs objects = catia fruit streams_xattrfruit:metadata = streamfruit:resource = streamfruit:encoding = nativefruit:locking = nonefruit:time machine = yes# File System supportea support = yeskernel oplocks = nokernel share modes = noposix locking = nomangled names = nosmbd max xattr size = 2097152# Performance tuningread raw = yeswrite raw = yesgetwd cache = yesstrict locking = no# Miscellaneouslocal master = nopreferred master = nodomain master = nowins support = no[tm]comment = Time Machine RPi4path = /usr/local/timemachine/%Ubrowseable = yesread only = novalid users = antranigvvfs objects = catia fruit streams_xattrfruit:time machine = yesfruit:advertise_fullsync = truefruit:time machine max size = 800G # Adjust the size according to your needscreate mask = 0600directory mask = 0700
That’s pretty much it.
Conclusion
I love running homebrew servers, home networks and home labs. I love that (almost) everything is FreeBSD. The switch itself runs Linux, and the Unifi Access Point also runs Linux, both of which I’m pretty happy with.
While most homelabbers used ESXi in the past, I’m happy to see that most people are moving to open source solutions like Proxmox and Xen, but I think that FreeBSD Jails and bhyve is much better. I still don’t have a need for bhyve at the moment, but I would use it if I needed hardware virtualization.
Most homelabbers would consider the lack of Web/GUI interfaces as a con, but I think that it’s a pro. If I need to “replicate” this network, all I need to do is to copy some text files and modify some IP addresses / Interface names.
I hope this was informative and that it would be useful for anyone in the future.
That’s all folks…
Reply via email.
https://antranigv.am/posts/2024/06/freebsd-server-network-homelab/
#Containers #Dell #DellLatitudeE5470 #FreeBSD #homeServer #HowTo #Jailer #Jails #macOS #Networking #pf #Samba #Unifi #Unix #VNET
-
Please consider taking this (very) short survey to provide guidance to the Snikket team about the development priorities of existing and potential users:
https://snikket.org/community/survey/#success -
In the newest edition of The Newsletter Leaf Journal, I recap the week that was at The New Leaf Journal, provide links from around the web, and discuss what looks like a very interesting FOSS project called sms4you, which allows for forwarding SMS messages to XMPP or email.
https://buttondown.email/newsletterleafjournal/archive/the-newsletter-leaf-journal-lxi-billy-the-pet svdesc/
#newsletter #aroundtheweb #links #owls #birds #foss #sms #xmpp #jmp #sms4you
-
It's #ThrowbackThursday again! (Or, #WaybackWednesday for the majority of the planet (as of this post)).
Back in the “first generation” (2008–2012) (just my personal grouping, not set in stone) of the #fediverse, one of the first to provide different services for the public was parlementum_net.
#Parlementum provided the following services:
* #StatusNet instance* #Friendica instance
* #TTRSS instance
* Email
The driving force back then was not a person (like Musk, today) but corporations and freedom.
Here are some:
* The license of your post / status update, was a big thing. “Do you really own your own content in Twitter and Facebook?”* #DeGoogle yourself. Made very evident to be important when Google announced the sunsetting of Google Reader. TT-RSS gained a lot of attention as the alternative.
Another one in the DeGoogle movement was Google Photos (was it called that already?) #MediaGoblin gained a lot of attention as the replacement. (JPope was the most prominent advocate for MediaGoblin.)
* The use of pseudonyms. It was a huge issue when services like Facebook strictly enforced the use of real names. Parlementum provided a safe-haven in the fediverse for those who needs their #privacy through the use of handles.
I'm sure I've forgotten a lot of other stuff of that period. But, it should give you an idea that the fight to take back our [Internet] freedom has been going on for 20 years now.
If these great people (advocates, personalities, developers, communities, non-profits) gave up the fight, the Fediverse as we know it today would not be here, or might look far differently. Imagine that, and think of the 2022 #TwitterMigration. ;)
Have a great Thursday!
-
Looks like the website will remain up! Alder IT (https://alderit.com/) has graciously stepped in now that #Fosshost is no more and has agreed to host our website. We really appreciate all the recommendations everyone provided.
-
Looks like the website will remain up! Alder IT (https://alderit.com/) has graciously stepped in now that #Fosshost is no more and has agreed to host our website. We really appreciate all the recommendations everyone provided.
-
Looks like the website will remain up! Alder IT (https://alderit.com/) has graciously stepped in now that #Fosshost is no more and has agreed to host our website. We really appreciate all the recommendations everyone provided.
-
If any peeps are looking for a #XMPP/#Jabber instance, I can provide invites for anyone asking :)
Can also try to help if you run into any issues. Still working on other stuff currently, but the server already has 100% compliance (according to https://compliance.conversations.im/server/xenofox.nexus/). #boostok -
The FreeBSD-native-ish home lab and network
For many years my setup was pretty simple: A FreeBSD home server running on my old laptop. It runs everything I need to be present on the internet, an email server, a web server (like the one you’ve accessed right now to see this blog post) and a public chat server (XMPP/Jabber) so I can be in touch with friends.
For my home network, I had a basic Access Point and a basic Router.
Lately, my setup has become more… intense. I have IPv6 thanks to Hurricane Electric, the network is passed to my home network (which we’ll talk about in a bit), a home network with multiple VLANs, since friends who come home also need WiFi.
I decided to blog about the details, hoping it would help someone in the future.
I’ll start with the simplest one.
The Home Server
I’ve been running home servers for a long time. I believe that every person/family needs a home server. Forget about buying your kids iPads and Smartphones. Their first devices should be a real computer (sorry Apple, iOS devices are still just a toy) like a desktop/laptop and a home server. The home server doesn’t need to be on the public internet, but mine is, for variety of reasons. This blog being one of them.
I get a static IP address from my ISP, Ucom. After the management change that happened couple of years ago, Ucom has become a very typical ISP (think shitty), but they are the only ones that provide a static IP address, instead of setting it on your router, where you have to do port forwarding.
My home server, hostnamed pingvinashen (meaning the town of the penguins, named after the Armenian cartoon) run FreeBSD. Historically this machine has run Debian, Funtoo, Gentoo and finally FreeBSD.
Hardware wise, here’s what it is:
root@pingvinashen:~ # dmidecode -s system-product-nameLatitude E5470root@pingvinashen:~ # sysctl hw.modelhw.model: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHzroot@pingvinashen:~ # sysctl hw.physmemhw.physmem: 17016950784root@pingvinashen:~ # zpool listNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTzroot 420G 178G 242G - - 64% 42% 1.00x ONLINE -
While most homelabbers use hardware virtualization, I think that resources are a tight thing, and should be managed properly. Any company that markets itself as “green/eco-friendly” and uses hardware virtualization should do calculations using a pen and paper and prove if going native would save power/resources or not. (sometimes it doesn’t, usually it does)
I use containers, the old-school ones, Jails to be more specific.
I manage jails using Jailer, my own tool, that tries to stay out of your way when working with Jails.
Here are my current jails:
root@pingvinashen:~ # jailer listNAME STATE JID HOSTNAME IPv4 GWantranig Active 1 antranig.bsd.am 192.168.10.42/24 192.168.10.1antranigv Active 2 antranigv.bsd.am 192.168.10.52/24 192.168.10.1git Stoppedhuginn0 Active 4 huginn0.bsd.am 192.168.10.34/24 192.168.10.1ifconfig Active 5 ifconfig.bsd.am 192.168.10.33/24 192.168.10.1lucy Active 6 lucy.vartanian.am 192.168.10.37/24 192.168.10.1mysql Active 7 mysql.antranigv.am 192.168.10.50/24 192.168.10.1newsletter Active 8 newsletter.bsd.am 192.168.10.65/24 192.168.10.1oragir Active 9 oragir.am 192.168.10.30/24 192.168.10.1psql Active 10 psql.pingvinashen.am 192.168.10.3/24 192.168.10.1rss Active 11 rss.bsd.am 192.168.10.5/24 192.168.10.1sarian Active 12 sarian.am 192.168.10.53/24 192.168.10.1syuneci Active 13 syuneci.am 192.168.10.60/24 192.168.10.1znc Active 14 znc.bsd.am 192.168.10.152/24 192.168.10.1
You already get a basic idea of how things are. Each of my blogs (Armenian and English) has its own Jail. Since I’m using WordPress, I need a database, so I have a MySQL jail (which ironically runs MariaDB) inside of it.
I also have a Git server, running gitea, which is down at the moment as I’m doing maintanence. The Git server (and many other services) requires PostgreSQL, hence the existence of a PostgreSQL jail. I run huginn for automation (RSS to Telegram, RSS to XMPP). My sister has her own blog, using WordPress, so that’s a Jail of its own. Same goes about my fiancée.
Other Jails are Newsletter using Listmonk, Sarian (the Armenian instance of lobste.rs) and a personal ZNC server.
As an avid RSS advocate, I also have a RSS Jail, which runs Miniflux. Many of my friends use this service.
Oragir is an instance of WriteFreely, as I advocate public blogging and ActivityPub. Our community uses that too.
The web server that forwards all this traffic from the public to the Jails is nginx. All it does is
proxy_passas needed. It runs on the host.Other services that run on the host are DNS (BIND9), an email service running
OpenSMTPd(which will be moved to a Jail soon), the chat service runningprosody(which will be moved to a Jail soon) and finally, WireGuard, because I love VPNs.Finally, there’s a IPv6-over-IPv4 tunnel that I use to obtain IPv6 thanks to Hurricane Electric.
Yes, I have a firewall, I use
pf(4).For the techies in the room, here’s what my
rc.conflooks like.# cat /etc/rc.conf# Defaultsclear_tmp_enable="YES"syslogd_flags="-ss"sendmail_enable="NONE"#local_unbound_enable="YES"sshd_enable="YES"moused_enable="YES"ntpd_enable="YES"# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disabledumpdev="AUTO"zfs_enable="YES"hostname="pingvinashen.am"# Networkingdefaultrouter="37.157.221.1"gateway_enable="YES"ifconfig_em0="up"vlans_em0="37 1000" # 1000 -> WAN; 37 -> Home Routerifconfig_em0_1000="inet 37.157.221.130 netmask 255.255.255.0"ifconfig_em0_37="inet 192.168.255.2 netmask 255.255.255.0"static_routes="home"route_home="-net 172.16.100.0/24 -gateway 192.168.255.1"cloned_interfaces="bridge0 bridge6 bridge10"ifconfig_bridge10="inet 192.168.10.1 netmask 255.255.255.0"## IPv6ipv6_gateway_enable="YES"gif_interfaces="gif0"gifconfig_gif0="37.157.221.130 216.66.84.46"ifconfig_gif0="inet6 2001:470:1f14:ef::2 2001:470:1f14:ef::1 prefixlen 128"ipv6_defaultrouter="2001:470:1f14:ef::1"ifconfig_em0_37_ipv6="inet6 2001:470:7914:7065::2 prefixlen 64"ipv6_static_routes="home guest"ipv6_route_home="-net 2001:470:7914:6a76::/64 -gateway 2001:470:7914:7065::1"ipv6_route_guest="-net 2001:470:7914:6969::/64 -gateway 2001:470:7914:7065::1"ifconfig_bridge6_ipv6="inet6 2001:470:1f15:e4::1 prefixlen 64"ifconfig_bridge6_aliases="inet6 2001:470:1f15:e4::25 prefixlen 64 \inet6 2001:470:1f15:e4::80 prefixlen 64 \inet6 2001:470:1f15:e4::5222 prefixlen 64 \inet6 2001:470:1f15:e4:c0fe::53 prefixlen 64 \"# VPNwireguard_enable="YES"wireguard_interfaces="wg0"# Firewallpf_enable="YES"# Jailsjail_enable="YES"jailer_dir="zfs:zroot/jails"# DNSnamed_enable="YES"# Mailsmtpd_enable="YES"smtpd_config="/usr/local/etc/smtpd.conf"# XMPPprosody_enable="YES"turnserver_enable="YES"# Webnginx_enable="YES"tor_enable="YES"
The
gif0interface is a IPv6-over-IPv4 tunnel. I have static routes to my home network, so I don’t go to my server over the ISP every time. This also gives me the ability to get IPv6 in my home network that is routed via my home server.As you have guessed from this config file, I do have VLANs setup. So let’s get into that.
The Home Network
First of all, here’s a very cheap diagram
I have the following VLANs setup on the switch.
VLAN IDPurpose1Switch Management1000pingvinashen (home server) WAN1001evn0 (home router) WAN37pingvinashen ↔ evn042Internal Management100Home LAN69Home GuestHere are the active ports
PortVLANsPurpose24untagged: 1Switch management, connects to Port 222untagged: 1000pingvinashen WAN, from ISP21untagged: 1001Home WAN, from ISP20tagged: 1000, 37To pingvinashen, portem019untagged: 1001To home router, portigb118tagged: 42, 100, 69, 99To home router, portigb217untagged: 37To home router, portigb016tagged: 42, 100, 69To Lenovo T480s15untagged: 100To Raspberri Pi 42untagged: 99From Port 24, for switch management1untagged: 42; tagged: 100, 69; PoETo UAP AC ProThe home router, hostnamed
evn0(named after the IATA code of Yerevan’s Zvartnots International Airport) runs FreeBSD as well, the hardware is the followingroot@evn0:~ # dmidecode -s system-product-nameAPU2root@evn0:~ # sysctl hw.modelhw.model: AMD GX-412TC SOC root@evn0:~ # sysctl hw.physmemhw.physmem: 4234399744root@evn0:~ # zpool listNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTzroot 12.5G 9.47G 3.03G - - 67% 75% 1.00x ONLINE -
The home router does… well, routing. It also does DHCP, DNS, SLAAC, and can act as a syslog server.
Here’s what the
rc.conflooks likeclear_tmp_enable="YES"sendmail_enable="NONE"syslogd_flags="-a '172.16.100.0/24:*' -H"zfs_enable="YES"dumpdev="AUTO"hostname="evn0.illuriasecurity.com"pf_enable="YES"gateway_enable="YES"ipv6_gateway_enable="YES"sshd_enable="YES"# Get an IP address from the ISP's GPONifconfig_igb1="DHCP"# Internal routes with pingvinashenifconfig_igb0="inet 192.168.255.1 netmask 255.255.255.0"ifconfig_igb0_ipv6="inet6 2001:470:7914:7065::1 prefixlen 64"static_routes="pingvinashen"route_pingvinashen="-net 37.157.221.130/32 -gateway 192.168.255.2"ipv6_defaultrouter="2001:470:7914:7065::2"# Home Mgmt, Switch Mgmt, Home LAN, Home Guestifconfig_igb2="up"vlans_igb2="42 99 100 69"ifconfig_igb2_42="inet 172.31.42.1 netmask 255.255.255.0"ifconfig_igb2_99="inet 172.16.99.1 netmask 255.255.255.0"ifconfig_igb2_100="inet 172.16.100.1 netmask 255.255.255.0"ifconfig_igb2_100_ipv6="inet6 2001:470:7914:6a76::1 prefixlen 64"ifconfig_igb2_69="inet 192.168.69.1 netmask 255.255.255.0"ifconfig_igb2_69_ipv6="inet6 2001:470:7914:6969::1 prefixlen 64"# DNS and DHCPnamed_enable="YES"dhcpd_enable="YES"named_flags=""# NTPntpd_enable="YES"# Router Advertisement and LLDPrtadvd_enable="YES"lldpd_enable="YES"lldpd_flags=""
Here’s
pf.conf, because security is important.ext_if="igb1"bsd_if="igb0"int_if="igb2.100"guest_if="igb2.69"mgmt_if="igb2.42"sw_if="igb2.99"ill_net="172.16.0.0/16"nat pass on $ext_if from $int_if:network to any -> ($ext_if)nat pass on $ext_if from $mgmt_if:network to any -> ($ext_if)nat pass on $ext_if from $guest_if:network to any -> ($ext_if)set skip on { lo0 }block in allpass on $int_if from $int_if:network to anypass on $mgmt_if from $mgmt_if:network to anypass on $sw_if from $sw_if:network to anypass on $guest_if from $guest_if:network to anyblock quick on $guest_if from any to { $int_if:network, $mgmt_if:network, $ill_net, $sw_if:network }pass in on illuria0 from $ill_net to { $ill_net, $mgmt_if:network }pass inet proto icmppass inet6 proto icmp6pass out all keep stateI’m sure there are places to improve, but it gets the job done and keeps the guest network isolated.
Here’s
rtadvd.conf, for my IPv6 folksigb2.100:\ :addr="2001:470:7914:6a76::":prefixlen#64:\ :rdnss="2001:470:7914:6a76::1":\ :dnssl="evn0.loc.illuriasecurity.com,loc.illuriasecurity.com":igb2.69:\ :addr="2001:470:7914:6969::":prefixlen#64:\ :rdnss="2001:470:7914:6969::1":
For DNS, I’m running BIND, here’s the important parts
listen-on { 127.0.0.1; 172.16.100.1; 172.16.99.1; 172.31.42.1; 192.168.69.1; };listen-on-v6 { 2001:470:7914:6a76::1; 2001:470:7914:6969::1; };allow-query { 127.0.0.1; 172.16.100.0/24; 172.31.42.0/24; 192.168.69.0/24; 2001:470:7914:6a76::/64; 2001:470:7914:6969::/64;};And for DHCP, here’s what it looks like
subnet 172.16.100.0 netmask 255.255.255.0 { range 172.16.100.100 172.16.100.150; option domain-name-servers 172.16.100.1; option subnet-mask 255.255.255.0; option routers 172.16.100.1; option domain-name "evn0.loc.illuriasecurity.com"; option domain-search "loc.illuriasecurity.com evn0.loc.illuriasecurity.com";}host zvartnots { hardware ethernet d4:57:63:f1:5a:36; fixed-address 172.16.100.7;}host unifi0 { hardware ethernet 58:9c:fc:93:d1:0b; fixed-address 172.31.42.42;}
[…]subnet 172.31.42.0 netmask 255.255.255.0 { range 172.31.42.100 172.31.42.150; option domain-name-servers 172.31.42.1; option subnet-mask 255.255.255.0; option routers 172.31.42.1;}subnet 192.168.69.0 netmask 255.255.255.0 { range 192.168.69.100 192.168.69.150; option domain-name-servers 192.168.69.1; option subnet-mask 255.255.255.0; option routers 192.168.69.1;}So you’re wondering, what’s this
unifi0? Well, that brings us toT480s
This laptop has been gifted to me by [REDACTED] for my contributions to the Armenian government (which means when a server goes down and no one knows how to fix it, they called me and I showed up)
Here’s the hardware
root@t480s:~ # dmidecode -s system-versionThinkPad T480sroot@t480s:~ # sysctl hw.modelhw.model: Intel(R) Core(TM) i5-8350U CPU @ 1.70GHzroot@t480s:~ # sysctl hw.physmemhw.physmem: 25602347008root@t480s:~ # zpool listNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTzroot 224G 109G 115G - - 44% 48% 1.00x ONLINE -
The T480s has access to VLAN 100, 42, 69, but the host itself has access only to VLAN 100 (LAN), while the jails can exist on other VLANs.
So I have a Jail named
unifi0that runs the Unifi Management thingie.Here’s what
rc.confof the host looks likeclear_tmp_enable="YES"syslogd_flags="-ss"sendmail_enable="NONE"sshd_enable="YES"ntpd_enable="YES"# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disabledumpdev="AUTO"zfs_enable="YES"hostname="t480s.evn0.loc.illuriasecurity.com"ifconfig_em0="up -rxcsum -txcsum"vlans_em0="100 42 69"ifconfig_em0_100="up"ifconfig_em0_42="up"ifconfig_em0_69="up"cloned_interfaces="bridge0 bridge100 bridge42 bridge69"create_args_bridge100="ether 8c:16:45:82:b4:10"ifconfig_bridge100="addm em0.100 SYNCDHCP"ifconfig_bridge100_ipv6="inet6 auto_linklocal"rtsold_flags="-i -F -m bridge100"rtsold_enable="YES"create_args_bridge42=" ether 8c:16:45:82:b4:42"create_args_bridge69=" ether 8c:16:45:82:b4:69"ifconfig_bridge42="addm em0.42"ifconfig_bridge69="addm em0.69"jail_enable="YES"jailer_dir="zfs:zroot/jailer"ifconfig_bridge0="inet 10.1.0.1/24 up"ngbuddy_enable="YES"ngbuddy_private_if="nghost0"dhcpd_enable="YES"lldpd_enable="YES"
I used Jailer to create the
unifi0jail, here’s what thejail.conflooks like# vim: set syntax=sh:exec.clean;allow.raw_sockets;mount.devfs;unifi0 { $id = "6"; devfs_ruleset = 10; $bridge = "bridge42"; $domain = "evn0.loc.illuriasecurity.com"; vnet; vnet.interface = "epair${id}b"; exec.prestart = "ifconfig epair${id} create up"; exec.prestart += "ifconfig epair${id}a up descr vnet-${name}"; exec.prestart += "ifconfig ${bridge} addm epair${id}a up"; exec.start = "/sbin/ifconfig lo0 127.0.0.1 up"; exec.start += "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown jail"; exec.poststop = "ifconfig ${bridge} deletem epair${id}a"; exec.poststop += "ifconfig epair${id}a destroy"; host.hostname = "${name}.${domain}"; path = "/usr/local/jailer/unifi0"; exec.consolelog = "/var/log/jail/${name}.log"; persist; mount.fdescfs; mount.procfs;}Here are the important parts inside the jail
root@t480s:~ # cat /usr/local/jailer/unifi0/etc/rc.confifconfig_epair6b="SYNCDHCP"sendmail_enable="NONE"syslogd_flags="-ss"mongod_enable="YES"unifi_enable="YES"root@t480s:~ # cat /usr/local/jailer/unifi0/etc/start_if.epair6b ifconfig epair6b ether 58:9c:fc:93:d1:0b
Don’t you love it that you can see what’s inside the jail from the host? God I love FreeBSD!
Did I miss anything? I hope not.
Oh, for the homelabbers out there, the T480s is the one that runs things like Jellyfin if needed.
Finally, the tiny
Raspberry Pi 4, Model B
I found this in a closed, so I decided to run it for TimeMachine.
I guess all you care about is
rc.confhostname="tm0.evn0.loc.illuriasecurity.com"ifconfig_DEFAULT="DHCP inet6 accept_rtadv"sshd_enable="YES"sendmail_enable="NONE"sendmail_submit_enable="NO"sendmail_outbound_enable="NO"sendmail_msp_queue_enable="NO"growfs_enable="YES"powerd_enable="YES"# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disabledumpdev="AUTO"zfs_enable="YES"rtsold_enable="YES"samba_server_enable="YES"
And the Samba Configuration
[global]# Network settingsworkgroup = WORKGROUPserver string = Samba Server %vnetbios name = RPi4# Logginglog file = /var/log/samba4/log.%mmax log size = 50log level = 0# Authenticationsecurity = userencrypt passwords = yespassdb backend = tdbsammap to guest = Bad Usermin protocol = SMB2max protocol = SMB3# Apple Time Machine settingsvfs objects = catia fruit streams_xattrfruit:metadata = streamfruit:resource = streamfruit:encoding = nativefruit:locking = nonefruit:time machine = yes# File System supportea support = yeskernel oplocks = nokernel share modes = noposix locking = nomangled names = nosmbd max xattr size = 2097152# Performance tuningread raw = yeswrite raw = yesgetwd cache = yesstrict locking = no# Miscellaneouslocal master = nopreferred master = nodomain master = nowins support = no[tm]comment = Time Machine RPi4path = /usr/local/timemachine/%Ubrowseable = yesread only = novalid users = antranigvvfs objects = catia fruit streams_xattrfruit:time machine = yesfruit:advertise_fullsync = truefruit:time machine max size = 800G # Adjust the size according to your needscreate mask = 0600directory mask = 0700
That’s pretty much it.
Conclusion
I love running homebrew servers, home networks and home labs. I love that (almost) everything is FreeBSD. The switch itself runs Linux, and the Unifi Access Point also runs Linux, both of which I’m pretty happy with.
While most homelabbers used ESXi in the past, I’m happy to see that most people are moving to open source solutions like Proxmox and Xen, but I think that FreeBSD Jails and bhyve is much better. I still don’t have a need for bhyve at the moment, but I would use it if I needed hardware virtualization.
Most homelabbers would consider the lack of Web/GUI interfaces as a con, but I think that it’s a pro. If I need to “replicate” this network, all I need to do is to copy some text files and modify some IP addresses / Interface names.
I hope this was informative and that it would be useful for anyone in the future.
That’s all folks…
Reply via email.
https://antranigv.am/posts/2024/06/freebsd-server-network-homelab/
#Containers #Dell #DellLatitudeE5470 #FreeBSD #homeServer #HowTo #Jailer #Jails #macOS #Networking #pf #Samba #Unifi #Unix #VNET
-
The FreeBSD-native-ish home lab and network
For many years my setup was pretty simple: A FreeBSD home server running on my old laptop. It runs everything I need to be present on the internet, an email server, a web server (like the one you’ve accessed right now to see this blog post) and a public chat server (XMPP/Jabber) so I can be in touch with friends.
For my home network, I had a basic Access Point and a basic Router.
Lately, my setup has become more… intense. I have IPv6 thanks to Hurricane Electric, the network is passed to my home network (which we’ll talk about in a bit), a home network with multiple VLANs, since friends who come home also need WiFi.
I decided to blog about the details, hoping it would help someone in the future.
I’ll start with the simplest one.
The Home Server
I’ve been running home servers for a long time. I believe that every person/family needs a home server. Forget about buying your kids iPads and Smartphones. Their first devices should be a real computer (sorry Apple, iOS devices are still just a toy) like a desktop/laptop and a home server. The home server doesn’t need to be on the public internet, but mine is, for variety of reasons. This blog being one of them.
I get a static IP address from my ISP, Ucom. After the management change that happened couple of years ago, Ucom has become a very typical ISP (think shitty), but they are the only ones that provide a static IP address, instead of setting it on your router, where you have to do port forwarding.
My home server, hostnamed pingvinashen (meaning the town of the penguins, named after the Armenian cartoon) run FreeBSD. Historically this machine has run Debian, Funtoo, Gentoo and finally FreeBSD.
Hardware wise, here’s what it is:
root@pingvinashen:~ # dmidecode -s system-product-nameLatitude E5470root@pingvinashen:~ # sysctl hw.modelhw.model: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHzroot@pingvinashen:~ # sysctl hw.physmemhw.physmem: 17016950784root@pingvinashen:~ # zpool listNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTzroot 420G 178G 242G - - 64% 42% 1.00x ONLINE -
While most homelabbers use hardware virtualization, I think that resources are a tight thing, and should be managed properly. Any company that markets itself as “green/eco-friendly” and uses hardware virtualization should do calculations using a pen and paper and prove if going native would save power/resources or not. (sometimes it doesn’t, usually it does)
I use containers, the old-school ones, Jails to be more specific.
I manage jails using Jailer, my own tool, that tries to stay out of your way when working with Jails.
Here are my current jails:
root@pingvinashen:~ # jailer listNAME STATE JID HOSTNAME IPv4 GWantranig Active 1 antranig.bsd.am 192.168.10.42/24 192.168.10.1antranigv Active 2 antranigv.bsd.am 192.168.10.52/24 192.168.10.1git Stoppedhuginn0 Active 4 huginn0.bsd.am 192.168.10.34/24 192.168.10.1ifconfig Active 5 ifconfig.bsd.am 192.168.10.33/24 192.168.10.1lucy Active 6 lucy.vartanian.am 192.168.10.37/24 192.168.10.1mysql Active 7 mysql.antranigv.am 192.168.10.50/24 192.168.10.1newsletter Active 8 newsletter.bsd.am 192.168.10.65/24 192.168.10.1oragir Active 9 oragir.am 192.168.10.30/24 192.168.10.1psql Active 10 psql.pingvinashen.am 192.168.10.3/24 192.168.10.1rss Active 11 rss.bsd.am 192.168.10.5/24 192.168.10.1sarian Active 12 sarian.am 192.168.10.53/24 192.168.10.1syuneci Active 13 syuneci.am 192.168.10.60/24 192.168.10.1znc Active 14 znc.bsd.am 192.168.10.152/24 192.168.10.1
You already get a basic idea of how things are. Each of my blogs (Armenian and English) has its own Jail. Since I’m using WordPress, I need a database, so I have a MySQL jail (which ironically runs MariaDB) inside of it.
I also have a Git server, running gitea, which is down at the moment as I’m doing maintanence. The Git server (and many other services) requires PostgreSQL, hence the existence of a PostgreSQL jail. I run huginn for automation (RSS to Telegram, RSS to XMPP). My sister has her own blog, using WordPress, so that’s a Jail of its own. Same goes about my fiancée.
Other Jails are Newsletter using Listmonk, Sarian (the Armenian instance of lobste.rs) and a personal ZNC server.
As an avid RSS advocate, I also have a RSS Jail, which runs Miniflux. Many of my friends use this service.
Oragir is an instance of WriteFreely, as I advocate public blogging and ActivityPub. Our community uses that too.
The web server that forwards all this traffic from the public to the Jails is nginx. All it does is
proxy_passas needed. It runs on the host.Other services that run on the host are DNS (BIND9), an email service running
OpenSMTPd(which will be moved to a Jail soon), the chat service runningprosody(which will be moved to a Jail soon) and finally, WireGuard, because I love VPNs.Finally, there’s a IPv6-over-IPv4 tunnel that I use to obtain IPv6 thanks to Hurricane Electric.
Yes, I have a firewall, I use
pf(4).For the techies in the room, here’s what my
rc.conflooks like.# cat /etc/rc.conf# Defaultsclear_tmp_enable="YES"syslogd_flags="-ss"sendmail_enable="NONE"#local_unbound_enable="YES"sshd_enable="YES"moused_enable="YES"ntpd_enable="YES"# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disabledumpdev="AUTO"zfs_enable="YES"hostname="pingvinashen.am"# Networkingdefaultrouter="37.157.221.1"gateway_enable="YES"ifconfig_em0="up"vlans_em0="37 1000" # 1000 -> WAN; 37 -> Home Routerifconfig_em0_1000="inet 37.157.221.130 netmask 255.255.255.0"ifconfig_em0_37="inet 192.168.255.2 netmask 255.255.255.0"static_routes="home"route_home="-net 172.16.100.0/24 -gateway 192.168.255.1"cloned_interfaces="bridge0 bridge6 bridge10"ifconfig_bridge10="inet 192.168.10.1 netmask 255.255.255.0"## IPv6ipv6_gateway_enable="YES"gif_interfaces="gif0"gifconfig_gif0="37.157.221.130 216.66.84.46"ifconfig_gif0="inet6 2001:470:1f14:ef::2 2001:470:1f14:ef::1 prefixlen 128"ipv6_defaultrouter="2001:470:1f14:ef::1"ifconfig_em0_37_ipv6="inet6 2001:470:7914:7065::2 prefixlen 64"ipv6_static_routes="home guest"ipv6_route_home="-net 2001:470:7914:6a76::/64 -gateway 2001:470:7914:7065::1"ipv6_route_guest="-net 2001:470:7914:6969::/64 -gateway 2001:470:7914:7065::1"ifconfig_bridge6_ipv6="inet6 2001:470:1f15:e4::1 prefixlen 64"ifconfig_bridge6_aliases="inet6 2001:470:1f15:e4::25 prefixlen 64 \inet6 2001:470:1f15:e4::80 prefixlen 64 \inet6 2001:470:1f15:e4::5222 prefixlen 64 \inet6 2001:470:1f15:e4:c0fe::53 prefixlen 64 \"# VPNwireguard_enable="YES"wireguard_interfaces="wg0"# Firewallpf_enable="YES"# Jailsjail_enable="YES"jailer_dir="zfs:zroot/jails"# DNSnamed_enable="YES"# Mailsmtpd_enable="YES"smtpd_config="/usr/local/etc/smtpd.conf"# XMPPprosody_enable="YES"turnserver_enable="YES"# Webnginx_enable="YES"tor_enable="YES"
The
gif0interface is a IPv6-over-IPv4 tunnel. I have static routes to my home network, so I don’t go to my server over the ISP every time. This also gives me the ability to get IPv6 in my home network that is routed via my home server.As you have guessed from this config file, I do have VLANs setup. So let’s get into that.
The Home Network
First of all, here’s a very cheap diagram
I have the following VLANs setup on the switch.
VLAN IDPurpose1Switch Management1000pingvinashen (home server) WAN1001evn0 (home router) WAN37pingvinashen ↔ evn042Internal Management100Home LAN69Home GuestHere are the active ports
PortVLANsPurpose24untagged: 1Switch management, connects to Port 222untagged: 1000pingvinashen WAN, from ISP21untagged: 1001Home WAN, from ISP20tagged: 1000, 37To pingvinashen, portem019untagged: 1001To home router, portigb118tagged: 42, 100, 69, 99To home router, portigb217untagged: 37To home router, portigb016tagged: 42, 100, 69To Lenovo T480s15untagged: 100To Raspberri Pi 42untagged: 99From Port 24, for switch management1untagged: 42; tagged: 100, 69; PoETo UAP AC ProThe home router, hostnamed
evn0(named after the IATA code of Yerevan’s Zvartnots International Airport) runs FreeBSD as well, the hardware is the followingroot@evn0:~ # dmidecode -s system-product-nameAPU2root@evn0:~ # sysctl hw.modelhw.model: AMD GX-412TC SOC root@evn0:~ # sysctl hw.physmemhw.physmem: 4234399744root@evn0:~ # zpool listNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTzroot 12.5G 9.47G 3.03G - - 67% 75% 1.00x ONLINE -
The home router does… well, routing. It also does DHCP, DNS, SLAAC, and can act as a syslog server.
Here’s what the
rc.conflooks likeclear_tmp_enable="YES"sendmail_enable="NONE"syslogd_flags="-a '172.16.100.0/24:*' -H"zfs_enable="YES"dumpdev="AUTO"hostname="evn0.illuriasecurity.com"pf_enable="YES"gateway_enable="YES"ipv6_gateway_enable="YES"sshd_enable="YES"# Get an IP address from the ISP's GPONifconfig_igb1="DHCP"# Internal routes with pingvinashenifconfig_igb0="inet 192.168.255.1 netmask 255.255.255.0"ifconfig_igb0_ipv6="inet6 2001:470:7914:7065::1 prefixlen 64"static_routes="pingvinashen"route_pingvinashen="-net 37.157.221.130/32 -gateway 192.168.255.2"ipv6_defaultrouter="2001:470:7914:7065::2"# Home Mgmt, Switch Mgmt, Home LAN, Home Guestifconfig_igb2="up"vlans_igb2="42 99 100 69"ifconfig_igb2_42="inet 172.31.42.1 netmask 255.255.255.0"ifconfig_igb2_99="inet 172.16.99.1 netmask 255.255.255.0"ifconfig_igb2_100="inet 172.16.100.1 netmask 255.255.255.0"ifconfig_igb2_100_ipv6="inet6 2001:470:7914:6a76::1 prefixlen 64"ifconfig_igb2_69="inet 192.168.69.1 netmask 255.255.255.0"ifconfig_igb2_69_ipv6="inet6 2001:470:7914:6969::1 prefixlen 64"# DNS and DHCPnamed_enable="YES"dhcpd_enable="YES"named_flags=""# NTPntpd_enable="YES"# Router Advertisement and LLDPrtadvd_enable="YES"lldpd_enable="YES"lldpd_flags=""
Here’s
pf.conf, because security is important.ext_if="igb1"bsd_if="igb0"int_if="igb2.100"guest_if="igb2.69"mgmt_if="igb2.42"sw_if="igb2.99"ill_net="172.16.0.0/16"nat pass on $ext_if from $int_if:network to any -> ($ext_if)nat pass on $ext_if from $mgmt_if:network to any -> ($ext_if)nat pass on $ext_if from $guest_if:network to any -> ($ext_if)set skip on { lo0 }block in allpass on $int_if from $int_if:network to anypass on $mgmt_if from $mgmt_if:network to anypass on $sw_if from $sw_if:network to anypass on $guest_if from $guest_if:network to anyblock quick on $guest_if from any to { $int_if:network, $mgmt_if:network, $ill_net, $sw_if:network }pass in on illuria0 from $ill_net to { $ill_net, $mgmt_if:network }pass inet proto icmppass inet6 proto icmp6pass out all keep stateI’m sure there are places to improve, but it gets the job done and keeps the guest network isolated.
Here’s
rtadvd.conf, for my IPv6 folksigb2.100:\ :addr="2001:470:7914:6a76::":prefixlen#64:\ :rdnss="2001:470:7914:6a76::1":\ :dnssl="evn0.loc.illuriasecurity.com,loc.illuriasecurity.com":igb2.69:\ :addr="2001:470:7914:6969::":prefixlen#64:\ :rdnss="2001:470:7914:6969::1":
For DNS, I’m running BIND, here’s the important parts
listen-on { 127.0.0.1; 172.16.100.1; 172.16.99.1; 172.31.42.1; 192.168.69.1; };listen-on-v6 { 2001:470:7914:6a76::1; 2001:470:7914:6969::1; };allow-query { 127.0.0.1; 172.16.100.0/24; 172.31.42.0/24; 192.168.69.0/24; 2001:470:7914:6a76::/64; 2001:470:7914:6969::/64;};And for DHCP, here’s what it looks like
subnet 172.16.100.0 netmask 255.255.255.0 { range 172.16.100.100 172.16.100.150; option domain-name-servers 172.16.100.1; option subnet-mask 255.255.255.0; option routers 172.16.100.1; option domain-name "evn0.loc.illuriasecurity.com"; option domain-search "loc.illuriasecurity.com evn0.loc.illuriasecurity.com";}host zvartnots { hardware ethernet d4:57:63:f1:5a:36; fixed-address 172.16.100.7;}host unifi0 { hardware ethernet 58:9c:fc:93:d1:0b; fixed-address 172.31.42.42;}
[…]subnet 172.31.42.0 netmask 255.255.255.0 { range 172.31.42.100 172.31.42.150; option domain-name-servers 172.31.42.1; option subnet-mask 255.255.255.0; option routers 172.31.42.1;}subnet 192.168.69.0 netmask 255.255.255.0 { range 192.168.69.100 192.168.69.150; option domain-name-servers 192.168.69.1; option subnet-mask 255.255.255.0; option routers 192.168.69.1;}So you’re wondering, what’s this
unifi0? Well, that brings us toT480s
This laptop has been gifted to me by [REDACTED] for my contributions to the Armenian government (which means when a server goes down and no one knows how to fix it, they called me and I showed up)
Here’s the hardware
root@t480s:~ # dmidecode -s system-versionThinkPad T480sroot@t480s:~ # sysctl hw.modelhw.model: Intel(R) Core(TM) i5-8350U CPU @ 1.70GHzroot@t480s:~ # sysctl hw.physmemhw.physmem: 25602347008root@t480s:~ # zpool listNAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOTzroot 224G 109G 115G - - 44% 48% 1.00x ONLINE -
The T480s has access to VLAN 100, 42, 69, but the host itself has access only to VLAN 100 (LAN), while the jails can exist on other VLANs.
So I have a Jail named
unifi0that runs the Unifi Management thingie.Here’s what
rc.confof the host looks likeclear_tmp_enable="YES"syslogd_flags="-ss"sendmail_enable="NONE"sshd_enable="YES"ntpd_enable="YES"# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disabledumpdev="AUTO"zfs_enable="YES"hostname="t480s.evn0.loc.illuriasecurity.com"ifconfig_em0="up -rxcsum -txcsum"vlans_em0="100 42 69"ifconfig_em0_100="up"ifconfig_em0_42="up"ifconfig_em0_69="up"cloned_interfaces="bridge0 bridge100 bridge42 bridge69"create_args_bridge100="ether 8c:16:45:82:b4:10"ifconfig_bridge100="addm em0.100 SYNCDHCP"ifconfig_bridge100_ipv6="inet6 auto_linklocal"rtsold_flags="-i -F -m bridge100"rtsold_enable="YES"create_args_bridge42=" ether 8c:16:45:82:b4:42"create_args_bridge69=" ether 8c:16:45:82:b4:69"ifconfig_bridge42="addm em0.42"ifconfig_bridge69="addm em0.69"jail_enable="YES"jailer_dir="zfs:zroot/jailer"ifconfig_bridge0="inet 10.1.0.1/24 up"ngbuddy_enable="YES"ngbuddy_private_if="nghost0"dhcpd_enable="YES"lldpd_enable="YES"
I used Jailer to create the
unifi0jail, here’s what thejail.conflooks like# vim: set syntax=sh:exec.clean;allow.raw_sockets;mount.devfs;unifi0 { $id = "6"; devfs_ruleset = 10; $bridge = "bridge42"; $domain = "evn0.loc.illuriasecurity.com"; vnet; vnet.interface = "epair${id}b"; exec.prestart = "ifconfig epair${id} create up"; exec.prestart += "ifconfig epair${id}a up descr vnet-${name}"; exec.prestart += "ifconfig ${bridge} addm epair${id}a up"; exec.start = "/sbin/ifconfig lo0 127.0.0.1 up"; exec.start += "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown jail"; exec.poststop = "ifconfig ${bridge} deletem epair${id}a"; exec.poststop += "ifconfig epair${id}a destroy"; host.hostname = "${name}.${domain}"; path = "/usr/local/jailer/unifi0"; exec.consolelog = "/var/log/jail/${name}.log"; persist; mount.fdescfs; mount.procfs;}Here are the important parts inside the jail
root@t480s:~ # cat /usr/local/jailer/unifi0/etc/rc.confifconfig_epair6b="SYNCDHCP"sendmail_enable="NONE"syslogd_flags="-ss"mongod_enable="YES"unifi_enable="YES"root@t480s:~ # cat /usr/local/jailer/unifi0/etc/start_if.epair6b ifconfig epair6b ether 58:9c:fc:93:d1:0b
Don’t you love it that you can see what’s inside the jail from the host? God I love FreeBSD!
Did I miss anything? I hope not.
Oh, for the homelabbers out there, the T480s is the one that runs things like Jellyfin if needed.
Finally, the tiny
Raspberry Pi 4, Model B
I found this in a closed, so I decided to run it for TimeMachine.
I guess all you care about is
rc.confhostname="tm0.evn0.loc.illuriasecurity.com"ifconfig_DEFAULT="DHCP inet6 accept_rtadv"sshd_enable="YES"sendmail_enable="NONE"sendmail_submit_enable="NO"sendmail_outbound_enable="NO"sendmail_msp_queue_enable="NO"growfs_enable="YES"powerd_enable="YES"# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disabledumpdev="AUTO"zfs_enable="YES"rtsold_enable="YES"samba_server_enable="YES"
And the Samba Configuration
[global]# Network settingsworkgroup = WORKGROUPserver string = Samba Server %vnetbios name = RPi4# Logginglog file = /var/log/samba4/log.%mmax log size = 50log level = 0# Authenticationsecurity = userencrypt passwords = yespassdb backend = tdbsammap to guest = Bad Usermin protocol = SMB2max protocol = SMB3# Apple Time Machine settingsvfs objects = catia fruit streams_xattrfruit:metadata = streamfruit:resource = streamfruit:encoding = nativefruit:locking = nonefruit:time machine = yes# File System supportea support = yeskernel oplocks = nokernel share modes = noposix locking = nomangled names = nosmbd max xattr size = 2097152# Performance tuningread raw = yeswrite raw = yesgetwd cache = yesstrict locking = no# Miscellaneouslocal master = nopreferred master = nodomain master = nowins support = no[tm]comment = Time Machine RPi4path = /usr/local/timemachine/%Ubrowseable = yesread only = novalid users = antranigvvfs objects = catia fruit streams_xattrfruit:time machine = yesfruit:advertise_fullsync = truefruit:time machine max size = 800G # Adjust the size according to your needscreate mask = 0600directory mask = 0700
That’s pretty much it.
Conclusion
I love running homebrew servers, home networks and home labs. I love that (almost) everything is FreeBSD. The switch itself runs Linux, and the Unifi Access Point also runs Linux, both of which I’m pretty happy with.
While most homelabbers used ESXi in the past, I’m happy to see that most people are moving to open source solutions like Proxmox and Xen, but I think that FreeBSD Jails and bhyve is much better. I still don’t have a need for bhyve at the moment, but I would use it if I needed hardware virtualization.
Most homelabbers would consider the lack of Web/GUI interfaces as a con, but I think that it’s a pro. If I need to “replicate” this network, all I need to do is to copy some text files and modify some IP addresses / Interface names.
I hope this was informative and that it would be useful for anyone in the future.
That’s all folks…
Reply via email.
https://antranigv.am/posts/2024/06/freebsd-server-network-homelab/
#Containers #Dell #DellLatitudeE5470 #FreeBSD #homeServer #HowTo #Jailer #Jails #macOS #Networking #pf #Samba #Unifi #Unix #VNET