home.social

Search

449 results for “cmdln”

  1. Melanie De Vrieze @[email protected] ·

    Interessant artikel van @jnwlrvn en @FrankOlbrechts over de rol en het belang van de openbare #bibliotheek. De bibliotheek is inderdaad meer dan alleen maar boeken uitlenen.

    #thirdplace #dekrook #krook #gent #muntpunt #library #apache

    apache.be/2023/01/27/paleizen-

    #krook #apache #library #muntpunt #gent #dekrook
  2. Melanie De Vrieze @[email protected] ·

    Interessant artikel van @jnwlrvn en @FrankOlbrechts over de rol en het belang van de openbare #bibliotheek. De bibliotheek is inderdaad meer dan alleen maar boeken uitlenen.

    #thirdplace #dekrook #krook #gent #muntpunt #library #apache

    apache.be/2023/01/27/paleizen-

    #krook #bibliotheek #thirdplace #dekrook #gent #muntpunt
  3. Melanie De Vrieze @[email protected] ·

    Interessant artikel van @jnwlrvn en @FrankOlbrechts over de rol en het belang van de openbare #bibliotheek. De bibliotheek is inderdaad meer dan alleen maar boeken uitlenen.

    #thirdplace #dekrook #krook #gent #muntpunt #library #apache

    apache.be/2023/01/27/paleizen-

    #krook #apache #library #muntpunt #gent #dekrook
  4. Kevin @[email protected] ·

    When two Hetzner servers died at the same time

    On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

    Several hours went into hypotheses that turned out to be wrong:

    • The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
    • PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
    • Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
    • Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
    • Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

    The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

    What it almost certainly was

    Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

    • Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
    • New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

    Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

    GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

    One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

    Why it matters for anyone running this stack

    If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

    Tooling

    While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

    → github.com/kevinveenbirkenbach/hetzner-arch-luks

    Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

    #ArchLinux #bootFailure #debugging #DevOps #DHCP #Dropbear #fullDiskEncryption #GRUB #Hetzner #initramfs #kernelUpgrade #Linux #LUKS #mkinitcpio #pacman #postmortem #PythonCLI #serverOutage #sysadmin #systemdNetworkd
    #archlinux #bootfailure #debugging #devops #dhcp #dropbear
  5. Kevin @[email protected] ·

    When two Hetzner servers died at the same time

    On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

    Several hours went into hypotheses that turned out to be wrong:

    • The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
    • PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
    • Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
    • Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
    • Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

    The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

    What it almost certainly was

    Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

    • Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
    • New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

    Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

    GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

    One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

    Why it matters for anyone running this stack

    If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

    Tooling

    While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

    → github.com/kevinveenbirkenbach/hetzner-arch-luks

    Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

    #ArchLinux #bootFailure #debugging #DevOps #DHCP #Dropbear #fullDiskEncryption #GRUB #Hetzner #initramfs #kernelUpgrade #Linux #LUKS #mkinitcpio #pacman #postmortem #PythonCLI #serverOutage #sysadmin #systemdNetworkd
    #archlinux #bootfailure #debugging #devops #dhcp #dropbear
  6. Kevin @[email protected] ·

    When two Hetzner servers died at the same time

    On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

    Several hours went into hypotheses that turned out to be wrong:

    • The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
    • PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
    • Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
    • Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
    • Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

    The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

    What it almost certainly was

    Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

    • Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
    • New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

    Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

    GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

    One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

    Why it matters for anyone running this stack

    If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

    Tooling

    While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

    → github.com/kevinveenbirkenbach/hetzner-arch-luks

    Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

    #ArchLinux #bootFailure #debugging #DevOps #DHCP #Dropbear #fullDiskEncryption #GRUB #Hetzner #initramfs #kernelUpgrade #Linux #LUKS #mkinitcpio #pacman #postmortem #PythonCLI #serverOutage #sysadmin #systemdNetworkd
    #archlinux #bootfailure #debugging #devops #dhcp #dropbear
  7. Kevin @[email protected] ·

    When two Hetzner servers died at the same time

    On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

    Several hours went into hypotheses that turned out to be wrong:

    • The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
    • PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
    • Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
    • Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
    • Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

    The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

    What it almost certainly was

    Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

    • Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
    • New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

    Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

    GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

    One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

    Why it matters for anyone running this stack

    If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

    Tooling

    While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

    → github.com/kevinveenbirkenbach/hetzner-arch-luks

    Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

    #ArchLinux #bootFailure #debugging #DevOps #DHCP #Dropbear #fullDiskEncryption #GRUB #Hetzner #initramfs #kernelUpgrade #Linux #LUKS #mkinitcpio #pacman #postmortem #PythonCLI #serverOutage #sysadmin #systemdNetworkd
    #systemdnetworkd #sysadmin #serveroutage #pythoncli #postmortem #pacman
  8. Kevin @[email protected] ·

    When two Hetzner servers died at the same time

    On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

    Several hours went into hypotheses that turned out to be wrong:

    • The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
    • PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
    • Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
    • Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
    • Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

    The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

    What it almost certainly was

    Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

    • Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
    • New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

    Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

    GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

    One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

    Why it matters for anyone running this stack

    If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

    Tooling

    While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

    → github.com/kevinveenbirkenbach/hetzner-arch-luks

    Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

    #ArchLinux #bootFailure #debugging #DevOps #DHCP #Dropbear #fullDiskEncryption #GRUB #Hetzner #initramfs #kernelUpgrade #Linux #LUKS #mkinitcpio #pacman #postmortem #PythonCLI #serverOutage #sysadmin #systemdNetworkd
    #archlinux #bootfailure #debugging #devops #dhcp #dropbear
  9. Kevin's Blog @[email protected] ·

    TD4 4-bit DIY CPU – Part 7

    Once the idea was floated, in Part 6 of creating an Arduino “direct to ROM” assembler, I had to just do it, so this post is a little diversion from the hardware discussion into how that could work.

    • Part 1 – Introduction, Discussion and Analysis
    • Part 2 – Building and Hardware
    • Part 3 – Programming and Simple Programs
    • Part 4 – Some hardware enhancements
    • Part 5 – My own PCB version
    • Part 6 – Replacing the ROM with a microcontroller
    • Part 7 – Creating an Arduino “assembler” for the TD4

    Basic Concepts

    This relies on using an Arduino as the ROM as described in Part 6, but the Arduino now has the option to change the ROM contents independently of the TD4 itself.

    The Arduino sketch will do the following:

    • Run the TD4 ROM routine off a timer interrupt so that it is always running and responsive.
    • Take input over the Arduino serial port to allow basic control, e.g. list, clear, etc.
    • Allow the direct input of assembler instructions, such as MOVE A,B or OUT B and so on.
    • Provide a means of selecting which line of the program to change.

    The code will thus have a number of key sections:

    • The TD4 ROM routine.
    • Some kind of serial-port command-line interpreter.
    • Handler routines for all the commands.
    • An assembler.
    • A disassembler.

    The TD4 ROM routine has already been fully described in Part 6. The only difference is that the scanning routine will be driven from a 1mS timer using the TimerOne library.

    As I want to still support a built-in demo, I now have the concept of ROM being the demo code and RAM being the “live” code to pass onto the TD4. The Arduino will initialise the RAM on startup from the ROM.

    As far as the TD4 is concerned of course, this is all still ROM.

    Command Line Interpreter

    The standard Arduino Serial routines will be used to scan for input via the serial port. It will support a line-oriented input as follows:

    bool cmdRunner (void) {
      while (Serial.available()) {
        char c = Serial.read();
        if (c == '\n') {
          strcpy(cmdSaved, cmdInput);
          cmdIdx = 0;
          return true;
        }
        else if (cmdIdx < CMD_BUFFER-1) {
          cmdInput[cmdIdx++] = c;
          cmdInput[cmdIdx] = '\0';
        }
      }
      return false;
    }

    This will keep adding any received characters to the cmdInput buffer until a newline is received, at which point the command is saved in cmdSaved and the routine will return true indicating a full line is ready to be processed.

    Once a complete line is received, then a processing function will parse it.

    Key to the processing of commands is a command table that stores the text to match and the handler function to call on finding a valid command. There is an additional parameter that will be passed into the handler function to allow the same handler function to support several commands. This will be used in the assembler itself later.

    struct cmd_t {
      char    cmd[CMD_BUFFER+1];
      hdlr_t  pFn;
      uint8_t idx;
    };

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
    };

    The algorithm for parsing commands is as follows:

    cmdProcess:
      Look for a space or newline
      IF found a space THEN
        This is the start of the parameter

      Look for the command in the command table
      IF command found THEN
        Call the handler function with the parameters

    The implementation is a bit complex, as it uses string pointers and has to chop and parse strings as it goes. It is also detailing with the command table in the Arduino’s PROGMEM which is an additional complication too.

    In order to be able to use the same command line interpreter for the input of assembler instructions, I’ve had to simplify the syntax. There are no spaces in opcodes and there has to be a space between the opcode and immediate value if used.

    Here are some examples:

    IN A       -> INA
    MOVE A,B   -> MOVAB
    OUT im     -> OUT im
    JNC im     -> JNC im
    ADD A,im   -> ADDA im

    Handler Routines

    All handler routines have the following prototype:

    typedef void (*hdlr_t)(int idx, char *param);

    void hdlrHelp(int idx, char *pParam) {
      Serial.print("\nHelp\n----\n");
      Serial.println("H: Help");
    }

    The idx parameter is the number in the last field of the command table. pParam will be a pointer to the parameter string for the command (if used).

    As we’re dealing with strings all the time, there are a number of helper functions to do things like convert strings to numbers as well as others to print numbers in various formats.

    Number formats are assumed to be as follows:

    0..9   - decimal digits
    0x0..F - hex digits
    b0..1  - binary digits

    The code provides the following:

    • str2num – the basic string parsing routine to recognise all three number formats as strings.
    • printbin – print a number in b0..1 format.
    • printhex – print a number in 0x0..F format, allowing for a possible leading zero if required.
    • printins – print an instruction in textual format.
    • printop – print an instruction in binary and hex opcode format.
    • printline – print a line number in a consistent binary and hex format.

    The code supports the following commands, so each has its own handler function:

    • H – help – show the list of commands.
    • L – list – show the disassembly of the whole working memory (RAM).
    • G – goto – set the working line number.
    • C – clear – reset all working memory (RAM) to zeros.
    • R – restore – restore the working memory (RAM) to the pre-build demo code (ROM).
    • O – opcodes – list the supported opcodes.

    Assembler

    As already mentioned, I’m using the same command line interpreter code to create the assembler. To do this, each opcode has an entry in the command table:

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      // Assembly commands - must be first
      {"ADDA", hdlrAsm, 0},
      {"MOVAB", hdlrAsm, 1},
      {"INA", hdlrAsm, 2},
      {"MOVA", hdlrAsm, 3},
      {"MOVBA", hdlrAsm, 4},
      {"ADDB", hdlrAsm, 5},
      {"INB", hdlrAsm, 6},
      {"MOVB", hdlrAsm, 7},
      {"OUTB", hdlrAsm, 8},
      {"OUT2B", hdlrAsm, 9},
      {"OUT", hdlrAsm, 10},
      {"OUT2", hdlrAsm, 11},
      {"JNCB", hdlrAsm, 12},
      {"JMPB", hdlrAsm, 13},
      {"JNC", hdlrAsm, 14},
      {"JMP", hdlrAsm, 15},

      // Other commands
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
      {"C", hdlrClear, 0},
      {"R", hdlrRestore, 0},
      {"O", hdlrOpcodes, 0},
    };

    The order corresponds to the opcode command value, as does the parameter. As these are at the start of the table, I can assume that the position in the table is the same as the command value. This does mean that I also need to account for the duplicated instructions even if I don’t need to use them.

    I’m making the following design decisions:

    • There is the concept of a “current line” which can be set with the G (goto) command.
    • Entering a valid opcode automatically moves the current line on by 1.
    • No line information is entered as part of the opcode.

    The main logic of the assembler handler is as follows:

    Assembler:
      Command value is the provided index parameter
      Determine the im value from the provided string parameter
      RAM[line] = cmd << 4 + im
      Increment current line

    Disassembler

    Disassembly is really largely a look-up table matching opcode command values to text. This is all hidden away behind the two print routines printins() and printop().

    void printins (uint8_t ins) {
      uint8_t cmd = ins >> 4;
      uint8_t im = ins & 0x0F;

      Serial.print(FSH(cmdTable[cmd].cmd));
      if (HASIM(cmd)) {
        Serial.print(" b");
        printbin(im,4);
      } else {
        Serial.print("      ");
      }
    }

    void printop (uint8_t op) {
      uint8_t cmd = op >> 4;
      uint8_t im = op & 0x0F;

      Serial.print("b");
      printbin(cmd,4);
      Serial.print(" ");
      printbin(im,4);
      Serial.print("\t0x");
      printhex(op,2);
    }

    The main complexity is pulling the strings out of the command table. I’ve had to include a macro to provide access to the strings from the Arduino’s PROGMEM:

    #define FSH(x) ((const __FlashStringHelper *)x)

    This feels like a bit of a hack, but apparently this is how it should be done for the kind of thing I need to do!

    There is another macro here that needs explaining:

    #define HASIM(op) (op==0||op==3||op==5||op==7||op>9)

    This is a set of conditions that if true means that the command supports an immediate value. This is used in a few places to know how to parse the commands.

    Whilst in principle all commands could use the immediate value, the “official” statement of how they work assumes im=0 in many cases. So, for example, OUT B does not require an immediate value, but if one is provided then OUT B becomes OUT B+im.

    I’m not really supporting that with this code at the moment.

    Putting it all together

    Here is a serial output log of a session using the assembler.

    > H
    Help
    ----
    H: Help
    L: List
    G: Goto
    C: Clear
    R: Restore
    O: Opcodes
    OpCode
    OpCode im

    Current line: b0000 [0]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: ADDA b0000b0000 00000x00
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b0010 [2]

    > G 13
    Goto line 13
    Current line: b1101 [D]

    > OUTB
    Assemble:
    b1101 [D] OUTB      b1000 00000x80
    Current line: b1110 [E]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: OUTB      b1000 00000x80
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b1110 [E]

    > O
    Supported OpCodes:
      b0000 dataADDA im
      b0001 0000MOVAB
      b0010 0000INA
      b0011 dataMOVA im
      b0100 0000MOVBA
      b0101 dataADDB im
      b0110 0000INB
      b0111 dataMOVB im
      b1000 0000OUTB
      b1001 0000OUT2B
      b1010 dataOUT im
      b1011 dataOUT2 im
      b1100 dataJNCB im
      b1101 dataJMPB im
      b1110 dataJNC im
      b1111 dataJMP im

    > C
    Clearing RAM ... Done

    Find the code on GitHub here.

    Conclusion

    The basics for this actually came together fairly quickly, but I must admit to spending a fair bit of time fiddling about with output formats and refactoring various bits of code to try to give some consistency in terms of when newlines are applied, what is shown in binary, what in hex, and so on.

    I can’t guarantee everything has been caught, but I’ve typed in all the code (using the newer, limited syntax) from Part 3 and they all seem to work.

    It would be nice to be able to automatically reset the TD4 from the Arduino, but for now, pressing the button when required is fine.

    For the most part, unless there is a loop to get caught in, the code will cycle back to the start anyway.

    In terms of possible updates and enhancements, there are a few on my mind:

    • It would be nice to support the undocumented use of immediate values somehow.
    • It might be nice to have a way to save/load the code. It only needs to be a string of 16 2-byte hex codes.
    • It might be nice to have several demo programs to choose from.

    If I expand the instruction set and architecture, then I’ll have to think again about chunks of this code, but for now, it seems to work pretty well.

    Kevin

    #4bit #arduinoUno #define #td4

    #4bit #arduinouno #define #td4
  10. jejb @[email protected] ·

    Back in my blog post about Securing the Google SIP Stack, I did say I’d look at re-enabling SIP in Android-12, so with a view to doing that I tried building and booting LineageOS 19.1, but it crashed really early in the boot sequence (after the boot splash but before the boot animation started). It turns out that information on debugging the android early boot sequence is a bit scarce, so I thought I should write a post about how I did it just in case it helps someone else who’s struggling with a similar early boot problem.

    How I usually Build and Boot Android

    My builds are standard LineageOS with my patches to fix SIP and not much else. However, I do replace the debug keys with my signing keys and I also have an AVB key installed in the phone’s third party keyslot with which I sign the vbmeta for boot. This actually means that my phone is effectively locked but with a user supplied key (Yellow as google puts it).

    My phone is now a pixel 3 (I had to say goodbye to the old Nexus One thanks to the US 3G turn off) and I do have a slightly broken Pixel 3 I play with for experimental patches, which is where I was trying to install Android-12.

    Signing Seems to be the Problem

    Just to verify my phone could actually boot a stock LineageOS (it could) I had to unlock it and this lead to the discovery that once unlocked, it would also boot my custom rom as well, so whatever was failing in early boot seemed to be connected with the device being locked.

    I also discovered an interesting bug in the recovery rom fastboot: If you’re booting locked with your own keys, it will still let you perform all the usually forbidden fastboot commands (the one I was using was set_active). It turns out to be because of a bug in AOSP which treats yellow devices as unlocked in fastboot. Somewhat handy for debugging, but not so hot for security …

    And so to Debugging Early Boot

    The big problem with Android is there’s no way to get the console messages for early boot. Even if you enable adb early, it doesn’t get started until quite far in to the boot animation (which was way after the crash I was tripping over). However, android does have a pstore (previously ramoops) driver that can give you access to the previously crashed boot’s kernel messages (early init, fortunately, mostly logs to the kernel message log).

    Forcing init to crash on failure

    Ordinarily an init failure prints a message and reboots (to the bootloader), which doesn’t excite pstore into saving the kernel message log. fortunately there is a boot option (androidboot.init_fatal_panic) which can be set in the boot options (or kernel command line for a pixel-3 which can only boot the 4.9 kernel). If you build your own android, it’s fairly easy to add things to the android commandline (which is in boot.img) because all you need to do is extract BOOT/cmdline from the intermediate zip file you sign add any boot options you need and place it back in the zip file (before you sign it).

    Unfortunately, this expedient didn’t work (no console logs appear in pstore). I did check that init was correctly panic’ing on failure by inducing an init failure in recovery mode and observing the panic (recovery mode allows you to run adb). But this induced panic also didn’t show up in pstore, meaning there’s actually some problem with pstore and early panics.

    Security is the problem (as usual)

    The actual problem turned out to be security (as usual): The pixel-3 does encrypted boot panic logs. The way this seems to work (at least in my reading of the google additional pstore patches) is that the bootloader itself encrypts the pstore ram area with a key on the /data partition, which means it only becomes visible after the device is unlocked. Unfortunately, if you trigger a panic before the device is unlocked (by echoing ‘c’ to /proc/sysrq-trigger) the panic message is lost, so pstore itself is useless for debugging early boot. There seems to be some communication of the keys by the vendor proprietary ramoops binary making it very difficult to figure out how it’s being done.

    Why the early panic message is lost is a bit mysterious, but unfortunately pstore on the pixel-3 has several proprietary components around the encrypted message handling that make it hard to debug. I suspect if you don’t set up the pstore encryption keys, the bootloader erases the pstore ram area instead of encrypting it, but I can’t prove that.

    Although it might be possible to fix the pstore drivers to preserve the ramoops from before device unlock, the participation of the proprietary bootloader in preserving the memory doesn’t make that look like a promising avenue to explore.

    Anatomy of the Pixel-3 Boot Sequence

    The Pixel-3 device boots through recovery. What this means is that the initial ramdisk (from boot.img) init is what boots both the recovery and normal boot paths. The only difference is that for recovery (and fastboot), the device stays in the ramdisk and for normal boot it mounts the /system partition and pivots to it. What makes this happen or not is the boot flag androidboot.force_normal_boot=1 which is added by the bootloader. Pretty much all the binary content and init rc files in the ramdisk are for recovery and its allied menus.

    Since the boot paths are pretty radically different, because the normal boot first pivots to a first stage before going on to a second, but in the manner of containers, it might be possible to boot recovery first, start a dmesg logger and then re-exec init through the normal path

    Forcing Re-Exec

    The idea is to signal init to re-exec itself for the normal path. Of course, there have to be a few changes to do this: An item has to be added to the recovery menu to signal init and init itself has to be modified to do the re-exec on the signal (note you can’t just kick off an init with a new command line because init must be pid 1 for booting). Once this is done, there are problems with selinux (it won’t actually allow init to re-exec) and some mount moves. The selinux problem is fixable by switching it from enforcing to permissive (boot option androidboot.selinux=permissive) and the mount moves (which are forbidden if you’re running binaries from the mount points being moved) can instead become bind mounts. The whole patch becomes 31 insertions across 7 files in android_system_core.

    The signal I chose was SIGUSR1, which isn’t usually used by anything in the bootloader and the addition of a menu item to recovery to send this signal to init was also another trivial patch. So finally we have a system from which I can start adb to trace the kernel log (adb shell dmesg -w) and then signal to init to re-exec. Surprisingly this worked and produced as the last message fragment:

    [ 190.966881] init: [libfs_mgr]Created logical partition system_a on device /dev/block/dm-0[ 190.967697] init: [libfs_mgr]Created logical partition vendor_a on device /dev/block/dm-1[ 190.968367] init: [libfs_mgr]Created logical partition product_a on device /dev/block/dm-2[ 190.969024] init: [libfs_mgr]Created logical partition system_ext_a on device /dev/block/dm-3[ 190.969067] init: DSU not detected, proceeding with normal boot[ 190.982957] init: [libfs_avb]Invalid hash size:[ 190.982967] init: [libfs_avb]Failed to verify vbmeta digest[ 190.982972] init: [libfs_avb]vbmeta digest error isn't allowed[ 190.982980] init: Failed to open AvbHandle: No such file or directory[ 190.982987] init: Failed to setup verity for '/system': No such file or directory[ 190.982993] init: Failed to mount /system: No such file or directory[ 190.983030] init: Failed to mount required partitions early …[ 190.983483] init: InitFatalReboot: signal 6[ 190.984849] init: #00 pc 0000000000123b38 /system/bin/init[ 190.984857] init: #01 pc 00000000000bc9a8 /system/bin/init[ 190.984864] init: #02 pc 000000000001595c /system/lib64/libbase.so[ 190.984869] init: #03 pc 0000000000014f8c /system/lib64/libbase.so[ 190.984874] init: #04 pc 00000000000e6984 /system/bin/init[ 190.984878] init: #05 pc 00000000000aa144 /system/bin/init[ 190.984883] init: #06 pc 00000000000487dc /system/lib64/libc.so[ 190.984889] init: Reboot ending, jumping to kernel

    Which indicates exactly where the problem is.

    Fixing the problem

    Once the messages are identified, the problem turns out to be in system/core ec10d3cf6 “libfs_avb: verifying vbmeta digest early”, which is inherited from AOSP and which even says in in it’s commit message “the device will not boot if: 1. The image is signed with FLAGS_VERIFICATION_DISABLED is set 2. The device state is locked” which is basically my boot state, so thanks for that one google. Reverting this commit can be done cleanly and now the signed image boots without a problem.

    I note that I could also simply add hashtree verification to my boot, but LineageOS is based on the eng target, which has FLAGS_VERIFICATION_DISABLED built into the main build Makefile. It might be possible to change it, but not easily I’m guessing … although I might try fixing it this way at some point, since it would make my phones much more secure.

    Conclusion

    Debugging android early boot is still a terribly hard problem. Probably someone with more patience for disassembling proprietary binaries could take apart pixel-3 vendor ramoops and figure out if it’s possible to get a pstore oops log out of early boot (which would be the easiest way to debug problems). But failing that the simple hack to re-exec init worked enough to show me where the problem was (of course, if init had continued longer it would likely have run into other issues caused by the way I hacked it).

    https://blog.hansenpartnership.com/debugging-android-early-boot-failures/

    #00 #01 #02 #03 #04 #05 #06 #android #androidBoot #androidDebugging #androidVerifiedBoot #AVB #lineageos

    #android #androidboot #androiddebugging #androidverifiedboot #avb #lineageos
  11. Club Furies @[email protected] ·

    CF Premiere: Kidz On Acid – Horizon [Cumulonimbus]

    The German label Cumulonimbus has put together a series of fundraising campaigns with a mission as clear as it is necessary: to reach people in Sudan and Gaza living through conditions of extreme vulnerability. The premise is both simple and forceful —music as a vehicle for real change, harnessing the reach that electronic culture has built to stand behind those who need it most. This is not a gesture for optics: it is a concrete, ongoing, and fully transparent effort.

    https://clubfuries.com.mx/2026/04/04/cfp-merv-f33d-cumulonimbus/

    The label wishes to express its sincere gratitude to every artist who has contributed and helped bring this initiative to life, along with anyone who may still choose to be part of it. Periodic updates will be published detailing the funds raised and how each contribution is allocated, with transparency as a non-negotiable foundation. The pledge stands firm: 100% of proceeds will be donated.

    https://soundcloud.com/clubfuriess/cfp-kidz-on-acid-cumulonimbus

    At Club Furies, being part of this project means something real to us. We believe electronic music holds a responsibility that reaches far beyond the club, and when a label like Cumulonimbus puts its catalog and its community behind something like this, our instinct is simply to help that message travel as far as it can. We are sharing two premieres through our channel —because if there is anything we can offer from here, it is making sure this reaches the widest possible audience, and that every euro raised goes to Palestine and Sudan.

    The second of those premieres arrives from France, from the artist Kidz On Acid, presenting Horizon. There is nothing to explain here, no framework to build around it or context to impose. Only sound —and within that sound, every ounce of solidarity and conviction we can give.

    Support the Fundraiser

    Title: c:fundraiser sudan/gaza vol. 1
    Artist: Various Artists
    Label: Cumulonimbus
    Catalogue: CMLNMBS24
    Format: Digital
    Genre: Electronic
    Style: Techno
    Mastering: Younes Jamil
    Artwork: DJ DEKADENT, Mara Vorberg

    Release date: April 15th, 2026
    Support & Buy: Bandcamp

    Tracklist
    1. Merv – F33D
    2. Carl Raban – Eria
    3. Svedstorm – Sonnenallee 05:24
    4. Tara Namir, Younes Jamil – In my bed
    5. Bekkler – Chaos Drifter
    6. Symbolism – Onir
    7. Indigo Plateaux – Gumo
    8. QLP – Now
    9. Lenny San – Aktor
    10. Nexus J – Missing You
    11. James Smyth – Sunday Nights
    12. Eramo – Bipolar System
    13. Antonio Fevola – Unity Protocol
    14. Alex Asci – Zeros and Ones
    15. dom.koski – arrangement 2
    16. Ikram Shinwari – Cheating Death
    17. Kidz On Acid – Horizon

    Kidz On Acid

    SoundCloud | Instagram | Bandcamp

    Cumulonimbus

    SoundCloud | Instagram | Bandcamp | Linktree

    Club Furies

    Website | SoundCloud | Instagram | Facebook | Bandcamp | Threads | Substack | Linktree

    #Acid #Berlin #cFundraiserSudanGazaVol1 #Electronic #Electronica #France #Fundraiser #Gaza #Germany #KidzOnAcid #Palestine #Sudan #Support #techno #VA #VariousArtists
    #acid #berlin #cfundraisersudangazavol1 #electronic #electronica #france
  12. Club Furies @[email protected] ·

    CF Premiere: Kidz On Acid – Horizon [Cumulonimbus]

    The German label Cumulonimbus has put together a series of fundraising campaigns with a mission as clear as it is necessary: to reach people in Sudan and Gaza living through conditions of extreme vulnerability. The premise is both simple and forceful —music as a vehicle for real change, harnessing the reach that electronic culture has built to stand behind those who need it most. This is not a gesture for optics: it is a concrete, ongoing, and fully transparent effort.

    https://clubfuries.com.mx/2026/04/04/cfp-merv-f33d-cumulonimbus/

    The label wishes to express its sincere gratitude to every artist who has contributed and helped bring this initiative to life, along with anyone who may still choose to be part of it. Periodic updates will be published detailing the funds raised and how each contribution is allocated, with transparency as a non-negotiable foundation. The pledge stands firm: 100% of proceeds will be donated.

    https://soundcloud.com/clubfuriess/cfp-kidz-on-acid-cumulonimbus

    At Club Furies, being part of this project means something real to us. We believe electronic music holds a responsibility that reaches far beyond the club, and when a label like Cumulonimbus puts its catalog and its community behind something like this, our instinct is simply to help that message travel as far as it can. We are sharing two premieres through our channel —because if there is anything we can offer from here, it is making sure this reaches the widest possible audience, and that every euro raised goes to Palestine and Sudan.

    The second of those premieres arrives from France, from the artist Kidz On Acid, presenting Horizon. There is nothing to explain here, no framework to build around it or context to impose. Only sound —and within that sound, every ounce of solidarity and conviction we can give.

    Support the Fundraiser

    Title: c:fundraiser sudan/gaza vol. 1
    Artist: Various Artists
    Label: Cumulonimbus
    Catalogue: CMLNMBS24
    Format: Digital
    Genre: Electronic
    Style: Techno
    Mastering: Younes Jamil
    Artwork: DJ DEKADENT, Mara Vorberg

    Release date: April 15th, 2026
    Support & Buy: Bandcamp

    Tracklist
    1. Merv – F33D
    2. Carl Raban – Eria
    3. Svedstorm – Sonnenallee 05:24
    4. Tara Namir, Younes Jamil – In my bed
    5. Bekkler – Chaos Drifter
    6. Symbolism – Onir
    7. Indigo Plateaux – Gumo
    8. QLP – Now
    9. Lenny San – Aktor
    10. Nexus J – Missing You
    11. James Smyth – Sunday Nights
    12. Eramo – Bipolar System
    13. Antonio Fevola – Unity Protocol
    14. Alex Asci – Zeros and Ones
    15. dom.koski – arrangement 2
    16. Ikram Shinwari – Cheating Death
    17. Kidz On Acid – Horizon

    Kidz On Acid

    SoundCloud | Instagram | Bandcamp

    Cumulonimbus

    SoundCloud | Instagram | Bandcamp | Linktree

    Club Furies

    Website | SoundCloud | Instagram | Facebook | Bandcamp | Threads | Substack | Linktree

    #Acid #Berlin #cFundraiserSudanGazaVol1 #Electronic #Electronica #France #Fundraiser #Gaza #Germany #KidzOnAcid #Palestine #Sudan #Support #techno #VA #VariousArtists
    #acid #berlin #cfundraisersudangazavol1 #electronic #electronica #france
  13. Club Furies @[email protected] ·

    CF Premiere: Kidz On Acid – Horizon [Cumulonimbus]

    The German label Cumulonimbus has put together a series of fundraising campaigns with a mission as clear as it is necessary: to reach people in Sudan and Gaza living through conditions of extreme vulnerability. The premise is both simple and forceful —music as a vehicle for real change, harnessing the reach that electronic culture has built to stand behind those who need it most. This is not a gesture for optics: it is a concrete, ongoing, and fully transparent effort.

    https://clubfuries.com.mx/2026/04/04/cfp-merv-f33d-cumulonimbus/

    The label wishes to express its sincere gratitude to every artist who has contributed and helped bring this initiative to life, along with anyone who may still choose to be part of it. Periodic updates will be published detailing the funds raised and how each contribution is allocated, with transparency as a non-negotiable foundation. The pledge stands firm: 100% of proceeds will be donated.

    https://soundcloud.com/clubfuriess/cfp-kidz-on-acid-cumulonimbus

    At Club Furies, being part of this project means something real to us. We believe electronic music holds a responsibility that reaches far beyond the club, and when a label like Cumulonimbus puts its catalog and its community behind something like this, our instinct is simply to help that message travel as far as it can. We are sharing two premieres through our channel —because if there is anything we can offer from here, it is making sure this reaches the widest possible audience, and that every euro raised goes to Palestine and Sudan.

    The second of those premieres arrives from France, from the artist Kidz On Acid, presenting Horizon. There is nothing to explain here, no framework to build around it or context to impose. Only sound —and within that sound, every ounce of solidarity and conviction we can give.

    Support the Fundraiser

    Title: c:fundraiser sudan/gaza vol. 1
    Artist: Various Artists
    Label: Cumulonimbus
    Catalogue: CMLNMBS24
    Format: Digital
    Genre: Electronic
    Style: Techno
    Mastering: Younes Jamil
    Artwork: DJ DEKADENT, Mara Vorberg

    Release date: April 15th, 2026
    Support & Buy: Bandcamp

    Tracklist
    1. Merv – F33D
    2. Carl Raban – Eria
    3. Svedstorm – Sonnenallee 05:24
    4. Tara Namir, Younes Jamil – In my bed
    5. Bekkler – Chaos Drifter
    6. Symbolism – Onir
    7. Indigo Plateaux – Gumo
    8. QLP – Now
    9. Lenny San – Aktor
    10. Nexus J – Missing You
    11. James Smyth – Sunday Nights
    12. Eramo – Bipolar System
    13. Antonio Fevola – Unity Protocol
    14. Alex Asci – Zeros and Ones
    15. dom.koski – arrangement 2
    16. Ikram Shinwari – Cheating Death
    17. Kidz On Acid – Horizon

    Kidz On Acid

    SoundCloud | Instagram | Bandcamp

    Cumulonimbus

    SoundCloud | Instagram | Bandcamp | Linktree

    Club Furies

    Website | SoundCloud | Instagram | Facebook | Bandcamp | Threads | Substack | Linktree

    #Acid #Berlin #cFundraiserSudanGazaVol1 #Electronic #Electronica #France #Fundraiser #Gaza #Germany #KidzOnAcid #Palestine #Sudan #Support #techno #VA #VariousArtists
    #variousartists #va #techno #support #sudan #palestine
  14. Wes Lambert @[email protected] ·

    🦖Day 74 of the @velocidex #velociraptor #ArtifactsOfAutumn series

    Artifact: Windows[.]Persistence[.]PowershellProfile

    Author: @mgreen27

    Link: docs.velociraptor.app/artifact

    ----

    PowerShell supports several profiles depending on the user or host program. Adversaries may create or modify these profiles to include arbitrary commands, functions, modules, and/or PowerShell drives to gain persistence.

    ----

    When a backdoored PowerShell session is opened, the modified script will be executed unless the '-NoProfile' flag is used upon launch.

    An adversary may also be able to escalate privileges if a script in a PS profile is loaded and executed by an account with higher privileges, for example, a domain administrator.

    ----

    In the past, Turla has used PowerShell profiles to maintain persistence on an infected machine.

    attack.mitre.org/groups/G0010
    welivesecurity.com/2019/05/29/

    ----

    This artifact will search and parse PowerShell profile scripts.

    By default, both user and system-wide profiles will be searched. The user can also use regex to target and exclude specific content.

    ----

    Here (image), we can see that the PowerShell profile for the user 'wlambert' specifies that 'Start-Process' should call 'C:\User\Downloads\wlambert\malz.exe'. Again, this would be called every time a PowerShell session is initiated. 👀

    ----

    In this instance, 'malz.exe' is simply a copy of good 'ol calc.exe 😀

    ----

    This profile modification was simulated by running the following commands from a PS session:

    - 'Add-Content $profile -Value ""'
    - 'Add-Content $profile -value "Start-Process C:\Users\wlambert\Downloads\malz.exe"'

    The profile content can be checked with 'Get-Content $profile'.

    ----

    That's it for now! Stay tuned to learn about more artifacts! 🦖

    Also, check out the links below for more information about Powershell Profiles!

    Atomic Red Team Test:
    atomicredteam.io/persistence/T

    MITRE ATT&CK Reference:
    attack.mitre.org/techniques/T1

    #DFIR
    #Forensics
    #Infosec
    #Persistence
    #Windows
    #T1546
    #T1546.013
    #ThreatHunting

    #velociraptor #artifactsofautumn #dfir #forensics #infosec #persistence
  15. Kevin's Blog @[email protected] ·

    TD4 4-bit DIY CPU – Part 7

    Once the idea was floated, in Part 6 of creating an Arduino “direct to ROM” assembler, I had to just do it, so this post is a little diversion from the hardware discussion into how that could work.

    • Part 1 – Introduction, Discussion and Analysis
    • Part 2 – Building and Hardware
    • Part 3 – Programming and Simple Programs
    • Part 4 – Some hardware enhancements
    • Part 5 – My own PCB version
    • Part 6 – Replacing the ROM with a microcontroller
    • Part 7 – Creating an Arduino “assembler” for the TD4

    Basic Concepts

    This relies on using an Arduino as the ROM as described in Part 6, but the Arduino now has the option to change the ROM contents independently of the TD4 itself.

    The Arduino sketch will do the following:

    • Run the TD4 ROM routine off a timer interrupt so that it is always running and responsive.
    • Take input over the Arduino serial port to allow basic control, e.g. list, clear, etc.
    • Allow the direct input of assembler instructions, such as MOVE A,B or OUT B and so on.
    • Provide a means of selecting which line of the program to change.

    The code will thus have a number of key sections:

    • The TD4 ROM routine.
    • Some kind of serial-port command-line interpreter.
    • Handler routines for all the commands.
    • An assembler.
    • A disassembler.

    The TD4 ROM routine has already been fully described in Part 6. The only difference is that the scanning routine will be driven from a 1mS timer using the TimerOne library.

    As I want to still support a built-in demo, I now have the concept of ROM being the demo code and RAM being the “live” code to pass onto the TD4. The Arduino will initialise the RAM on startup from the ROM.

    As far as the TD4 is concerned of course, this is all still ROM.

    Command Line Interpreter

    The standard Arduino Serial routines will be used to scan for input via the serial port. It will support a line-oriented input as follows:

    bool cmdRunner (void) {
      while (Serial.available()) {
        char c = Serial.read();
        if (c == '\n') {
          strcpy(cmdSaved, cmdInput);
          cmdIdx = 0;
          return true;
        }
        else if (cmdIdx < CMD_BUFFER-1) {
          cmdInput[cmdIdx++] = c;
          cmdInput[cmdIdx] = '\0';
        }
      }
      return false;
    }

    This will keep adding any received characters to the cmdInput buffer until a newline is received, at which point the command is saved in cmdSaved and the routine will return true indicating a full line is ready to be processed.

    Once a complete line is received, then a processing function will parse it.

    Key to the processing of commands is a command table that stores the text to match and the handler function to call on finding a valid command. There is an additional parameter that will be passed into the handler function to allow the same handler function to support several commands. This will be used in the assembler itself later.

    struct cmd_t {
      char    cmd[CMD_BUFFER+1];
      hdlr_t  pFn;
      uint8_t idx;
    };

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
    };

    The algorithm for parsing commands is as follows:

    cmdProcess:
      Look for a space or newline
      IF found a space THEN
        This is the start of the parameter

      Look for the command in the command table
      IF command found THEN
        Call the handler function with the parameters

    The implementation is a bit complex, as it uses string pointers and has to chop and parse strings as it goes. It is also detailing with the command table in the Arduino’s PROGMEM which is an additional complication too.

    In order to be able to use the same command line interpreter for the input of assembler instructions, I’ve had to simplify the syntax. There are no spaces in opcodes and there has to be a space between the opcode and immediate value if used.

    Here are some examples:

    IN A       -> INA
    MOVE A,B   -> MOVAB
    OUT im     -> OUT im
    JNC im     -> JNC im
    ADD A,im   -> ADDA im

    Handler Routines

    All handler routines have the following prototype:

    typedef void (*hdlr_t)(int idx, char *param);

    void hdlrHelp(int idx, char *pParam) {
      Serial.print("\nHelp\n----\n");
      Serial.println("H: Help");
    }

    The idx parameter is the number in the last field of the command table. pParam will be a pointer to the parameter string for the command (if used).

    As we’re dealing with strings all the time, there are a number of helper functions to do things like convert strings to numbers as well as others to print numbers in various formats.

    Number formats are assumed to be as follows:

    0..9   - decimal digits
    0x0..F - hex digits
    b0..1  - binary digits

    The code provides the following:

    • str2num – the basic string parsing routine to recognise all three number formats as strings.
    • printbin – print a number in b0..1 format.
    • printhex – print a number in 0x0..F format, allowing for a possible leading zero if required.
    • printins – print an instruction in textual format.
    • printop – print an instruction in binary and hex opcode format.
    • printline – print a line number in a consistent binary and hex format.

    The code supports the following commands, so each has its own handler function:

    • H – help – show the list of commands.
    • L – list – show the disassembly of the whole working memory (RAM).
    • G – goto – set the working line number.
    • C – clear – reset all working memory (RAM) to zeros.
    • R – restore – restore the working memory (RAM) to the pre-build demo code (ROM).
    • O – opcodes – list the supported opcodes.

    Assembler

    As already mentioned, I’m using the same command line interpreter code to create the assembler. To do this, each opcode has an entry in the command table:

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      // Assembly commands - must be first
      {"ADDA", hdlrAsm, 0},
      {"MOVAB", hdlrAsm, 1},
      {"INA", hdlrAsm, 2},
      {"MOVA", hdlrAsm, 3},
      {"MOVBA", hdlrAsm, 4},
      {"ADDB", hdlrAsm, 5},
      {"INB", hdlrAsm, 6},
      {"MOVB", hdlrAsm, 7},
      {"OUTB", hdlrAsm, 8},
      {"OUT2B", hdlrAsm, 9},
      {"OUT", hdlrAsm, 10},
      {"OUT2", hdlrAsm, 11},
      {"JNCB", hdlrAsm, 12},
      {"JMPB", hdlrAsm, 13},
      {"JNC", hdlrAsm, 14},
      {"JMP", hdlrAsm, 15},

      // Other commands
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
      {"C", hdlrClear, 0},
      {"R", hdlrRestore, 0},
      {"O", hdlrOpcodes, 0},
    };

    The order corresponds to the opcode command value, as does the parameter. As these are at the start of the table, I can assume that the position in the table is the same as the command value. This does mean that I also need to account for the duplicated instructions even if I don’t need to use them.

    I’m making the following design decisions:

    • There is the concept of a “current line” which can be set with the G (goto) command.
    • Entering a valid opcode automatically moves the current line on by 1.
    • No line information is entered as part of the opcode.

    The main logic of the assembler handler is as follows:

    Assembler:
      Command value is the provided index parameter
      Determine the im value from the provided string parameter
      RAM[line] = cmd << 4 + im
      Increment current line

    Disassembler

    Disassembly is really largely a look-up table matching opcode command values to text. This is all hidden away behind the two print routines printins() and printop().

    void printins (uint8_t ins) {
      uint8_t cmd = ins >> 4;
      uint8_t im = ins & 0x0F;

      Serial.print(FSH(cmdTable[cmd].cmd));
      if (HASIM(cmd)) {
        Serial.print(" b");
        printbin(im,4);
      } else {
        Serial.print("      ");
      }
    }

    void printop (uint8_t op) {
      uint8_t cmd = op >> 4;
      uint8_t im = op & 0x0F;

      Serial.print("b");
      printbin(cmd,4);
      Serial.print(" ");
      printbin(im,4);
      Serial.print("\t0x");
      printhex(op,2);
    }

    The main complexity is pulling the strings out of the command table. I’ve had to include a macro to provide access to the strings from the Arduino’s PROGMEM:

    #define FSH(x) ((const __FlashStringHelper *)x)

    This feels like a bit of a hack, but apparently this is how it should be done for the kind of thing I need to do!

    There is another macro here that needs explaining:

    #define HASIM(op) (op==0||op==3||op==5||op==7||op>9)

    This is a set of conditions that if true means that the command supports an immediate value. This is used in a few places to know how to parse the commands.

    Whilst in principle all commands could use the immediate value, the “official” statement of how they work assumes im=0 in many cases. So, for example, OUT B does not require an immediate value, but if one is provided then OUT B becomes OUT B+im.

    I’m not really supporting that with this code at the moment.

    Putting it all together

    Here is a serial output log of a session using the assembler.

    > H
    Help
    ----
    H: Help
    L: List
    G: Goto
    C: Clear
    R: Restore
    O: Opcodes
    OpCode
    OpCode im

    Current line: b0000 [0]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: ADDA b0000b0000 00000x00
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b0010 [2]

    > G 13
    Goto line 13
    Current line: b1101 [D]

    > OUTB
    Assemble:
    b1101 [D] OUTB      b1000 00000x80
    Current line: b1110 [E]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: OUTB      b1000 00000x80
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b1110 [E]

    > O
    Supported OpCodes:
      b0000 dataADDA im
      b0001 0000MOVAB
      b0010 0000INA
      b0011 dataMOVA im
      b0100 0000MOVBA
      b0101 dataADDB im
      b0110 0000INB
      b0111 dataMOVB im
      b1000 0000OUTB
      b1001 0000OUT2B
      b1010 dataOUT im
      b1011 dataOUT2 im
      b1100 dataJNCB im
      b1101 dataJMPB im
      b1110 dataJNC im
      b1111 dataJMP im

    > C
    Clearing RAM ... Done

    Find the code on GitHub here.

    Conclusion

    The basics for this actually came together fairly quickly, but I must admit to spending a fair bit of time fiddling about with output formats and refactoring various bits of code to try to give some consistency in terms of when newlines are applied, what is shown in binary, what in hex, and so on.

    I can’t guarantee everything has been caught, but I’ve typed in all the code (using the newer, limited syntax) from Part 3 and they all seem to work.

    It would be nice to be able to automatically reset the TD4 from the Arduino, but for now, pressing the button when required is fine.

    For the most part, unless there is a loop to get caught in, the code will cycle back to the start anyway.

    In terms of possible updates and enhancements, there are a few on my mind:

    • It would be nice to support the undocumented use of immediate values somehow.
    • It might be nice to have a way to save/load the code. It only needs to be a string of 16 2-byte hex codes.
    • It might be nice to have several demo programs to choose from.

    If I expand the instruction set and architecture, then I’ll have to think again about chunks of this code, but for now, it seems to work pretty well.

    Kevin

    #4bit #arduinoUno #define #td4

    #4bit #arduinouno #define #td4
  16. Kevin's Blog @[email protected] ·

    TD4 4-bit DIY CPU – Part 7

    Once the idea was floated, in Part 6 of creating an Arduino “direct to ROM” assembler, I had to just do it, so this post is a little diversion from the hardware discussion into how that could work.

    • Part 1 – Introduction, Discussion and Analysis
    • Part 2 – Building and Hardware
    • Part 3 – Programming and Simple Programs
    • Part 4 – Some hardware enhancements
    • Part 5 – My own PCB version
    • Part 6 – Replacing the ROM with a microcontroller
    • Part 7 – Creating an Arduino “assembler” for the TD4

    Basic Concepts

    This relies on using an Arduino as the ROM as described in Part 6, but the Arduino now has the option to change the ROM contents independently of the TD4 itself.

    The Arduino sketch will do the following:

    • Run the TD4 ROM routine off a timer interrupt so that it is always running and responsive.
    • Take input over the Arduino serial port to allow basic control, e.g. list, clear, etc.
    • Allow the direct input of assembler instructions, such as MOVE A,B or OUT B and so on.
    • Provide a means of selecting which line of the program to change.

    The code will thus have a number of key sections:

    • The TD4 ROM routine.
    • Some kind of serial-port command-line interpreter.
    • Handler routines for all the commands.
    • An assembler.
    • A disassembler.

    The TD4 ROM routine has already been fully described in Part 6. The only difference is that the scanning routine will be driven from a 1mS timer using the TimerOne library.

    As I want to still support a built-in demo, I now have the concept of ROM being the demo code and RAM being the “live” code to pass onto the TD4. The Arduino will initialise the RAM on startup from the ROM.

    As far as the TD4 is concerned of course, this is all still ROM.

    Command Line Interpreter

    The standard Arduino Serial routines will be used to scan for input via the serial port. It will support a line-oriented input as follows:

    bool cmdRunner (void) {
      while (Serial.available()) {
        char c = Serial.read();
        if (c == '\n') {
          strcpy(cmdSaved, cmdInput);
          cmdIdx = 0;
          return true;
        }
        else if (cmdIdx < CMD_BUFFER-1) {
          cmdInput[cmdIdx++] = c;
          cmdInput[cmdIdx] = '\0';
        }
      }
      return false;
    }

    This will keep adding any received characters to the cmdInput buffer until a newline is received, at which point the command is saved in cmdSaved and the routine will return true indicating a full line is ready to be processed.

    Once a complete line is received, then a processing function will parse it.

    Key to the processing of commands is a command table that stores the text to match and the handler function to call on finding a valid command. There is an additional parameter that will be passed into the handler function to allow the same handler function to support several commands. This will be used in the assembler itself later.

    struct cmd_t {
      char    cmd[CMD_BUFFER+1];
      hdlr_t  pFn;
      uint8_t idx;
    };

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
    };

    The algorithm for parsing commands is as follows:

    cmdProcess:
      Look for a space or newline
      IF found a space THEN
        This is the start of the parameter

      Look for the command in the command table
      IF command found THEN
        Call the handler function with the parameters

    The implementation is a bit complex, as it uses string pointers and has to chop and parse strings as it goes. It is also detailing with the command table in the Arduino’s PROGMEM which is an additional complication too.

    In order to be able to use the same command line interpreter for the input of assembler instructions, I’ve had to simplify the syntax. There are no spaces in opcodes and there has to be a space between the opcode and immediate value if used.

    Here are some examples:

    IN A       -> INA
    MOVE A,B   -> MOVAB
    OUT im     -> OUT im
    JNC im     -> JNC im
    ADD A,im   -> ADDA im

    Handler Routines

    All handler routines have the following prototype:

    typedef void (*hdlr_t)(int idx, char *param);

    void hdlrHelp(int idx, char *pParam) {
      Serial.print("\nHelp\n----\n");
      Serial.println("H: Help");
    }

    The idx parameter is the number in the last field of the command table. pParam will be a pointer to the parameter string for the command (if used).

    As we’re dealing with strings all the time, there are a number of helper functions to do things like convert strings to numbers as well as others to print numbers in various formats.

    Number formats are assumed to be as follows:

    0..9   - decimal digits
    0x0..F - hex digits
    b0..1  - binary digits

    The code provides the following:

    • str2num – the basic string parsing routine to recognise all three number formats as strings.
    • printbin – print a number in b0..1 format.
    • printhex – print a number in 0x0..F format, allowing for a possible leading zero if required.
    • printins – print an instruction in textual format.
    • printop – print an instruction in binary and hex opcode format.
    • printline – print a line number in a consistent binary and hex format.

    The code supports the following commands, so each has its own handler function:

    • H – help – show the list of commands.
    • L – list – show the disassembly of the whole working memory (RAM).
    • G – goto – set the working line number.
    • C – clear – reset all working memory (RAM) to zeros.
    • R – restore – restore the working memory (RAM) to the pre-build demo code (ROM).
    • O – opcodes – list the supported opcodes.

    Assembler

    As already mentioned, I’m using the same command line interpreter code to create the assembler. To do this, each opcode has an entry in the command table:

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      // Assembly commands - must be first
      {"ADDA", hdlrAsm, 0},
      {"MOVAB", hdlrAsm, 1},
      {"INA", hdlrAsm, 2},
      {"MOVA", hdlrAsm, 3},
      {"MOVBA", hdlrAsm, 4},
      {"ADDB", hdlrAsm, 5},
      {"INB", hdlrAsm, 6},
      {"MOVB", hdlrAsm, 7},
      {"OUTB", hdlrAsm, 8},
      {"OUT2B", hdlrAsm, 9},
      {"OUT", hdlrAsm, 10},
      {"OUT2", hdlrAsm, 11},
      {"JNCB", hdlrAsm, 12},
      {"JMPB", hdlrAsm, 13},
      {"JNC", hdlrAsm, 14},
      {"JMP", hdlrAsm, 15},

      // Other commands
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
      {"C", hdlrClear, 0},
      {"R", hdlrRestore, 0},
      {"O", hdlrOpcodes, 0},
    };

    The order corresponds to the opcode command value, as does the parameter. As these are at the start of the table, I can assume that the position in the table is the same as the command value. This does mean that I also need to account for the duplicated instructions even if I don’t need to use them.

    I’m making the following design decisions:

    • There is the concept of a “current line” which can be set with the G (goto) command.
    • Entering a valid opcode automatically moves the current line on by 1.
    • No line information is entered as part of the opcode.

    The main logic of the assembler handler is as follows:

    Assembler:
      Command value is the provided index parameter
      Determine the im value from the provided string parameter
      RAM[line] = cmd << 4 + im
      Increment current line

    Disassembler

    Disassembly is really largely a look-up table matching opcode command values to text. This is all hidden away behind the two print routines printins() and printop().

    void printins (uint8_t ins) {
      uint8_t cmd = ins >> 4;
      uint8_t im = ins & 0x0F;

      Serial.print(FSH(cmdTable[cmd].cmd));
      if (HASIM(cmd)) {
        Serial.print(" b");
        printbin(im,4);
      } else {
        Serial.print("      ");
      }
    }

    void printop (uint8_t op) {
      uint8_t cmd = op >> 4;
      uint8_t im = op & 0x0F;

      Serial.print("b");
      printbin(cmd,4);
      Serial.print(" ");
      printbin(im,4);
      Serial.print("\t0x");
      printhex(op,2);
    }

    The main complexity is pulling the strings out of the command table. I’ve had to include a macro to provide access to the strings from the Arduino’s PROGMEM:

    #define FSH(x) ((const __FlashStringHelper *)x)

    This feels like a bit of a hack, but apparently this is how it should be done for the kind of thing I need to do!

    There is another macro here that needs explaining:

    #define HASIM(op) (op==0||op==3||op==5||op==7||op>9)

    This is a set of conditions that if true means that the command supports an immediate value. This is used in a few places to know how to parse the commands.

    Whilst in principle all commands could use the immediate value, the “official” statement of how they work assumes im=0 in many cases. So, for example, OUT B does not require an immediate value, but if one is provided then OUT B becomes OUT B+im.

    I’m not really supporting that with this code at the moment.

    Putting it all together

    Here is a serial output log of a session using the assembler.

    > H
    Help
    ----
    H: Help
    L: List
    G: Goto
    C: Clear
    R: Restore
    O: Opcodes
    OpCode
    OpCode im

    Current line: b0000 [0]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: ADDA b0000b0000 00000x00
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b0010 [2]

    > G 13
    Goto line 13
    Current line: b1101 [D]

    > OUTB
    Assemble:
    b1101 [D] OUTB      b1000 00000x80
    Current line: b1110 [E]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: OUTB      b1000 00000x80
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b1110 [E]

    > O
    Supported OpCodes:
      b0000 dataADDA im
      b0001 0000MOVAB
      b0010 0000INA
      b0011 dataMOVA im
      b0100 0000MOVBA
      b0101 dataADDB im
      b0110 0000INB
      b0111 dataMOVB im
      b1000 0000OUTB
      b1001 0000OUT2B
      b1010 dataOUT im
      b1011 dataOUT2 im
      b1100 dataJNCB im
      b1101 dataJMPB im
      b1110 dataJNC im
      b1111 dataJMP im

    > C
    Clearing RAM ... Done

    Find the code on GitHub here.

    Conclusion

    The basics for this actually came together fairly quickly, but I must admit to spending a fair bit of time fiddling about with output formats and refactoring various bits of code to try to give some consistency in terms of when newlines are applied, what is shown in binary, what in hex, and so on.

    I can’t guarantee everything has been caught, but I’ve typed in all the code (using the newer, limited syntax) from Part 3 and they all seem to work.

    It would be nice to be able to automatically reset the TD4 from the Arduino, but for now, pressing the button when required is fine.

    For the most part, unless there is a loop to get caught in, the code will cycle back to the start anyway.

    In terms of possible updates and enhancements, there are a few on my mind:

    • It would be nice to support the undocumented use of immediate values somehow.
    • It might be nice to have a way to save/load the code. It only needs to be a string of 16 2-byte hex codes.
    • It might be nice to have several demo programs to choose from.

    If I expand the instruction set and architecture, then I’ll have to think again about chunks of this code, but for now, it seems to work pretty well.

    Kevin

    #4bit #arduinoUno #define #td4

    #td4 #define #arduinouno #4bit
  17. Kevin's Blog @[email protected] ·

    TD4 4-bit DIY CPU – Part 7

    Once the idea was floated, in Part 6 of creating an Arduino “direct to ROM” assembler, I had to just do it, so this post is a little diversion from the hardware discussion into how that could work.

    • Part 1 – Introduction, Discussion and Analysis
    • Part 2 – Building and Hardware
    • Part 3 – Programming and Simple Programs
    • Part 4 – Some hardware enhancements
    • Part 5 – My own PCB version
    • Part 6 – Replacing the ROM with a microcontroller
    • Part 7 – Creating an Arduino “assembler” for the TD4
    • Part 8 – Extending the address space to 5-bits and an Arduino ROM PCB

    Basic Concepts

    This relies on using an Arduino as the ROM as described in Part 6, but the Arduino now has the option to change the ROM contents independently of the TD4 itself.

    The Arduino sketch will do the following:

    • Run the TD4 ROM routine off a timer interrupt so that it is always running and responsive.
    • Take input over the Arduino serial port to allow basic control, e.g. list, clear, etc.
    • Allow the direct input of assembler instructions, such as MOVE A,B or OUT B and so on.
    • Provide a means of selecting which line of the program to change.

    The code will thus have a number of key sections:

    • The TD4 ROM routine.
    • Some kind of serial-port command-line interpreter.
    • Handler routines for all the commands.
    • An assembler.
    • A disassembler.

    The TD4 ROM routine has already been fully described in Part 6. The only difference is that the scanning routine will be driven from a 1mS timer using the TimerOne library.

    As I want to still support a built-in demo, I now have the concept of ROM being the demo code and RAM being the “live” code to pass onto the TD4. The Arduino will initialise the RAM on startup from the ROM.

    As far as the TD4 is concerned of course, this is all still ROM.

    Command Line Interpreter

    The standard Arduino Serial routines will be used to scan for input via the serial port. It will support a line-oriented input as follows:

    bool cmdRunner (void) {
      while (Serial.available()) {
        char c = Serial.read();
        if (c == '\n') {
          strcpy(cmdSaved, cmdInput);
          cmdIdx = 0;
          return true;
        }
        else if (cmdIdx < CMD_BUFFER-1) {
          cmdInput[cmdIdx++] = c;
          cmdInput[cmdIdx] = '\0';
        }
      }
      return false;
    }

    This will keep adding any received characters to the cmdInput buffer until a newline is received, at which point the command is saved in cmdSaved and the routine will return true indicating a full line is ready to be processed.

    Once a complete line is received, then a processing function will parse it.

    Key to the processing of commands is a command table that stores the text to match and the handler function to call on finding a valid command. There is an additional parameter that will be passed into the handler function to allow the same handler function to support several commands. This will be used in the assembler itself later.

    struct cmd_t {
      char    cmd[CMD_BUFFER+1];
      hdlr_t  pFn;
      uint8_t idx;
    };

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
    };

    The algorithm for parsing commands is as follows:

    cmdProcess:
      Look for a space or newline
      IF found a space THEN
        This is the start of the parameter

      Look for the command in the command table
      IF command found THEN
        Call the handler function with the parameters

    The implementation is a bit complex, as it uses string pointers and has to chop and parse strings as it goes. It is also detailing with the command table in the Arduino’s PROGMEM which is an additional complication too.

    In order to be able to use the same command line interpreter for the input of assembler instructions, I’ve had to simplify the syntax. There are no spaces in opcodes and there has to be a space between the opcode and immediate value if used.

    Here are some examples:

    IN A       -> INA
    MOVE A,B   -> MOVAB
    OUT im     -> OUT im
    JNC im     -> JNC im
    ADD A,im   -> ADDA im

    Handler Routines

    All handler routines have the following prototype:

    typedef void (*hdlr_t)(int idx, char *param);

    void hdlrHelp(int idx, char *pParam) {
      Serial.print("\nHelp\n----\n");
      Serial.println("H: Help");
    }

    The idx parameter is the number in the last field of the command table. pParam will be a pointer to the parameter string for the command (if used).

    As we’re dealing with strings all the time, there are a number of helper functions to do things like convert strings to numbers as well as others to print numbers in various formats.

    Number formats are assumed to be as follows:

    0..9   - decimal digits
    0x0..F - hex digits
    b0..1  - binary digits

    The code provides the following:

    • str2num – the basic string parsing routine to recognise all three number formats as strings.
    • printbin – print a number in b0..1 format.
    • printhex – print a number in 0x0..F format, allowing for a possible leading zero if required.
    • printins – print an instruction in textual format.
    • printop – print an instruction in binary and hex opcode format.
    • printline – print a line number in a consistent binary and hex format.

    The code supports the following commands, so each has its own handler function:

    • H – help – show the list of commands.
    • L – list – show the disassembly of the whole working memory (RAM).
    • G – goto – set the working line number.
    • C – clear – reset all working memory (RAM) to zeros.
    • R – restore – restore the working memory (RAM) to the pre-build demo code (ROM).
    • O – opcodes – list the supported opcodes.

    Assembler

    As already mentioned, I’m using the same command line interpreter code to create the assembler. To do this, each opcode has an entry in the command table:

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      // Assembly commands - must be first
      {"ADDA", hdlrAsm, 0},
      {"MOVAB", hdlrAsm, 1},
      {"INA", hdlrAsm, 2},
      {"MOVA", hdlrAsm, 3},
      {"MOVBA", hdlrAsm, 4},
      {"ADDB", hdlrAsm, 5},
      {"INB", hdlrAsm, 6},
      {"MOVB", hdlrAsm, 7},
      {"OUTB", hdlrAsm, 8},
      {"OUT2B", hdlrAsm, 9},
      {"OUT", hdlrAsm, 10},
      {"OUT2", hdlrAsm, 11},
      {"JNCB", hdlrAsm, 12},
      {"JMPB", hdlrAsm, 13},
      {"JNC", hdlrAsm, 14},
      {"JMP", hdlrAsm, 15},

      // Other commands
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
      {"C", hdlrClear, 0},
      {"R", hdlrRestore, 0},
      {"O", hdlrOpcodes, 0},
    };

    The order corresponds to the opcode command value, as does the parameter. As these are at the start of the table, I can assume that the position in the table is the same as the command value. This does mean that I also need to account for the duplicated instructions even if I don’t need to use them.

    I’m making the following design decisions:

    • There is the concept of a “current line” which can be set with the G (goto) command.
    • Entering a valid opcode automatically moves the current line on by 1.
    • No line information is entered as part of the opcode.

    The main logic of the assembler handler is as follows:

    Assembler:
      Command value is the provided index parameter
      Determine the im value from the provided string parameter
      RAM[line] = cmd << 4 + im
      Increment current line

    Disassembler

    Disassembly is really largely a look-up table matching opcode command values to text. This is all hidden away behind the two print routines printins() and printop().

    void printins (uint8_t ins) {
      uint8_t cmd = ins >> 4;
      uint8_t im = ins & 0x0F;

      Serial.print(FSH(cmdTable[cmd].cmd));
      if (HASIM(cmd)) {
        Serial.print(" b");
        printbin(im,4);
      } else {
        Serial.print("      ");
      }
    }

    void printop (uint8_t op) {
      uint8_t cmd = op >> 4;
      uint8_t im = op & 0x0F;

      Serial.print("b");
      printbin(cmd,4);
      Serial.print(" ");
      printbin(im,4);
      Serial.print("\t0x");
      printhex(op,2);
    }

    The main complexity is pulling the strings out of the command table. I’ve had to include a macro to provide access to the strings from the Arduino’s PROGMEM:

    #define FSH(x) ((const __FlashStringHelper *)x)

    This feels like a bit of a hack, but apparently this is how it should be done for the kind of thing I need to do!

    There is another macro here that needs explaining:

    #define HASIM(op) (op==0||op==3||op==5||op==7||op>9)

    This is a set of conditions that if true means that the command supports an immediate value. This is used in a few places to know how to parse the commands.

    Whilst in principle all commands could use the immediate value, the “official” statement of how they work assumes im=0 in many cases. So, for example, OUT B does not require an immediate value, but if one is provided then OUT B becomes OUT B+im.

    I’m not really supporting that with this code at the moment.

    Putting it all together

    Here is a serial output log of a session using the assembler.

    > H
    Help
    ----
    H: Help
    L: List
    G: Goto
    C: Clear
    R: Restore
    O: Opcodes
    OpCode
    OpCode im

    Current line: b0000 [0]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: ADDA b0000b0000 00000x00
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b0010 [2]

    > G 13
    Goto line 13
    Current line: b1101 [D]

    > OUTB
    Assemble:
    b1101 [D] OUTB      b1000 00000x80
    Current line: b1110 [E]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: OUTB      b1000 00000x80
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b1110 [E]

    > O
    Supported OpCodes:
      b0000 dataADDA im
      b0001 0000MOVAB
      b0010 0000INA
      b0011 dataMOVA im
      b0100 0000MOVBA
      b0101 dataADDB im
      b0110 0000INB
      b0111 dataMOVB im
      b1000 0000OUTB
      b1001 0000OUT2B
      b1010 dataOUT im
      b1011 dataOUT2 im
      b1100 dataJNCB im
      b1101 dataJMPB im
      b1110 dataJNC im
      b1111 dataJMP im

    > C
    Clearing RAM ... Done

    Find the code on GitHub here.

    Conclusion

    The basics for this actually came together fairly quickly, but I must admit to spending a fair bit of time fiddling about with output formats and refactoring various bits of code to try to give some consistency in terms of when newlines are applied, what is shown in binary, what in hex, and so on.

    I can’t guarantee everything has been caught, but I’ve typed in all the code (using the newer, limited syntax) from Part 3 and they all seem to work.

    It would be nice to be able to automatically reset the TD4 from the Arduino, but for now, pressing the button when required is fine.

    For the most part, unless there is a loop to get caught in, the code will cycle back to the start anyway.

    In terms of possible updates and enhancements, there are a few on my mind:

    • It would be nice to support the undocumented use of immediate values somehow.
    • It might be nice to have a way to save/load the code. It only needs to be a string of 16 2-byte hex codes.
    • It might be nice to have several demo programs to choose from.

    If I expand the instruction set and architecture, then I’ll have to think again about chunks of this code, but for now, it seems to work pretty well.

    Kevin

    #4bit #arduinoUno #define #td4

    #4bit #arduinouno #define #td4
  18. Kevin's Blog @[email protected] ·

    TD4 4-bit DIY CPU – Part 7

    Once the idea was floated, in Part 6 of creating an Arduino “direct to ROM” assembler, I had to just do it, so this post is a little diversion from the hardware discussion into how that could work.

    • Part 1 – Introduction, Discussion and Analysis
    • Part 2 – Building and Hardware
    • Part 3 – Programming and Simple Programs
    • Part 4 – Some hardware enhancements
    • Part 5 – My own PCB version
    • Part 6 – Replacing the ROM with a microcontroller
    • Part 7 – Creating an Arduino “assembler” for the TD4
    • Part 8 – Extending the address space to 5-bits and an Arduino ROM PCB

    Basic Concepts

    This relies on using an Arduino as the ROM as described in Part 6, but the Arduino now has the option to change the ROM contents independently of the TD4 itself.

    The Arduino sketch will do the following:

    • Run the TD4 ROM routine off a timer interrupt so that it is always running and responsive.
    • Take input over the Arduino serial port to allow basic control, e.g. list, clear, etc.
    • Allow the direct input of assembler instructions, such as MOVE A,B or OUT B and so on.
    • Provide a means of selecting which line of the program to change.

    The code will thus have a number of key sections:

    • The TD4 ROM routine.
    • Some kind of serial-port command-line interpreter.
    • Handler routines for all the commands.
    • An assembler.
    • A disassembler.

    The TD4 ROM routine has already been fully described in Part 6. The only difference is that the scanning routine will be driven from a 1mS timer using the TimerOne library.

    As I want to still support a built-in demo, I now have the concept of ROM being the demo code and RAM being the “live” code to pass onto the TD4. The Arduino will initialise the RAM on startup from the ROM.

    As far as the TD4 is concerned of course, this is all still ROM.

    Command Line Interpreter

    The standard Arduino Serial routines will be used to scan for input via the serial port. It will support a line-oriented input as follows:

    bool cmdRunner (void) {
      while (Serial.available()) {
        char c = Serial.read();
        if (c == '\n') {
          strcpy(cmdSaved, cmdInput);
          cmdIdx = 0;
          return true;
        }
        else if (cmdIdx < CMD_BUFFER-1) {
          cmdInput[cmdIdx++] = c;
          cmdInput[cmdIdx] = '\0';
        }
      }
      return false;
    }

    This will keep adding any received characters to the cmdInput buffer until a newline is received, at which point the command is saved in cmdSaved and the routine will return true indicating a full line is ready to be processed.

    Once a complete line is received, then a processing function will parse it.

    Key to the processing of commands is a command table that stores the text to match and the handler function to call on finding a valid command. There is an additional parameter that will be passed into the handler function to allow the same handler function to support several commands. This will be used in the assembler itself later.

    struct cmd_t {
      char    cmd[CMD_BUFFER+1];
      hdlr_t  pFn;
      uint8_t idx;
    };

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
    };

    The algorithm for parsing commands is as follows:

    cmdProcess:
      Look for a space or newline
      IF found a space THEN
        This is the start of the parameter

      Look for the command in the command table
      IF command found THEN
        Call the handler function with the parameters

    The implementation is a bit complex, as it uses string pointers and has to chop and parse strings as it goes. It is also detailing with the command table in the Arduino’s PROGMEM which is an additional complication too.

    In order to be able to use the same command line interpreter for the input of assembler instructions, I’ve had to simplify the syntax. There are no spaces in opcodes and there has to be a space between the opcode and immediate value if used.

    Here are some examples:

    IN A       -> INA
    MOVE A,B   -> MOVAB
    OUT im     -> OUT im
    JNC im     -> JNC im
    ADD A,im   -> ADDA im

    Handler Routines

    All handler routines have the following prototype:

    typedef void (*hdlr_t)(int idx, char *param);

    void hdlrHelp(int idx, char *pParam) {
      Serial.print("\nHelp\n----\n");
      Serial.println("H: Help");
    }

    The idx parameter is the number in the last field of the command table. pParam will be a pointer to the parameter string for the command (if used).

    As we’re dealing with strings all the time, there are a number of helper functions to do things like convert strings to numbers as well as others to print numbers in various formats.

    Number formats are assumed to be as follows:

    0..9   - decimal digits
    0x0..F - hex digits
    b0..1  - binary digits

    The code provides the following:

    • str2num – the basic string parsing routine to recognise all three number formats as strings.
    • printbin – print a number in b0..1 format.
    • printhex – print a number in 0x0..F format, allowing for a possible leading zero if required.
    • printins – print an instruction in textual format.
    • printop – print an instruction in binary and hex opcode format.
    • printline – print a line number in a consistent binary and hex format.

    The code supports the following commands, so each has its own handler function:

    • H – help – show the list of commands.
    • L – list – show the disassembly of the whole working memory (RAM).
    • G – goto – set the working line number.
    • C – clear – reset all working memory (RAM) to zeros.
    • R – restore – restore the working memory (RAM) to the pre-build demo code (ROM).
    • O – opcodes – list the supported opcodes.

    Assembler

    As already mentioned, I’m using the same command line interpreter code to create the assembler. To do this, each opcode has an entry in the command table:

    const cmd_t PROGMEM cmdTable[NUM_CMDS] = {
      // Assembly commands - must be first
      {"ADDA", hdlrAsm, 0},
      {"MOVAB", hdlrAsm, 1},
      {"INA", hdlrAsm, 2},
      {"MOVA", hdlrAsm, 3},
      {"MOVBA", hdlrAsm, 4},
      {"ADDB", hdlrAsm, 5},
      {"INB", hdlrAsm, 6},
      {"MOVB", hdlrAsm, 7},
      {"OUTB", hdlrAsm, 8},
      {"OUT2B", hdlrAsm, 9},
      {"OUT", hdlrAsm, 10},
      {"OUT2", hdlrAsm, 11},
      {"JNCB", hdlrAsm, 12},
      {"JMPB", hdlrAsm, 13},
      {"JNC", hdlrAsm, 14},
      {"JMP", hdlrAsm, 15},

      // Other commands
      {"H", hdlrHelp, 0},
      {"L", hdlrList, 0},
      {"G", hdlrGoto, 0},
      {"C", hdlrClear, 0},
      {"R", hdlrRestore, 0},
      {"O", hdlrOpcodes, 0},
    };

    The order corresponds to the opcode command value, as does the parameter. As these are at the start of the table, I can assume that the position in the table is the same as the command value. This does mean that I also need to account for the duplicated instructions even if I don’t need to use them.

    I’m making the following design decisions:

    • There is the concept of a “current line” which can be set with the G (goto) command.
    • Entering a valid opcode automatically moves the current line on by 1.
    • No line information is entered as part of the opcode.

    The main logic of the assembler handler is as follows:

    Assembler:
      Command value is the provided index parameter
      Determine the im value from the provided string parameter
      RAM[line] = cmd << 4 + im
      Increment current line

    Disassembler

    Disassembly is really largely a look-up table matching opcode command values to text. This is all hidden away behind the two print routines printins() and printop().

    void printins (uint8_t ins) {
      uint8_t cmd = ins >> 4;
      uint8_t im = ins & 0x0F;

      Serial.print(FSH(cmdTable[cmd].cmd));
      if (HASIM(cmd)) {
        Serial.print(" b");
        printbin(im,4);
      } else {
        Serial.print("      ");
      }
    }

    void printop (uint8_t op) {
      uint8_t cmd = op >> 4;
      uint8_t im = op & 0x0F;

      Serial.print("b");
      printbin(cmd,4);
      Serial.print(" ");
      printbin(im,4);
      Serial.print("\t0x");
      printhex(op,2);
    }

    The main complexity is pulling the strings out of the command table. I’ve had to include a macro to provide access to the strings from the Arduino’s PROGMEM:

    #define FSH(x) ((const __FlashStringHelper *)x)

    This feels like a bit of a hack, but apparently this is how it should be done for the kind of thing I need to do!

    There is another macro here that needs explaining:

    #define HASIM(op) (op==0||op==3||op==5||op==7||op>9)

    This is a set of conditions that if true means that the command supports an immediate value. This is used in a few places to know how to parse the commands.

    Whilst in principle all commands could use the immediate value, the “official” statement of how they work assumes im=0 in many cases. So, for example, OUT B does not require an immediate value, but if one is provided then OUT B becomes OUT B+im.

    I’m not really supporting that with this code at the moment.

    Putting it all together

    Here is a serial output log of a session using the assembler.

    > H
    Help
    ----
    H: Help
    L: List
    G: Goto
    C: Clear
    R: Restore
    O: Opcodes
    OpCode
    OpCode im

    Current line: b0000 [0]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: ADDA b0000b0000 00000x00
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b0010 [2]

    > G 13
    Goto line 13
    Current line: b1101 [D]

    > OUTB
    Assemble:
    b1101 [D] OUTB      b1000 00000x80
    Current line: b1110 [E]

    > L
    RAM Disassembly

    b0000 [0]: JNC b1000b1110 10000xE8
    b0001 [1]: JMP b0011b1111 00110xF3
    b0010 [2]: OUT b0010b1010 00100xA2
    b0011 [3]: ADDB b0001b0101 00010x51
    b0100 [4]: OUT b0100b1010 01000xA4
    b0101 [5]: ADDA b0001b0000 00010x01
    b0110 [6]: OUT b1000b1010 10000xA8
    b0111 [7]: ADDB b0001b0101 00010x51
    b1000 [8]: OUT b0100b1010 01000xA4
    b1001 [9]: ADDA b0001b0000 00010x01
    b1010 [A]: OUT b0010b1010 00100xA2
    b1011 [B]: ADDB b0001b0101 00010x51
    b1100 [C]: JMP b0000b1111 00000xF0
    b1101 [D]: OUTB      b1000 00000x80
    b1110 [E]: ADDA b0000b0000 00000x00
    b1111 [F]: ADDA b0000b0000 00000x00
    Current line: b1110 [E]

    > O
    Supported OpCodes:
      b0000 dataADDA im
      b0001 0000MOVAB
      b0010 0000INA
      b0011 dataMOVA im
      b0100 0000MOVBA
      b0101 dataADDB im
      b0110 0000INB
      b0111 dataMOVB im
      b1000 0000OUTB
      b1001 0000OUT2B
      b1010 dataOUT im
      b1011 dataOUT2 im
      b1100 dataJNCB im
      b1101 dataJMPB im
      b1110 dataJNC im
      b1111 dataJMP im

    > C
    Clearing RAM ... Done

    Find the code on GitHub here.

    Conclusion

    The basics for this actually came together fairly quickly, but I must admit to spending a fair bit of time fiddling about with output formats and refactoring various bits of code to try to give some consistency in terms of when newlines are applied, what is shown in binary, what in hex, and so on.

    I can’t guarantee everything has been caught, but I’ve typed in all the code (using the newer, limited syntax) from Part 3 and they all seem to work.

    It would be nice to be able to automatically reset the TD4 from the Arduino, but for now, pressing the button when required is fine.

    For the most part, unless there is a loop to get caught in, the code will cycle back to the start anyway.

    In terms of possible updates and enhancements, there are a few on my mind:

    • It would be nice to support the undocumented use of immediate values somehow.
    • It might be nice to have a way to save/load the code. It only needs to be a string of 16 2-byte hex codes.
    • It might be nice to have several demo programs to choose from.

    If I expand the instruction set and architecture, then I’ll have to think again about chunks of this code, but for now, it seems to work pretty well.

    Kevin

    #4bit #arduinoUno #define #td4

    #4bit #arduinouno #define #td4
  19. jejb @[email protected] ·

    Back in my blog post about Securing the Google SIP Stack, I did say I’d look at re-enabling SIP in Android-12, so with a view to doing that I tried building and booting LineageOS 19.1, but it crashed really early in the boot sequence (after the boot splash but before the boot animation started). It turns out that information on debugging the android early boot sequence is a bit scarce, so I thought I should write a post about how I did it just in case it helps someone else who’s struggling with a similar early boot problem.

    How I usually Build and Boot Android

    My builds are standard LineageOS with my patches to fix SIP and not much else. However, I do replace the debug keys with my signing keys and I also have an AVB key installed in the phone’s third party keyslot with which I sign the vbmeta for boot. This actually means that my phone is effectively locked but with a user supplied key (Yellow as google puts it).

    My phone is now a pixel 3 (I had to say goodbye to the old Nexus One thanks to the US 3G turn off) and I do have a slightly broken Pixel 3 I play with for experimental patches, which is where I was trying to install Android-12.

    Signing Seems to be the Problem

    Just to verify my phone could actually boot a stock LineageOS (it could) I had to unlock it and this lead to the discovery that once unlocked, it would also boot my custom rom as well, so whatever was failing in early boot seemed to be connected with the device being locked.

    I also discovered an interesting bug in the recovery rom fastboot: If you’re booting locked with your own keys, it will still let you perform all the usually forbidden fastboot commands (the one I was using was set_active). It turns out to be because of a bug in AOSP which treats yellow devices as unlocked in fastboot. Somewhat handy for debugging, but not so hot for security …

    And so to Debugging Early Boot

    The big problem with Android is there’s no way to get the console messages for early boot. Even if you enable adb early, it doesn’t get started until quite far in to the boot animation (which was way after the crash I was tripping over). However, android does have a pstore (previously ramoops) driver that can give you access to the previously crashed boot’s kernel messages (early init, fortunately, mostly logs to the kernel message log).

    Forcing init to crash on failure

    Ordinarily an init failure prints a message and reboots (to the bootloader), which doesn’t excite pstore into saving the kernel message log. fortunately there is a boot option (androidboot.init_fatal_panic) which can be set in the boot options (or kernel command line for a pixel-3 which can only boot the 4.9 kernel). If you build your own android, it’s fairly easy to add things to the android commandline (which is in boot.img) because all you need to do is extract BOOT/cmdline from the intermediate zip file you sign add any boot options you need and place it back in the zip file (before you sign it).

    Unfortunately, this expedient didn’t work (no console logs appear in pstore). I did check that init was correctly panic’ing on failure by inducing an init failure in recovery mode and observing the panic (recovery mode allows you to run adb). But this induced panic also didn’t show up in pstore, meaning there’s actually some problem with pstore and early panics.

    Security is the problem (as usual)

    The actual problem turned out to be security (as usual): The pixel-3 does encrypted boot panic logs. The way this seems to work (at least in my reading of the google additional pstore patches) is that the bootloader itself encrypts the pstore ram area with a key on the /data partition, which means it only becomes visible after the device is unlocked. Unfortunately, if you trigger a panic before the device is unlocked (by echoing ‘c’ to /proc/sysrq-trigger) the panic message is lost, so pstore itself is useless for debugging early boot. There seems to be some communication of the keys by the vendor proprietary ramoops binary making it very difficult to figure out how it’s being done.

    Why the early panic message is lost is a bit mysterious, but unfortunately pstore on the pixel-3 has several proprietary components around the encrypted message handling that make it hard to debug. I suspect if you don’t set up the pstore encryption keys, the bootloader erases the pstore ram area instead of encrypting it, but I can’t prove that.

    Although it might be possible to fix the pstore drivers to preserve the ramoops from before device unlock, the participation of the proprietary bootloader in preserving the memory doesn’t make that look like a promising avenue to explore.

    Anatomy of the Pixel-3 Boot Sequence

    The Pixel-3 device boots through recovery. What this means is that the initial ramdisk (from boot.img) init is what boots both the recovery and normal boot paths. The only difference is that for recovery (and fastboot), the device stays in the ramdisk and for normal boot it mounts the /system partition and pivots to it. What makes this happen or not is the boot flag androidboot.force_normal_boot=1 which is added by the bootloader. Pretty much all the binary content and init rc files in the ramdisk are for recovery and its allied menus.

    Since the boot paths are pretty radically different, because the normal boot first pivots to a first stage before going on to a second, but in the manner of containers, it might be possible to boot recovery first, start a dmesg logger and then re-exec init through the normal path

    Forcing Re-Exec

    The idea is to signal init to re-exec itself for the normal path. Of course, there have to be a few changes to do this: An item has to be added to the recovery menu to signal init and init itself has to be modified to do the re-exec on the signal (note you can’t just kick off an init with a new command line because init must be pid 1 for booting). Once this is done, there are problems with selinux (it won’t actually allow init to re-exec) and some mount moves. The selinux problem is fixable by switching it from enforcing to permissive (boot option androidboot.selinux=permissive) and the mount moves (which are forbidden if you’re running binaries from the mount points being moved) can instead become bind mounts. The whole patch becomes 31 insertions across 7 files in android_system_core.

    The signal I chose was SIGUSR1, which isn’t usually used by anything in the bootloader and the addition of a menu item to recovery to send this signal to init was also another trivial patch. So finally we have a system from which I can start adb to trace the kernel log (adb shell dmesg -w) and then signal to init to re-exec. Surprisingly this worked and produced as the last message fragment:

    [ 190.966881] init: [libfs_mgr]Created logical partition system_a on device /dev/block/dm-0[ 190.967697] init: [libfs_mgr]Created logical partition vendor_a on device /dev/block/dm-1[ 190.968367] init: [libfs_mgr]Created logical partition product_a on device /dev/block/dm-2[ 190.969024] init: [libfs_mgr]Created logical partition system_ext_a on device /dev/block/dm-3[ 190.969067] init: DSU not detected, proceeding with normal boot[ 190.982957] init: [libfs_avb]Invalid hash size:[ 190.982967] init: [libfs_avb]Failed to verify vbmeta digest[ 190.982972] init: [libfs_avb]vbmeta digest error isn't allowed[ 190.982980] init: Failed to open AvbHandle: No such file or directory[ 190.982987] init: Failed to setup verity for '/system': No such file or directory[ 190.982993] init: Failed to mount /system: No such file or directory[ 190.983030] init: Failed to mount required partitions early …[ 190.983483] init: InitFatalReboot: signal 6[ 190.984849] init: #00 pc 0000000000123b38 /system/bin/init[ 190.984857] init: #01 pc 00000000000bc9a8 /system/bin/init[ 190.984864] init: #02 pc 000000000001595c /system/lib64/libbase.so[ 190.984869] init: #03 pc 0000000000014f8c /system/lib64/libbase.so[ 190.984874] init: #04 pc 00000000000e6984 /system/bin/init[ 190.984878] init: #05 pc 00000000000aa144 /system/bin/init[ 190.984883] init: #06 pc 00000000000487dc /system/lib64/libc.so[ 190.984889] init: Reboot ending, jumping to kernel

    Which indicates exactly where the problem is.

    Fixing the problem

    Once the messages are identified, the problem turns out to be in system/core ec10d3cf6 “libfs_avb: verifying vbmeta digest early”, which is inherited from AOSP and which even says in in it’s commit message “the device will not boot if: 1. The image is signed with FLAGS_VERIFICATION_DISABLED is set 2. The device state is locked” which is basically my boot state, so thanks for that one google. Reverting this commit can be done cleanly and now the signed image boots without a problem.

    I note that I could also simply add hashtree verification to my boot, but LineageOS is based on the eng target, which has FLAGS_VERIFICATION_DISABLED built into the main build Makefile. It might be possible to change it, but not easily I’m guessing … although I might try fixing it this way at some point, since it would make my phones much more secure.

    Conclusion

    Debugging android early boot is still a terribly hard problem. Probably someone with more patience for disassembling proprietary binaries could take apart pixel-3 vendor ramoops and figure out if it’s possible to get a pstore oops log out of early boot (which would be the easiest way to debug problems). But failing that the simple hack to re-exec init worked enough to show me where the problem was (of course, if init had continued longer it would likely have run into other issues caused by the way I hacked it).

    https://blog.hansenpartnership.com/debugging-android-early-boot-failures/

    #00 #01 #02 #03 #04 #05 #06 #android #androidBoot #androidDebugging #androidVerifiedBoot #AVB #lineageos

    #android #androidboot #androiddebugging #androidverifiedboot #avb #lineageos
  20. jejb @[email protected] ·

    Back in my blog post about Securing the Google SIP Stack, I did say I’d look at re-enabling SIP in Android-12, so with a view to doing that I tried building and booting LineageOS 19.1, but it crashed really early in the boot sequence (after the boot splash but before the boot animation started). It turns out that information on debugging the android early boot sequence is a bit scarce, so I thought I should write a post about how I did it just in case it helps someone else who’s struggling with a similar early boot problem.

    How I usually Build and Boot Android

    My builds are standard LineageOS with my patches to fix SIP and not much else. However, I do replace the debug keys with my signing keys and I also have an AVB key installed in the phone’s third party keyslot with which I sign the vbmeta for boot. This actually means that my phone is effectively locked but with a user supplied key (Yellow as google puts it).

    My phone is now a pixel 3 (I had to say goodbye to the old Nexus One thanks to the US 3G turn off) and I do have a slightly broken Pixel 3 I play with for experimental patches, which is where I was trying to install Android-12.

    Signing Seems to be the Problem

    Just to verify my phone could actually boot a stock LineageOS (it could) I had to unlock it and this lead to the discovery that once unlocked, it would also boot my custom rom as well, so whatever was failing in early boot seemed to be connected with the device being locked.

    I also discovered an interesting bug in the recovery rom fastboot: If you’re booting locked with your own keys, it will still let you perform all the usually forbidden fastboot commands (the one I was using was set_active). It turns out to be because of a bug in AOSP which treats yellow devices as unlocked in fastboot. Somewhat handy for debugging, but not so hot for security …

    And so to Debugging Early Boot

    The big problem with Android is there’s no way to get the console messages for early boot. Even if you enable adb early, it doesn’t get started until quite far in to the boot animation (which was way after the crash I was tripping over). However, android does have a pstore (previously ramoops) driver that can give you access to the previously crashed boot’s kernel messages (early init, fortunately, mostly logs to the kernel message log).

    Forcing init to crash on failure

    Ordinarily an init failure prints a message and reboots (to the bootloader), which doesn’t excite pstore into saving the kernel message log. fortunately there is a boot option (androidboot.init_fatal_panic) which can be set in the boot options (or kernel command line for a pixel-3 which can only boot the 4.9 kernel). If you build your own android, it’s fairly easy to add things to the android commandline (which is in boot.img) because all you need to do is extract BOOT/cmdline from the intermediate zip file you sign add any boot options you need and place it back in the zip file (before you sign it).

    Unfortunately, this expedient didn’t work (no console logs appear in pstore). I did check that init was correctly panic’ing on failure by inducing an init failure in recovery mode and observing the panic (recovery mode allows you to run adb). But this induced panic also didn’t show up in pstore, meaning there’s actually some problem with pstore and early panics.

    Security is the problem (as usual)

    The actual problem turned out to be security (as usual): The pixel-3 does encrypted boot panic logs. The way this seems to work (at least in my reading of the google additional pstore patches) is that the bootloader itself encrypts the pstore ram area with a key on the /data partition, which means it only becomes visible after the device is unlocked. Unfortunately, if you trigger a panic before the device is unlocked (by echoing ‘c’ to /proc/sysrq-trigger) the panic message is lost, so pstore itself is useless for debugging early boot. There seems to be some communication of the keys by the vendor proprietary ramoops binary making it very difficult to figure out how it’s being done.

    Why the early panic message is lost is a bit mysterious, but unfortunately pstore on the pixel-3 has several proprietary components around the encrypted message handling that make it hard to debug. I suspect if you don’t set up the pstore encryption keys, the bootloader erases the pstore ram area instead of encrypting it, but I can’t prove that.

    Although it might be possible to fix the pstore drivers to preserve the ramoops from before device unlock, the participation of the proprietary bootloader in preserving the memory doesn’t make that look like a promising avenue to explore.

    Anatomy of the Pixel-3 Boot Sequence

    The Pixel-3 device boots through recovery. What this means is that the initial ramdisk (from boot.img) init is what boots both the recovery and normal boot paths. The only difference is that for recovery (and fastboot), the device stays in the ramdisk and for normal boot it mounts the /system partition and pivots to it. What makes this happen or not is the boot flag androidboot.force_normal_boot=1 which is added by the bootloader. Pretty much all the binary content and init rc files in the ramdisk are for recovery and its allied menus.

    Since the boot paths are pretty radically different, because the normal boot first pivots to a first stage before going on to a second, but in the manner of containers, it might be possible to boot recovery first, start a dmesg logger and then re-exec init through the normal path

    Forcing Re-Exec

    The idea is to signal init to re-exec itself for the normal path. Of course, there have to be a few changes to do this: An item has to be added to the recovery menu to signal init and init itself has to be modified to do the re-exec on the signal (note you can’t just kick off an init with a new command line because init must be pid 1 for booting). Once this is done, there are problems with selinux (it won’t actually allow init to re-exec) and some mount moves. The selinux problem is fixable by switching it from enforcing to permissive (boot option androidboot.selinux=permissive) and the mount moves (which are forbidden if you’re running binaries from the mount points being moved) can instead become bind mounts. The whole patch becomes 31 insertions across 7 files in android_system_core.

    The signal I chose was SIGUSR1, which isn’t usually used by anything in the bootloader and the addition of a menu item to recovery to send this signal to init was also another trivial patch. So finally we have a system from which I can start adb to trace the kernel log (adb shell dmesg -w) and then signal to init to re-exec. Surprisingly this worked and produced as the last message fragment:

    [ 190.966881] init: [libfs_mgr]Created logical partition system_a on device /dev/block/dm-0[ 190.967697] init: [libfs_mgr]Created logical partition vendor_a on device /dev/block/dm-1[ 190.968367] init: [libfs_mgr]Created logical partition product_a on device /dev/block/dm-2[ 190.969024] init: [libfs_mgr]Created logical partition system_ext_a on device /dev/block/dm-3[ 190.969067] init: DSU not detected, proceeding with normal boot[ 190.982957] init: [libfs_avb]Invalid hash size:[ 190.982967] init: [libfs_avb]Failed to verify vbmeta digest[ 190.982972] init: [libfs_avb]vbmeta digest error isn't allowed[ 190.982980] init: Failed to open AvbHandle: No such file or directory[ 190.982987] init: Failed to setup verity for '/system': No such file or directory[ 190.982993] init: Failed to mount /system: No such file or directory[ 190.983030] init: Failed to mount required partitions early …[ 190.983483] init: InitFatalReboot: signal 6[ 190.984849] init: #00 pc 0000000000123b38 /system/bin/init[ 190.984857] init: #01 pc 00000000000bc9a8 /system/bin/init[ 190.984864] init: #02 pc 000000000001595c /system/lib64/libbase.so[ 190.984869] init: #03 pc 0000000000014f8c /system/lib64/libbase.so[ 190.984874] init: #04 pc 00000000000e6984 /system/bin/init[ 190.984878] init: #05 pc 00000000000aa144 /system/bin/init[ 190.984883] init: #06 pc 00000000000487dc /system/lib64/libc.so[ 190.984889] init: Reboot ending, jumping to kernel

    Which indicates exactly where the problem is.

    Fixing the problem

    Once the messages are identified, the problem turns out to be in system/core ec10d3cf6 “libfs_avb: verifying vbmeta digest early”, which is inherited from AOSP and which even says in in it’s commit message “the device will not boot if: 1. The image is signed with FLAGS_VERIFICATION_DISABLED is set 2. The device state is locked” which is basically my boot state, so thanks for that one google. Reverting this commit can be done cleanly and now the signed image boots without a problem.

    I note that I could also simply add hashtree verification to my boot, but LineageOS is based on the eng target, which has FLAGS_VERIFICATION_DISABLED built into the main build Makefile. It might be possible to change it, but not easily I’m guessing … although I might try fixing it this way at some point, since it would make my phones much more secure.

    Conclusion

    Debugging android early boot is still a terribly hard problem. Probably someone with more patience for disassembling proprietary binaries could take apart pixel-3 vendor ramoops and figure out if it’s possible to get a pstore oops log out of early boot (which would be the easiest way to debug problems). But failing that the simple hack to re-exec init worked enough to show me where the problem was (of course, if init had continued longer it would likely have run into other issues caused by the way I hacked it).

    https://blog.hansenpartnership.com/debugging-android-early-boot-failures/

    #00 #01 #02 #03 #04 #05 #06 #android #androidBoot #androidDebugging #androidVerifiedBoot #AVB #lineageos

    #android #androidboot #androiddebugging #androidverifiedboot #avb #lineageos
  21. jejb @[email protected] ·

    Back in my blog post about Securing the Google SIP Stack, I did say I’d look at re-enabling SIP in Android-12, so with a view to doing that I tried building and booting LineageOS 19.1, but it crashed really early in the boot sequence (after the boot splash but before the boot animation started). It turns out that information on debugging the android early boot sequence is a bit scarce, so I thought I should write a post about how I did it just in case it helps someone else who’s struggling with a similar early boot problem.

    How I usually Build and Boot Android

    My builds are standard LineageOS with my patches to fix SIP and not much else. However, I do replace the debug keys with my signing keys and I also have an AVB key installed in the phone’s third party keyslot with which I sign the vbmeta for boot. This actually means that my phone is effectively locked but with a user supplied key (Yellow as google puts it).

    My phone is now a pixel 3 (I had to say goodbye to the old Nexus One thanks to the US 3G turn off) and I do have a slightly broken Pixel 3 I play with for experimental patches, which is where I was trying to install Android-12.

    Signing Seems to be the Problem

    Just to verify my phone could actually boot a stock LineageOS (it could) I had to unlock it and this lead to the discovery that once unlocked, it would also boot my custom rom as well, so whatever was failing in early boot seemed to be connected with the device being locked.

    I also discovered an interesting bug in the recovery rom fastboot: If you’re booting locked with your own keys, it will still let you perform all the usually forbidden fastboot commands (the one I was using was set_active). It turns out to be because of a bug in AOSP which treats yellow devices as unlocked in fastboot. Somewhat handy for debugging, but not so hot for security …

    And so to Debugging Early Boot

    The big problem with Android is there’s no way to get the console messages for early boot. Even if you enable adb early, it doesn’t get started until quite far in to the boot animation (which was way after the crash I was tripping over). However, android does have a pstore (previously ramoops) driver that can give you access to the previously crashed boot’s kernel messages (early init, fortunately, mostly logs to the kernel message log).

    Forcing init to crash on failure

    Ordinarily an init failure prints a message and reboots (to the bootloader), which doesn’t excite pstore into saving the kernel message log. fortunately there is a boot option (androidboot.init_fatal_panic) which can be set in the boot options (or kernel command line for a pixel-3 which can only boot the 4.9 kernel). If you build your own android, it’s fairly easy to add things to the android commandline (which is in boot.img) because all you need to do is extract BOOT/cmdline from the intermediate zip file you sign add any boot options you need and place it back in the zip file (before you sign it).

    Unfortunately, this expedient didn’t work (no console logs appear in pstore). I did check that init was correctly panic’ing on failure by inducing an init failure in recovery mode and observing the panic (recovery mode allows you to run adb). But this induced panic also didn’t show up in pstore, meaning there’s actually some problem with pstore and early panics.

    Security is the problem (as usual)

    The actual problem turned out to be security (as usual): The pixel-3 does encrypted boot panic logs. The way this seems to work (at least in my reading of the google additional pstore patches) is that the bootloader itself encrypts the pstore ram area with a key on the /data partition, which means it only becomes visible after the device is unlocked. Unfortunately, if you trigger a panic before the device is unlocked (by echoing ‘c’ to /proc/sysrq-trigger) the panic message is lost, so pstore itself is useless for debugging early boot. There seems to be some communication of the keys by the vendor proprietary ramoops binary making it very difficult to figure out how it’s being done.

    Why the early panic message is lost is a bit mysterious, but unfortunately pstore on the pixel-3 has several proprietary components around the encrypted message handling that make it hard to debug. I suspect if you don’t set up the pstore encryption keys, the bootloader erases the pstore ram area instead of encrypting it, but I can’t prove that.

    Although it might be possible to fix the pstore drivers to preserve the ramoops from before device unlock, the participation of the proprietary bootloader in preserving the memory doesn’t make that look like a promising avenue to explore.

    Anatomy of the Pixel-3 Boot Sequence

    The Pixel-3 device boots through recovery. What this means is that the initial ramdisk (from boot.img) init is what boots both the recovery and normal boot paths. The only difference is that for recovery (and fastboot), the device stays in the ramdisk and for normal boot it mounts the /system partition and pivots to it. What makes this happen or not is the boot flag androidboot.force_normal_boot=1 which is added by the bootloader. Pretty much all the binary content and init rc files in the ramdisk are for recovery and its allied menus.

    Since the boot paths are pretty radically different, because the normal boot first pivots to a first stage before going on to a second, but in the manner of containers, it might be possible to boot recovery first, start a dmesg logger and then re-exec init through the normal path

    Forcing Re-Exec

    The idea is to signal init to re-exec itself for the normal path. Of course, there have to be a few changes to do this: An item has to be added to the recovery menu to signal init and init itself has to be modified to do the re-exec on the signal (note you can’t just kick off an init with a new command line because init must be pid 1 for booting). Once this is done, there are problems with selinux (it won’t actually allow init to re-exec) and some mount moves. The selinux problem is fixable by switching it from enforcing to permissive (boot option androidboot.selinux=permissive) and the mount moves (which are forbidden if you’re running binaries from the mount points being moved) can instead become bind mounts. The whole patch becomes 31 insertions across 7 files in android_system_core.

    The signal I chose was SIGUSR1, which isn’t usually used by anything in the bootloader and the addition of a menu item to recovery to send this signal to init was also another trivial patch. So finally we have a system from which I can start adb to trace the kernel log (adb shell dmesg -w) and then signal to init to re-exec. Surprisingly this worked and produced as the last message fragment:

    [ 190.966881] init: [libfs_mgr]Created logical partition system_a on device /dev/block/dm-0[ 190.967697] init: [libfs_mgr]Created logical partition vendor_a on device /dev/block/dm-1[ 190.968367] init: [libfs_mgr]Created logical partition product_a on device /dev/block/dm-2[ 190.969024] init: [libfs_mgr]Created logical partition system_ext_a on device /dev/block/dm-3[ 190.969067] init: DSU not detected, proceeding with normal boot[ 190.982957] init: [libfs_avb]Invalid hash size:[ 190.982967] init: [libfs_avb]Failed to verify vbmeta digest[ 190.982972] init: [libfs_avb]vbmeta digest error isn't allowed[ 190.982980] init: Failed to open AvbHandle: No such file or directory[ 190.982987] init: Failed to setup verity for '/system': No such file or directory[ 190.982993] init: Failed to mount /system: No such file or directory[ 190.983030] init: Failed to mount required partitions early …[ 190.983483] init: InitFatalReboot: signal 6[ 190.984849] init: #00 pc 0000000000123b38 /system/bin/init[ 190.984857] init: #01 pc 00000000000bc9a8 /system/bin/init[ 190.984864] init: #02 pc 000000000001595c /system/lib64/libbase.so[ 190.984869] init: #03 pc 0000000000014f8c /system/lib64/libbase.so[ 190.984874] init: #04 pc 00000000000e6984 /system/bin/init[ 190.984878] init: #05 pc 00000000000aa144 /system/bin/init[ 190.984883] init: #06 pc 00000000000487dc /system/lib64/libc.so[ 190.984889] init: Reboot ending, jumping to kernel

    Which indicates exactly where the problem is.

    Fixing the problem

    Once the messages are identified, the problem turns out to be in system/core ec10d3cf6 “libfs_avb: verifying vbmeta digest early”, which is inherited from AOSP and which even says in in it’s commit message “the device will not boot if: 1. The image is signed with FLAGS_VERIFICATION_DISABLED is set 2. The device state is locked” which is basically my boot state, so thanks for that one google. Reverting this commit can be done cleanly and now the signed image boots without a problem.

    I note that I could also simply add hashtree verification to my boot, but LineageOS is based on the eng target, which has FLAGS_VERIFICATION_DISABLED built into the main build Makefile. It might be possible to change it, but not easily I’m guessing … although I might try fixing it this way at some point, since it would make my phones much more secure.

    Conclusion

    Debugging android early boot is still a terribly hard problem. Probably someone with more patience for disassembling proprietary binaries could take apart pixel-3 vendor ramoops and figure out if it’s possible to get a pstore oops log out of early boot (which would be the easiest way to debug problems). But failing that the simple hack to re-exec init worked enough to show me where the problem was (of course, if init had continued longer it would likely have run into other issues caused by the way I hacked it).

    https://blog.hansenpartnership.com/debugging-android-early-boot-failures/

    #00 #01 #02 #03 #04 #05 #06 #android #androidBoot #androidDebugging #androidVerifiedBoot #AVB #lineageos

    #lineageos #avb #androidverifiedboot #androiddebugging #androidboot #android
  22. jejb @[email protected] ·

    Back in my blog post about Securing the Google SIP Stack, I did say I’d look at re-enabling SIP in Android-12, so with a view to doing that I tried building and booting LineageOS 19.1, but it crashed really early in the boot sequence (after the boot splash but before the boot animation started). It turns out that information on debugging the android early boot sequence is a bit scarce, so I thought I should write a post about how I did it just in case it helps someone else who’s struggling with a similar early boot problem.

    How I usually Build and Boot Android

    My builds are standard LineageOS with my patches to fix SIP and not much else. However, I do replace the debug keys with my signing keys and I also have an AVB key installed in the phone’s third party keyslot with which I sign the vbmeta for boot. This actually means that my phone is effectively locked but with a user supplied key (Yellow as google puts it).

    My phone is now a pixel 3 (I had to say goodbye to the old Nexus One thanks to the US 3G turn off) and I do have a slightly broken Pixel 3 I play with for experimental patches, which is where I was trying to install Android-12.

    Signing Seems to be the Problem

    Just to verify my phone could actually boot a stock LineageOS (it could) I had to unlock it and this lead to the discovery that once unlocked, it would also boot my custom rom as well, so whatever was failing in early boot seemed to be connected with the device being locked.

    I also discovered an interesting bug in the recovery rom fastboot: If you’re booting locked with your own keys, it will still let you perform all the usually forbidden fastboot commands (the one I was using was set_active). It turns out to be because of a bug in AOSP which treats yellow devices as unlocked in fastboot. Somewhat handy for debugging, but not so hot for security …

    And so to Debugging Early Boot

    The big problem with Android is there’s no way to get the console messages for early boot. Even if you enable adb early, it doesn’t get started until quite far in to the boot animation (which was way after the crash I was tripping over). However, android does have a pstore (previously ramoops) driver that can give you access to the previously crashed boot’s kernel messages (early init, fortunately, mostly logs to the kernel message log).

    Forcing init to crash on failure

    Ordinarily an init failure prints a message and reboots (to the bootloader), which doesn’t excite pstore into saving the kernel message log. fortunately there is a boot option (androidboot.init_fatal_panic) which can be set in the boot options (or kernel command line for a pixel-3 which can only boot the 4.9 kernel). If you build your own android, it’s fairly easy to add things to the android commandline (which is in boot.img) because all you need to do is extract BOOT/cmdline from the intermediate zip file you sign add any boot options you need and place it back in the zip file (before you sign it).

    Unfortunately, this expedient didn’t work (no console logs appear in pstore). I did check that init was correctly panic’ing on failure by inducing an init failure in recovery mode and observing the panic (recovery mode allows you to run adb). But this induced panic also didn’t show up in pstore, meaning there’s actually some problem with pstore and early panics.

    Security is the problem (as usual)

    The actual problem turned out to be security (as usual): The pixel-3 does encrypted boot panic logs. The way this seems to work (at least in my reading of the google additional pstore patches) is that the bootloader itself encrypts the pstore ram area with a key on the /data partition, which means it only becomes visible after the device is unlocked. Unfortunately, if you trigger a panic before the device is unlocked (by echoing ‘c’ to /proc/sysrq-trigger) the panic message is lost, so pstore itself is useless for debugging early boot. There seems to be some communication of the keys by the vendor proprietary ramoops binary making it very difficult to figure out how it’s being done.

    Why the early panic message is lost is a bit mysterious, but unfortunately pstore on the pixel-3 has several proprietary components around the encrypted message handling that make it hard to debug. I suspect if you don’t set up the pstore encryption keys, the bootloader erases the pstore ram area instead of encrypting it, but I can’t prove that.

    Although it might be possible to fix the pstore drivers to preserve the ramoops from before device unlock, the participation of the proprietary bootloader in preserving the memory doesn’t make that look like a promising avenue to explore.

    Anatomy of the Pixel-3 Boot Sequence

    The Pixel-3 device boots through recovery. What this means is that the initial ramdisk (from boot.img) init is what boots both the recovery and normal boot paths. The only difference is that for recovery (and fastboot), the device stays in the ramdisk and for normal boot it mounts the /system partition and pivots to it. What makes this happen or not is the boot flag androidboot.force_normal_boot=1 which is added by the bootloader. Pretty much all the binary content and init rc files in the ramdisk are for recovery and its allied menus.

    Since the boot paths are pretty radically different, because the normal boot first pivots to a first stage before going on to a second, but in the manner of containers, it might be possible to boot recovery first, start a dmesg logger and then re-exec init through the normal path

    Forcing Re-Exec

    The idea is to signal init to re-exec itself for the normal path. Of course, there have to be a few changes to do this: An item has to be added to the recovery menu to signal init and init itself has to be modified to do the re-exec on the signal (note you can’t just kick off an init with a new command line because init must be pid 1 for booting). Once this is done, there are problems with selinux (it won’t actually allow init to re-exec) and some mount moves. The selinux problem is fixable by switching it from enforcing to permissive (boot option androidboot.selinux=permissive) and the mount moves (which are forbidden if you’re running binaries from the mount points being moved) can instead become bind mounts. The whole patch becomes 31 insertions across 7 files in android_system_core.

    The signal I chose was SIGUSR1, which isn’t usually used by anything in the bootloader and the addition of a menu item to recovery to send this signal to init was also another trivial patch. So finally we have a system from which I can start adb to trace the kernel log (adb shell dmesg -w) and then signal to init to re-exec. Surprisingly this worked and produced as the last message fragment:

    [ 190.966881] init: [libfs_mgr]Created logical partition system_a on device /dev/block/dm-0[ 190.967697] init: [libfs_mgr]Created logical partition vendor_a on device /dev/block/dm-1[ 190.968367] init: [libfs_mgr]Created logical partition product_a on device /dev/block/dm-2[ 190.969024] init: [libfs_mgr]Created logical partition system_ext_a on device /dev/block/dm-3[ 190.969067] init: DSU not detected, proceeding with normal boot[ 190.982957] init: [libfs_avb]Invalid hash size:[ 190.982967] init: [libfs_avb]Failed to verify vbmeta digest[ 190.982972] init: [libfs_avb]vbmeta digest error isn't allowed[ 190.982980] init: Failed to open AvbHandle: No such file or directory[ 190.982987] init: Failed to setup verity for '/system': No such file or directory[ 190.982993] init: Failed to mount /system: No such file or directory[ 190.983030] init: Failed to mount required partitions early …[ 190.983483] init: InitFatalReboot: signal 6[ 190.984849] init: #00 pc 0000000000123b38 /system/bin/init[ 190.984857] init: #01 pc 00000000000bc9a8 /system/bin/init[ 190.984864] init: #02 pc 000000000001595c /system/lib64/libbase.so[ 190.984869] init: #03 pc 0000000000014f8c /system/lib64/libbase.so[ 190.984874] init: #04 pc 00000000000e6984 /system/bin/init[ 190.984878] init: #05 pc 00000000000aa144 /system/bin/init[ 190.984883] init: #06 pc 00000000000487dc /system/lib64/libc.so[ 190.984889] init: Reboot ending, jumping to kernel

    Which indicates exactly where the problem is.

    Fixing the problem

    Once the messages are identified, the problem turns out to be in system/core ec10d3cf6 “libfs_avb: verifying vbmeta digest early”, which is inherited from AOSP and which even says in in it’s commit message “the device will not boot if: 1. The image is signed with FLAGS_VERIFICATION_DISABLED is set 2. The device state is locked” which is basically my boot state, so thanks for that one google. Reverting this commit can be done cleanly and now the signed image boots without a problem.

    I note that I could also simply add hashtree verification to my boot, but LineageOS is based on the eng target, which has FLAGS_VERIFICATION_DISABLED built into the main build Makefile. It might be possible to change it, but not easily I’m guessing … although I might try fixing it this way at some point, since it would make my phones much more secure.

    Conclusion

    Debugging android early boot is still a terribly hard problem. Probably someone with more patience for disassembling proprietary binaries could take apart pixel-3 vendor ramoops and figure out if it’s possible to get a pstore oops log out of early boot (which would be the easiest way to debug problems). But failing that the simple hack to re-exec init worked enough to show me where the problem was (of course, if init had continued longer it would likely have run into other issues caused by the way I hacked it).

    https://blog.hansenpartnership.com/debugging-android-early-boot-failures/

    #00 #01 #02 #03 #04 #05 #06 #android #androidBoot #androidDebugging #androidVerifiedBoot #AVB #lineageos

    #android #androidboot #androiddebugging #androidverifiedboot #avb #lineageos
  23. Shick @[email protected] ·

    Reposting this old meme #nightbound #originalcharacter #ocart #artist

    #nightbound #originalcharacter #ocart #artist
  24. abc+ | Notícias, esportes, entretenimento e muito + para você [Unofficial] @[email protected] ·

    Com 13º no bolso, população corre para o comércio para garantir os presentes de Natal

    fed.brid.gy/r/https://www.abcm

    #novohamburgo #regiao #13osalario #cdlnh #comercio #comerciodenovohamburgo
  25. Shick @[email protected] ·

    Love ain’t meant for people like me and you, Stephanie #jokerout #fanart #artist #souvenirpop

    #jokerout #fanart #artist #souvenirpop
  26. Shick @[email protected] ·

    Love ain’t meant for people like me and you, Stephanie #jokerout #fanart #artist #souvenirpop

    #jokerout #fanart #artist #souvenirpop
  27. Shick @[email protected] ·

    Love ain’t meant for people like me and you, Stephanie #jokerout #fanart #artist #souvenirpop

    #jokerout #fanart #artist #souvenirpop
  28. Shick @[email protected] ·

    Love ain’t meant for people like me and you, Stephanie #jokerout #fanart #artist #souvenirpop

    #souvenirpop #artist #fanart #jokerout
  29. Shick @[email protected] ·

    Love ain’t meant for people like me and you, Stephanie #jokerout #fanart #artist #souvenirpop

    #jokerout #fanart #artist #souvenirpop