#paperlessngx — Public Fediverse posts

n8n batch processing

In this posting I’ll describe how I set up my n8n workflow to update documents in Paperless NGX. My initial workflow was ok, but left room for improvement.

Initial Workflow

I’m using Paperless NGX to store all my analog and digital documents for archival. While Paperless does an OK job to recognise the document type and tags automatically with its built-in machine learning models, I wanted to also have a summary of the document and better document titles.

Since I’m scanning the analog documents with my printer all those documents are titled “EPSON WF-4830”.

I’ve set up a workflow in Paperless which call’s a web hook whenever a new document arrives or a special tag is added to a document.

The web hook is the trigger of a workflow on my N8N instance. The workflow would consume the document from Paperless, generate the summary, generate the title and update said document with the results.

To make it a little more interesting the LLM that N8N is using sits on my laptop, which n8n reaches via a Tailscale tunnel. That is not always on obviously.

First improvement attempt

In a first attempt to improve the workflow I wanted to use a queue for the documents. So that if my laptop isn’t available the documents from Paperless would just be added to the queue and processed later.

With the initial setup, the n8n workflow would just fail if it can’t reach the LLM for processing.

I create a separate workflow in n8n which takes the document id from the Paperless and inserts that into a simple data table in n8n.

A second workflow, which is triggered by a timer every 5 minutes, gets all the entries from the data table and processes them.

Most N8N nodes can deal with input arrays. Means if their input contains multiple elements, they will automatically iterate of all the elements in the input and execute their function on each item. The output of the node will again be an array.

This works fine if the process of the node is very robust. Means if every item is processed for sure. If a single items causes a problem, the node execution will fail entirely.

Subsequent steps are never reached. In my case that meant for longer batches there was a high chance that the whole batch failed because of a network issue or me shutting down my laptop.

The Improvement

N8N also has a dedicated “Loop” node. This one will take an array as input and pass a single item onto the next node until all items have been processed.

Using this node my workflow would only ever process a single document, update it and remove it from the data table. Failure to process one document will not stop the entire workflow.

Thus even if my laptop dies in the middle of the processing the remaining queue would be process next time it comes online and timer kicks off the processing.

Of course I had to add a mutex to prevent the processing queue from running twice at the same time. The entire processing workflow in N8N now looks something like this:

#weeklyreview 33/2026

Week 33 started with cooking tomatoe sauce from a bucket of tomatoes I got from my dad. Let it simmer for a good 8 hours. Eventually that made 4 portions of delicious pasta sauce.

Tried the new Korean restaurant in the Oranienburger Strasse named “Bandol”. I like it a lot. They put some effort in the decoration of the job and the food was delicious. Only complaint is about them handing out single use plastic gloves so people don’t get dirty hands. There are napkins and a bathroom. Why produce more plastic trash? At first we were a little puzzled why they add condoms to the meal?

Tuesday evening the Beer & Burger crew met at “Die Alte Turnhalle” in Friedrichshain. The verdict is: ok food. Little choice, high prices. Was a fun evening nevertheless with the folks.

Had to pass on the Hackerstammtisch though this month due to the scheduling conflict.

living dangerously

Our tenant renting a room in our flat really went over the top this week. I was working from home and all of a sudden my monitors went off. Odd time for a power outage I thought. Checked the rest of the flat and it became apparent that it must be something in our flat. The lights were still working. Just the power sockets were off. Checked the fuse box and found the FI (error current fuse ) triggered. Started checking all the power sockets to see what’s causing this. Eventually the tenant came out of her room complaining about the power loss. I asked her to remove everything from the power sockets to find the cause. Eventually the fuse stayed in. So that must have been something in her room. I asked whether I can check and she let me in. I found a power distribution unit sitting in the floor in a puddle of water. And she was happily operating it. The PDU was also wet.

Apparently she had accidentally poured water over it and didn’t bother to disconnect or dry it. Just kept on with her business of using her face sauna. I was utterly speechless about such stupidity and ignorance.

Contract terminated.

Praying Mantis

Next morning I saw a pretty large insect on our house door in Berlin. A beautiful green praying mantis. I had never seen them in the wild here before. Thought they are not endemic here. But turns out they are meanwhile. And just by accident I saw two other people post pictures about their sightings in the following days around here.

#project25

Got rid of a whole bunch of cardboard and started removing the floor in the next room. Also checked how thick the concrete in the ground floor is over the bare earth.

N8N, Paperless, Mastodon, LinkDing

Spent some time optimising my document workflow. When I upload new documents into my PaperlessNGX instance a N8N workflow is triggered. This will add a summary into a custom field and generate a meaningful document title. Things Paperless doesn’t do out of the box yet.

Since that depends on my Laptop with a running Ollama to be online, it fails when the laptop is not connected. So I split the workflow into a queue and processor. Adding documents in Paperless just adds them to a queue table in my N8n. A timer kicks off the processing workflow every 5 minutes. This one iterates over the documents one by one now and removes the document from the queue only when it finishes successfully. This way the documents get processed eventually. Even when my laptop isn’t online all the time.

Second workflow I was working on is the archival of my Mastodon Bookmarks. A regular check reads my latest bookmarks in Mastodon and adds them to my LinkDing instance if they are not already there.

LinkDing will also create an HTML Archive of every bookmark upon submission. This way I have a copy of content that might vanish over time.

@_g_baer_ @3nt3 ich würde die PDF im #paperlessNGX schieben und mich über eine PDF mit Text freuen... Welches Tool da läuft ist sicher raus zu bekommen.

Habe heute im Bekanntenkreis jemandem Paperless-ngx gezeigt. Besonders beeindruckt hat ihn, dass die Dokumente in einer "normalen" Verzeichnisstruktur mit den Original-Namen abgelegt werden (sofern man Paperless-ngx entsprechend konfiguriert). Das hat ihm die Sorge genommen, was passiert, wenn Paperless-ngx mal plötzlich gar nicht mehr funktionieren sollte.

#paperless #paperlessngx

Paperless-ngx v3.0.5 is a massive bug squash with 17+ fixes including AI suggestion caching, MariaDB compatibility, PDF viewer loops, and better search query handling. #selfhosted #paperlessngx

https://github.com/paperless-ngx/paperless-ngx/releases/tag/v3.0.5

Too optimistic for paperless upgrade 😰
I have nightly backups, no changes today. restore was easy.
Then I could login with SSO, but no rights, no documents found.. panic.

Backup failed since a month 🙈
I changed compose directory, so I got new volumes 🙊

Everything good after fixing my stupid issues 🎉😅
#upgrade #paperless #paperlessngx

Paperless-ngx:

Aufräumen des Datenbestands = mit dem Datumsfilter lassen sich rasch sehr alte Dokumente anzeigen, die evtl. nicht mehr benötigt werden und gelöscht werden können:

#paperless #paperlessngx

Neuer Blog-Artikel zu Paperless-ngx 3 - Schwerpunkt: Versionierung

https://digital-cleaning.de/index.php/paperless-ngx-teil-26-versionierung-und-erweiterte-freigabemoeglichkeiten/

#paperless #paperlessngx

@_DigitalWriter_ ich sehe Dich immer wieder zu #paperlessngx trööten. Schau mal auf https://github.com/stefanbaur/rrpi-base-install vorbei, Branch docker-paperless-ngx, vielleicht ist das für Dich oder Deine Bubble interessant.

Paperless-ngx Versionierung

Das Umschalten zwischen Versionen ist sehr gut gelöst - damit erhält man rasch einen visuellen Vergleich:

#paperless #paperlessngx #versionierung

Paperless-ngx 3.0.2 ist erschienen.

KI-Verknüpfungsmöglichkeiten = werde ich nicht nutzen, da es mir bei meinem System keine Vorteile bringt sondern eher für Intransparenz sorgen würde.

Auch die neuen Freigaben von Ansichten oder Dokumentenbündel sind nicht gar so spannend, da ich so etwas lieber über die Nextcloud erledige.

Nützlich finde ich hingegen die neuen Versionierungsmöglichkeiten. Und die schnellere Suche in großen Datenbeständen.

#paperless #paperlessngx

Three things I learned after a few days of running paperless-ngx
1. If you want to distinguish documents by their type (like invoice, contract, etc.), use “document types”, not tags.
2. If you want to distinguish documents by “author” (like your employer for the paychecks, customers, service providers for invoices, etc.), use “correspondents”, not tags.
3. You don’t need a ton of tags if you use the other fields correctly.
#paperless #paperlessngx

Apparently, the document management system #PaperlessNGX got tempted by the dark side and now started to use #Claude.

https://lemmy.ml/post/50409569/26856320

Maybe it's time to check out a few alternatives…

Is there already a no-AI-fork of #paperlessngx?

@_DigitalWriter_
Soetwas habe ich schon. Ich nutze für Dockumente #Paperlessngx .
Meine Archive beinhalten alles was kein Text oder PDF dokument ist.

So I'm with #PaperlessNgx. Which also has some nice 3rd party #iOS clients.

#PaperParrot even supports scanning with my #Canon network MFD.

Both lack a very important feature - offline access. Well, PaperParrot has some, but it can download all your documents at once. You can't choose.

#selfhosted #selfhosting #selfhost

#PaperlessNgx is a huge, complicated, and slow. #Papra is modern, fast, and simple. Both have good OCR. Also, Papra has an official mobile app in the works.

Despite both storing originals in files on disk, with Papra you'll get all your documents in a single folder, while Paperless-ngx is able to carefully sort them by folders. This is critical to me, because in case of emergency my family should be able to detach our #HomeCloud storage and use it as an external drive on any PC.

Today I needed to find an old paper document. Also today I realized how badly I need a #PaperlessNgx or #Papra. Coincidence? I don't think so.

This is crazy how many useless papers we have accumulated over the years.

I set up both to figure out a better option.

#selfhosted #homelab #selfhosting #selfhost #personalCloud #HomeCloud

TBH: Das war gut investiertes Geld. 5200 Dokumente mit ca. 1000 Korrespondenten hätte ich nicht selber neu zuweisen können. Damit konnte ich eine "Jugendsünde" in meinem Paperless ziemlich erfolgreich fixen. #Paperlessngx #ClaudeCode

Paperless-ngx

Gelegentlich nehme ich Links zu bestimmten Dokumenten meines Paperless-ngx-Archivs in das Notizenfeld meiner ToDo-Anwendung auf.

Wenn ich auf mehrere Dokumente verweisen möchte, dann gibt es verschiedene Wege. Der einfachste: Dokumente kurz in Tabs aufrufen. Da Firefox, Vivaldi usw. Links "gebündelt" kopieren können, ist das mit einem Klick erledigt.

#paperless #paperlessngx #todo

Mal eine vielleicht provokante aber ernst gemeinte Frage: Welchen Vorteil bringt paperless-ngx gegenüber den Funktionen, die ein modernes OS (z. B. macOS) bietet?

Die Software vom Dokumentenscanner versieht die PDFs direkt mit Informationen aus dem OCR-Processing und legt sie in einer Directory-Struktur auf dem Rechner ab. Sie lassen sich sofort mit Spotlight bequem durchsuchen. Wenn man will, … 1/2

#paperlessngx

Drei Aktenordner voller Dokumente wollen in das paperless-ngx importiert werden. Spürt ihr auch meinen Elan?

#paperless #paperlessngx #scan

Uh, and that on Debian Bookworm you’re stuck with a broken GhostScript 10 is /probably/ mentioned somewhere in the fine print, but it’d have been nice if that featured a bit more prominently in the instructions/tooling than as a runtime error when you’ve done the update.

Fortunately upgrading in LXC from `bookworm` to `trixie` is quite easy (modulo running out of disk-space during the ensuing package-bonanza, which `apt` should’ve probably warned about).

#paperless #paperlessngx

Über "den kleinen Bruder von Paperless-ngx" Papra habe ich ja vor einiger Zeit in meinem Blog berichtet. https://digital-cleaning.de/index.php/papra-der-kleine-bruder-von-paperless-ngx/

Hat inzwischen eine Vielzahl neuer Funktionen erhalten, Verbesserungen von OCR und nun auch benutzerdefinierte Felder

#papra #paperless #paperlessngx

VPS läuft #KitchenOwl als ablöse für #Bring! und #FamilyWall, dann noch ein #CalDav Kalender, #UpTimeKuma als Meldestelle falls ein Service offline wäre. #Portainer für die visuelle Verwaltung der Dockerstruktur. Und zu guter letzt noch #OpenClaw. Weiterhin steht noch eine ausgediente #SchneiderWorkstation im Keller, was zum #HomeServer umgebaut werden soll. Hier sollte dann #Immich und #paperlessngx inkl. lokaler #KI das zuhause finden. (2/2)