#ioer_fdz — Public Fediverse posts on home.social

Alexander Dunkel @[email protected] · 2026-08-21 · 08:33 UTC

At #IOER_FDz we faced a common challenge: as our technical workflows, data publication guidelines, and infrastructure grows, we realized colleagues rarely read through our extensive documentation. Attention is limited, and when someone needs to solve a specific problem, large guides are rarely read. We needed a communication agent that could bridge the gap between individual questions and our shared internal knowledge base.

To address this, we now have an embedded #AnythingLLM assistant into our Material for MkDocs documentation. It works great! Here are a few interesting technical infos:

- To keep internal data locally and avoid commercial APIs, the inference is handled on university HPC hardware via ScaDS.AI API (TU Dresden). We paired `meta-llama/Llama-3.3-70B-Instruct` with `Qwen/Qwen3-Embedding-4B` to bypass the context window bottlenecks of smaller default embedders.

Feeding raw documentation into a RAG vector database was not _so_ trivial:

1. Ingesting raw repository Markdown caused the LLM to hallucinate URLs. It outputted broken relative `.md` paths, lacked awareness of the actual navigation hierarchy, and character-based chunking cut long tables off from metadata headers.
2. We rely on a number of mkdocs plugins (macros, live database tables pulled in CI&CD, `gen-files`). This meant that the raw repository files do not contain the final compiled text, so the LLM was first missing part of the information.

I added a native MkDocs build hook (`mkdocs_llm_hook.py`). The hook intercepts the compiled Markdown after macros are executed, resolves internal relative links to absolute production URLs, semantically pre-chunks content along Markdown headings (`#`, `##`, `###`), and produces hash-anchored links for the LLM. Metadata (`Canonical_URL` and `Document_Hierarchy`) is added to each chunk before syncing with the AnythingLLM API during `mkdocs build`.

I documented the hook script, system prompt, model parameters, and UI fixes in a Gist for those interested here:

https://gist.github.com/Sieboldianus/8c14da1370891ba2fcdf09882bd78daf

It is a nice way to help colleagues find needed information faster and with better relevance filtering. Accessibility bonus: You can ask it in any language!

#MkDocs #AnythingLLM #RAG #ScaDS #Python #DevOps #OpenSource

#ioer_fdz #anythingllm #mkdocs #rag #scads #python

Alexander Dunkel @[email protected] · 2026-08-21 · 08:33 UTC

At #IOER_FDz we faced a common challenge: as our technical workflows, data publication guidelines, and infrastructure grows, we realized colleagues rarely read through our extensive documentation. Attention is limited, and when someone needs to solve a specific problem, large guides are rarely read. We needed a communication agent that could bridge the gap between individual questions and our shared internal knowledge base.

To address this, we now have an embedded #AnythingLLM assistant into our Material for MkDocs documentation. It works great! Here are a few interesting technical infos:

- To keep internal data locally and avoid commercial APIs, the inference is handled on university HPC hardware via ScaDS.AI API (TU Dresden). We paired `meta-llama/Llama-3.3-70B-Instruct` with `Qwen/Qwen3-Embedding-4B` to bypass the context window bottlenecks of smaller default embedders.

Feeding raw documentation into a RAG vector database was not _so_ trivial:

1. Ingesting raw repository Markdown caused the LLM to hallucinate URLs. It outputted broken relative `.md` paths, lacked awareness of the actual navigation hierarchy, and character-based chunking cut long tables off from metadata headers.
2. We rely on a number of mkdocs plugins (macros, live database tables pulled in CI&CD, `gen-files`). This meant that the raw repository files do not contain the final compiled text, so the LLM was first missing part of the information.

I added a native MkDocs build hook (`mkdocs_llm_hook.py`). The hook intercepts the compiled Markdown after macros are executed, resolves internal relative links to absolute production URLs, semantically pre-chunks content along Markdown headings (`#`, `##`, `###`), and produces hash-anchored links for the LLM. Metadata (`Canonical_URL` and `Document_Hierarchy`) is added to each chunk before syncing with the AnythingLLM API during `mkdocs build`.

I documented the hook script, system prompt, model parameters, and UI fixes in a Gist for those interested here:

https://gist.github.com/Sieboldianus/8c14da1370891ba2fcdf09882bd78daf

It is a nice way to help colleagues find needed information faster and with better relevance filtering. Accessibility bonus: You can ask it in any language!

#MkDocs #AnythingLLM #RAG #ScaDS #Python #DevOps #OpenSource

#ioer_fdz #anythingllm #mkdocs #rag #scads #python

Alexander Dunkel @[email protected] · 2026-08-21 · 08:33 UTC

At #IOER_FDz we faced a common challenge: as our technical workflows, data publication guidelines, and infrastructure grows, we realized colleagues rarely read through our extensive documentation. Attention is limited, and when someone needs to solve a specific problem, large guides are rarely read. We needed a communication agent that could bridge the gap between individual questions and our shared internal knowledge base.

To address this, we now have an embedded #AnythingLLM assistant into our Material for MkDocs documentation. It works great! Here are a few interesting technical infos:

- To keep internal data locally and avoid commercial APIs, the inference is handled on university HPC hardware via ScaDS.AI API (TU Dresden). We paired `meta-llama/Llama-3.3-70B-Instruct` with `Qwen/Qwen3-Embedding-4B` to bypass the context window bottlenecks of smaller default embedders.

Feeding raw documentation into a RAG vector database was not _so_ trivial:

1. Ingesting raw repository Markdown caused the LLM to hallucinate URLs. It outputted broken relative `.md` paths, lacked awareness of the actual navigation hierarchy, and character-based chunking cut long tables off from metadata headers.
2. We rely on a number of mkdocs plugins (macros, live database tables pulled in CI&CD, `gen-files`). This meant that the raw repository files do not contain the final compiled text, so the LLM was first missing part of the information.

I added a native MkDocs build hook (`mkdocs_llm_hook.py`). The hook intercepts the compiled Markdown after macros are executed, resolves internal relative links to absolute production URLs, semantically pre-chunks content along Markdown headings (`#`, `##`, `###`), and produces hash-anchored links for the LLM. Metadata (`Canonical_URL` and `Document_Hierarchy`) is added to each chunk before syncing with the AnythingLLM API during `mkdocs build`.

I documented the hook script, system prompt, model parameters, and UI fixes in a Gist for those interested here:

https://gist.github.com/Sieboldianus/8c14da1370891ba2fcdf09882bd78daf

It is a nice way to help colleagues find needed information faster and with better relevance filtering. Accessibility bonus: You can ask it in any language!

#MkDocs #AnythingLLM #RAG #ScaDS #Python #DevOps #OpenSource

#ioer_fdz #anythingllm #mkdocs #rag #scads #python