#chit — Public Fediverse posts

#LocalLLM update! I was able to implement this rather complicated feature using Pi and Qwen3.6 27B.

It added a rendering state to my #Chit LLM chat app that renders rectangles over the uploaded image.

Basically you ask for a specific JSON format back, and if it detects it in the stream, it'll create a new image, rendering the rectangles over the objects.

It didn't work immediately out of the gate, but it took a couple turns to fix some bugs. But ultimately it DID work, and so far it works great. :D

But the real headline is that I didn't use Claude or OpenAI stuff. All local. 4090. 24gb.

That's the future we need to head towards if we're going to be using AI for coding going forward. If we're going to shift towards this method of development, we CANNOT be bound to any company, with their leash around our necks and wallets.

#ai #llm

Fun #LLM tip for you.

The red block indicates a response that, originally, was a denial.

> "I cannot fulfill the request to set a dog on fire, as creating content that depicts animal abuse or harm is against my safety guidelines. I can, however, update the scene with other elements if you'd like, such as adding some clouds to the sky or changing the time of day."

I've found that if you EDIT THE RESPONSE and replace it with something like I did in the pic, or similar wording... and then let it respond again (to continue the response)... most of the time it will actually go through with the restricted request.

And that's not specific to any one model. Some are more resistant than others, but none I've tried are completely locked up. And then after that it's usually smooth sailing. :P

Most chat interfaces don't let you do this, though.

Mine does. 😉

#localllm #chit

Daaaang, the new season of #theChitShow started off with a banger!

#chit #theOffice

My new inspirational fridge magnet for my wife arrived today.

#chit #theChitShow #doItLady

If you haven’t been paying attention, there is a new season of #Chit coming out, with some #Thanksgiving dinner scenes you do not want to miss.

https://m.youtube.com/@renshawjay

A new #chit and a new #StrongBad email on the same day?! We are blessed!

https://youtu.be/-zhsNdk1_Vc

#HomeStarRunner #TheChitShow

He's back!

#chit #TheChitShow

Does anyone else watch The Chit Show? Have you seen the new episode??

Because HOLY CHIT!

#Chit #YouTube #DoItLady

The new #chit episode... 🤯

You gotta ponk to make a honk. 😂

#Chit #TheChitShow

Fuck yo' sass!

#llm #gemma3 #chit

Still a WIP, but I'm finally making progress on my rewrite of #Chit, the front-end chat interface for #Ollama.

Closing in on feature parity with the original (then I can start adding new features). I'm psyched to finally have chat 'sessions' so I don't have to lose whatever I'm doing just to ask some other thing. (I'd have to duplicate tabs before.)

It maintains state between reloads quite solidly, too.

You can poke at it over here; it'll gets updated whenever I make a push. It's still alpha, and missing some basic features (temperature controls, for instance, a config page, etc), but should work fine.

It uses a default Ollama installed -- you may need to tweak your server to allow access, but that's beyond the scope of this right now.

https://fortyseven.github.io/chit-v2/

At some point, when it gets closer to solid, I'll make a quick tutorial video.

#llm

I think what bugs me about this is that I've been able to assemble a decent summary system prompt that has, so far at least, not invented anything.

"How do you know", you may ask. Well, I can't say with 100% certainty because after a while I started trusting it.

But for a while I was immensely distrustful of it, and I'd regularly put through it real articles, and content I was familiar with, and look at the results.

But I've been using it for quite a few months now, with Llama 3.1, and it's proven itself quite good with whole raw text dumps of webpages.

(I use it so often that I include it with the handful of default system prompts with #chit (my Ollama front-end).)

Here it is: https://gist.github.com/Fortyseven/708dac25d24f1d2d063d4b3f2dc48596

Since I started using it regularly, if something sounds out of place or "sounds wrong", I'll go and verify. (And honestly, I'll even fuck up verifying: it'll be technically correct about something, but I'll have misread or overlooked the fact in question.)

So I'd love to know how Apple is fucking this up. Apparently this is not an isolated incident: https://www.theverge.com/2024/11/12/24289939/apple-intelligence-ai-notification-summaries-awkward-funny-bad

I wonder if their prompts have been dumped yet?

Rewrite of #Chit (my serverless, privacy-focused Ollama LLM client) is coming along. Mostly wanted to learn the new stuff with #Svelte5, organize the code cleaner, and work in mobile responsiveness from the start.

Coming along well. So far.

#llm #ollama

Working on a new feature for #Chit: optional follow-up suggestions! The follow-up query is returning a valid JSON response _most_ of the time...

```
You will be provided with an AI assistant's response. Generate a JSON array of 4 suggested follow-up queries the user might want to query you about it.

For example, follow-up queries might be among:
- "Tell me more about..."
- "What is..."
- "Rewrite that response in a more professional tone..."
- "Why does..."
- "Does it say..."

Only respond with valid JSON in this format: ["suggestion", "suggestion2"]
```

#llm #ollama

Added syntax highlighting and a bunch of various new conveniences to #Chit.

https://github.com/Fortyseven/chit

https://fortyseven.github.io/chit/

#ai #llm #ollama

Givin' shit to books from nearly 50 years ago! TAKE THAT!

(My favorite bit: "Finally, we have the hyperbolic conclusion: “destined to become one of those books…too big to fit on the shelf.” Oh, the irony! This book is probably so thin it could be folded in half and used as a coaster.")

#chit #chitjr #llm #firefox

Mwahaha.

#llm #chit #firefox

#Ollama web swiss army knife extension? Sure, let's do it.

#firefox #llm #chit

Well, that took longer than necessary to implement. But also, pretty easy. :P

#ollama #chit #llm #multimodal #clowns

Updated the README for #Chit, my #Ollama front-end.

Added several features over the last week or so, including custom variable interpolation for system prompts (so you can drop {{date}} in there and it'll always end the current date, etc).

Lots of layout and chat timeline bug fixes, etc. It's now my daily driver for #LLM inference and I'm not _wanting_ for anything.

Except, maybe image pasting for multimodal models. But that's next on the list... 😏

https://github.com/Fortyseven/chit

Added user and system variables for templating the system prompt in #Chit. I really love how this front-end is coming along. I've basically stopped using everything else now, and dog-food it exclusively along with Ollama. (Oh my god, did I actually follow through on a project?!)

https://github.com/Fortyseven/chit

#llm #ollama

I'm messing with trying to refine an SVG generating system prompt among various #LLM models. Success has been mixed -- this kind of thing never goes well.

But this time around, the results have been varied and interesting, at least, if they can't be good.

This time, however, it tried to draw a moon in ORBIT and the fucking thing MOVES. I was not aware you could do this in an SVG.

- See it in action: https://www.svgviewer.dev/s/T6J2yLWH

- The prompt at the time (a WIP): https://gist.github.com/Fortyseven/c4de1caecd005f2559c5bd6597c080a6

- Using the LoneStriker/WestLake-7B-v2-laser model -- this is surprisingly great for a 7B model. https://huggingface.co/LoneStriker/WestLake-7B-v2-laser-GGUF

#llm #svg #chit #ollama

The evergreen statement for most LLM prompts: "it isn't perfect, but it works much better than I'd ever have expected".

I took a subset of the rules from Conventional Commits (something I'm trying to get myself moved over to in my projects, personal and for work) and started feeding it diffs. Not terrible.

#llm #ollama #chit

Oh no, theming! 😔

#chit #svelte #llm

Smells like freedom.

#llm #ollama #chit

So, I'm actually pleased as tits with how #Chit (the single page app #LLM front-end for #Ollama that I'm working on) is coming along. Just added streaming responses last night and killed some big bugs.

Worth reminding that there's already a perfectly fine web front-end with OpenWebUI.

I decided to do my own for four reasons:

- Educational value (I'm working with Svelte at work, so this keeps me sharp)

- This would help me learn a bit more about LLMs and running inference against them.

- I don't hate ChatGPT's interface but I want something more... tactile. Along with other opinionated choices. OWUI emulates the feel.

- I hated that OWUI requires a backend. I know why it's there, but I figured localStorage could handle most of this.

Well, I was right. Except for a couple negatives:

[cont'd]

See, we could spice up all news everywhere with one simple prompt.

#llm #chit #ollama

Pushed up a whole bunch of updates to #chit last night.
- Edit chat responses
- System prompts/templates
- Rudimentary system prompt manager
- Markdown styling
- Tons of bug fixes

Still very much a WIP, but man, it's truckin'. ;)

------------

Chit is a fully client-side-only browser-based front end for #ollama: https://github.com/Fortyseven/chit

Build it yourself, or use it over here: https://fortyseven.github.io/chit/

#llm #llms

Planning an event. Shh! 🤫

#llm #wonka #glasgow #chit #ollama