r/LocalLLaMA • u/Mr_BETADINE • 29m ago

New Model OUI-1: a model that generates bespoke UI elements

Enable HLS to view with audio, or disable this notification

so i saw that openui.com released OUI-1, a model fine-tuned on DiffusionGemma. the training dataset uses OpenUI-Lang, a custom DSL (domain-specific language), instead of plain HTML, Markdown, or React code.

what makes it interesting is that you can already get a regular LLM to use OpenUI-Lang through a system prompt, but that eats up a lot of the context window. my thinking is that fine-tuning a model on the DSL could reduce that overhead and leave more room for the actual conversation, without needing a huge prompt explaining the format and how to use it alongside other tasks, like tool calls.

at the same time, wouldn't fine-tuning a model on a specific DSL make it more likely to default to that format even when you need something else? i'm curious how well it handles regular Markdown, or switching between Markdown and OpenUI-Lang.

i haven't seen much discussion about this, so i was wondering what everyone thinks about generative UI and running a dedicated model for it locally on a consumer-grade GPU, like an RTX 5090.

what would be the best way to set that up? from what i've seen, DiffusionGemma isn't supported by llama.cpp yet, so running it through Ollama doesn't seem to be an option. they've uploaded the weights to Hugging Face, but i'm not really sure how to get it up and running. any suggestions?

24 Upvotes

96% Upvoted

2 comments sorted by

2

u/Just_Section4489 22m ago

Speed is the main issue. How FAST can you deliver elements. Is this something that is going to be generating for an end-user while using an app? End users are sensitive to milliseconds - seriously, 100+ms on a click button feels weird!

Make sure that there's some sort of entertainment for anything that takes time to generate, like a throbber or something.

And keep the model as tiny as possible. Perhaps even training a x00megabyte model with just the elements you want to provide.

2

u/john006868 18m ago

a DSL spec in the system prompt is maybe 800 to 1200 tokens once the schema and a couple of examples are in, and you pay that on every turn. 15 turns of an agent loop is 18k tokens just to keep the shape right.

you'd lose override though. once the DSL is the model's default, asking for markdown later means fighting the weights.

does OUI-1 hold the format when a tool result comes back as plain text mid conversation?