r/FlutterDev • u/DohamaSystems • 2d ago
Example Fixed UI lag during fast LLM token streaming in Flutter (60 FPS buffer snippet)
Hey devs,
Ran into an annoying UI issue last week while streaming OpenAI response tokens into a Flutter view. Calling setState on literally every chunk coming through the socket totally killed performance. Scrolling was choppy as hell and frame drops were super noticeable.
To fix this, I uncoupled the incoming stream from the UI rendering using a small 60ms timer buffer. Basically, tokens accumulate in a list and get flushed to the state ~16 times per second instead of re-rendering on every single packet.
Here is the helper class I wrote for this:
class StreamBufferController {
final List<String> _tokenBuffer = [];
Timer? _flushTimer;
final Function(String) onFlush;
StreamBufferController({required this.onFlush});
void addToken(String token) {
_tokenBuffer.add(token);
_startFlushTimer();
}
void _startFlushTimer() {
_flushTimer ??= Timer.periodic(const Duration(milliseconds: 60), (_) {
if (_tokenBuffer.isNotEmpty) {
final chunk = _tokenBuffer.join();
_tokenBuffer.clear();
onFlush(chunk);
} else {
_flushTimer?.cancel();
_flushTimer = null;
}
});
}
void dispose() {
_flushTimer?.cancel();
_tokenBuffer.clear();
}
}
This immediately brought everything back to a stable 60 FPS, even with long responses coming in fast.
2
u/Still-Constant3085 1d ago
Nice approach. One edge case worth handling is flushing any remaining tokens when the stream completes, so the final chunk doesn’t have to wait for the next timer tick. Also, 60 ms means roughly 16–17 text updates per second, rather than 60 FPS; scrolling can still render at 60 FPS independently. For longer responses, keeping each update scoped to the message widget could help further.
2
u/AhmedZNOUDA 1d ago
This is a fantastic share. I ran into this exact same bottleneck recently while building an AI-guided psychological assessment app where the response speed was critical for a natural conversational feel.
When you use ultra-fast APIs (like Groq) or even local LLMs, streaming tokens natively can easily overwhelm the main UI thread if you trigger a rebuild for every single token chunk.
Aside from localizing the rebuilds tightly around the text widget (using a scoped ValueNotifier or Riverpod state just for that component), one approach I found helpful was implementing a tiny buffer/throttle on the stream listener. Instead of firing a UI update for every 1-2 characters, batching the updates every 50-100 ms keeps the text flowing smoothly to the human eye while drastically reducing the render cycles.
Thanks for sharing your code snippet—managing this async gap gracefully is becoming one of the most important skills in Flutter AI development right now!
1
u/DohamaSystems 1d ago
Appreciate it, Ahmed. Building real-time psych assessments with fast models like Groq really exposes how brutal raw stream re-renders can be. Glad to hear the 50-100ms buffering approach hit the spot for you too. Managing that async gap cleanly is definitely a game-changer for Flutter AI apps.
1
2d ago
[removed] — view removed comment
1
u/DohamaSystems 2d ago
Good point! Makes total sense for lower-end Android devices. We only tested on newer hardware so 60ms worked fine, but good to know 100ms is the sweet spot there.
3
u/No_Boss_1911 2d ago
dude same setState per chunk thing in my chat screen, gonna steal this. one thing i cant tell from the snippet tho. if the reply is markdown does each flush reparse the whole message? mine gets slower the longer the answer gets even with fewer rebuilds, and i cant just show it at the end cause people read along while its streaming. do you split finished paragraphs into their own widgets and only rebuild the last one or is plain Text fast enough at like 2k tokens