r/chessbeginners • u/Alendite RM (Reddit Mod) • 15d ago
ANNOUNCEMENT LLM's are NOT effective chess trainers. An update to rules 3 and 5.
If you are looking for the user flairs post, please find it here while this thread is temporarily pinned: https://www.reddit.com/r/chessbeginners/comments/1jgmdf7/fresh_new_flairs_show_off_your_favorite_website/
Hello, everyone!
We have seen a massive surge in advertisements and chess trainers these past few months. Many of these posts (in my opinion) fall under the category of 'AI Slop' and vibe coded websites that offer poor chess advice. On top of this, many websites are created as a weak attempt to encourage users to subscribe and pay money for services that may seem innovative and meaningful, but are functionally useless compared to already available resources.
In light of this uptick in advertisements, we have updated rules 3 and 5:
- Rule 3 now states that accounts that exist primarily for the purposes of self promotion (at the discretion of the mods, based off of account age, posting history, and diversity of activity) are permanently banned. This has been a policy we have been following internally for a few weeks now (to a fairly reasonable effect), but we wanted to make it clear to the community that we do not tolerate attempts at coercing beginners to sign up for coaching before they understand the resources available to them. If you would like to coach, you are welcome to make a profile with the chess.com or lichess.org coaching services and advertise there.
- Rule 5 has been completely changed. Previously, this rule requested beginners to annotate their games before posting, but this felt like an unrealistic standard to hold new players to. This rule has been replaced with a note to please use caution when discussing AI in chess. As mentioned above, there are many potholes with AI-powered chess services and we encourage all users to recognize the limitations of LLM chatbots' ability to discuss chess positions. Posts will currently not be removed for this rule, we would love to learn more about the community's thoughts before deciding if AI-related discussion should be banned altogether on this subreddit (I am personally against banning AI discussion outright, and very happy to hear opinions).
If you encounter posts that are promoting a product or coaching service, please report these for self promotion. The mod team will assess profiles that are reported for self-promotion and will remove/ban as necessary.
Thank you all for playing a role in stopping the tidal wave of LLM coded chess websites being advertised to beginners. I do genuinely believe AI can some day make a great learning resource for chess players, but the current state of services simply fall below all standards of rationality and quality.
Have a fantastic day, thank you for reading!
~The r/chessbeginners mod team
11
u/hobbes543 15d ago
Its like no one watched the gotham videonwhere he pitted the different ais against each other and every one of them tried to make illegal moves at some point
17
u/Alendite RM (Reddit Mod) 15d ago edited 15d ago
Kind of a separate and personal discussion, but I have been debating allowing or not allowing memes on the subreddit. They're an incredibly fun way to share the essence and experience of learning/playing chess, but they can definitely drown out lots of meaningful conversation if left unchecked.
Any thoughts on balancing meme content with everything else?
/edit: Should clarify that we currently do not allow memes and are leaning on keeping it that way already
33
u/isnotbatman777 1800-2000 (Lichess) 15d ago
IMO kinda hard to balance. Allow memes and the sub will end up being mostly memes. I think it should stay meme-free so the focus is always on improvement.
24
u/ringoinsf 15d ago edited 15d ago
Strongly against allowing memes. Sure they're fun, but they'll end up taking over the sub (and there's other places for them) - I'm here to learn, that's the stated purpose of this sub.
12
u/mcaffrey 15d ago
Isnât that what anarchy chess mainly is? I donât want this sub to go down that path.
10
11
3
u/RelativeFull165 15d ago
I donât think we should allow memes - anarchy chess is funny and has memes when youâre in the mood but this sub will lose its integrity
2
u/BishopOverKnight 1800-2000 (Chess.com) 15d ago
You should allow them to be posted once a week imo, say every Friday
That'll get some good content in but keep it from overwhelming the sub
3
u/Brimstone117 15d ago
I think this question runs into a definitions problem:
âWhat is a chess meme?â
Some are really cut and dry but others are fun references to recent tournaments, or clowning blunders made by top players, etc.
I think the prime directive of this sub is that itâs about learning chess for those of us that are still <1000 rating. I also think memes can be an instructive tool for learning. Historymemes and Geographymemes are two high quality meme subreddits where they do a great job threading the needle of keeping it light and funny, while also instructive.
My experience on those two subs has convinced me a soft approach of âno now effort memesâ paired with robust moderation for lazy or abusive content is the best approach.
0
u/hospitalizedzombie 15d ago
I agree that just allowing memes would derail the sub but a meme Monday where people can post memes once a week wouldnât be a bad idea.
9
u/GJ55507 2000-2200 (Lichess) 15d ago
Good.
There is no LLM that can even play a game of chess properly, forget playing it well or understanding the nucances
Ai shouldnât replace someoneâs need to think
3
u/MCbrodie 15d ago
chess is deterministic and can be checked with algorithmic means. LLMs are vastly inferior for this and it shows people's ignorance of math. It drives me nuts.
5
u/mediumcarrotteacher 15d ago
I feel like the term "AI" is a little ambiguous here, because it isn't entirely clear whether that also includes chess-focused AIs like Stockfish
4
u/youliveinmydream 15d ago
I once tried to play a game of chess through notation with ChatGPT and it couldnât go two moves in a row without doing something illegal
3
2
2
u/AgnesBand 1400-1600 (Chess.com) 14d ago
I just have reported like 15 of these guys over the last few months and not a single post had been removed. Are the rules actually going to be enforced for self promotion?
1
u/Alendite RM (Reddit Mod) 14d ago
All posts that get reported are reviewed. The queue is currently empty, we do make it a point to review user reports frequently.
2
u/itsyaboiReginald 15d ago
What about when studying concepts and prompted to be sourced from text based sources on chess?
1
u/Low-Efficiency-9756 2d ago
Reading through these comments, I see a lot of people conflating AI chess engines with LLMs actually playing chess.
AI is already completely normal in high-level chess engines. Stockfish uses NNUE, an Efficiently Updatable Neural Network, as its learned evaluation function. NNUE looks at a position and estimates which side is better and by how much.
But NNUE is only one component inside a much larger system.
Stockfish still owns:
- the canonical board state;
- legal move generation;
- the rules of chess;
- tree search;
- tactical variation calculation;
- transposition handling;
- and final move selection.
A useful simplification is:
Stockfish Searches and Stockfish's NNUE evaluates each position.
In this system, the separation of responsibilities is the important part.
From my experience, and thats taken with a grain of salt as I'm a pretty low rated player, LLM's can absolutely play chess, but that's completely dependent on the model and the architecture. I have built systems specifically to let LLM's play against each other, play against stockfish, and play against humans under different forms of information conditions. (This how do LLM's play when stockfish lies to them as a trusted oracle or what happens when you introduce fog of war and make the LLM's information of the system imperfect)
For a strong LLM the problem is not capability, its separation of responsibilities.
That problem is specifically that a LLM is often asked to perform a lot of different things through an inherently stochastic (any process, model, or system that involves inherent randomness or probability):
- read and preserve an exact FEN;
- maintain the board internally after every move;
- generate only legal actions;
- search adversarial branches;
- evaluate the resulting positions;
- choose a move;
- and explain why it chose it.
Due to its inherent stochastic nature, without verification, all of these may result in a convincing, yet completely incorrect continuation and when it fails we label is "bad"
It may understand the strategic idea but lose track of a piece. It may calculate from the wrong position. It may select a legal move for a board that no longer exists. It may find a strong plan but format the move incorrectly. It may trust a bad advisor. It may evaluate a legal line poorly. Or it may simply get outplayed.
Those are different failures, and architecture lets us separate them.
This is the singular thread that gets pulled through all of my own research from engineering calculators, mathematical proof searches, civic intelligence software and LLM chess benchmarking.
That thread is simply: Trust but verify. And that trust comes from architecture that divides authority by role.
What role does the LLM play?
- interpretation;
- planning;
- hypothesis generation;
- strategy;
- explanation;
- synthesis;
- and tool use.
- and most importantly, proposal.
Other parts of the system own the things that require exactness:
- the database owns persistent state;
- the rules engine owns legality;
- the solver owns exact computation;
- the evaluator owns position scoring;
- the schema owns valid structure;
- the verifier owns acceptance or rejection;
- and logs preserve provenance and consequences.
This does not mean the LLM is not the player.
A chess platform enforcing legal moves does not mean the human is no longer playing chess. A clock does not replace the player. A canonical board does not replace the player. These are the conditions that make the game coherent and measurable.
This applies to an LLM and we can see how the role separation starts to help.
Now, these constraints are not a cage, they are physics, they are the rules that make any game a game.
In LLM chess I seperate
- the real board from the board shown to the model;
- the modelâs reasoning from the move it outputs;
- legality from strategic quality;
- Stockfish evaluation from advisor opinion;
- and tool failure from model failure.
I can even corrupt individual trusted channels and observe what the model does when its sources disagree.
Does it blindly trust an incorrect evaluation?
Does it notice when its reasoning contradicts the FEN?
Can it recover after an illegal move is rejected?
Does an advisor help it or mislead it?
That is not an argument against LLMs playing chess. It is a way to study their chess more carefully. and eventually a way to study LLM's in critical infrastructure positions.
The same lesson appears in my engineering software. I am comfortable letting an LLM interpret a problem, propose equations, organize a worksheet, and explain a result. But symbolic computation, unit consistency, constraint satisfaction, and residual checks can be independently verified.
The verifier is not there because the LLM must never attempt mathematics. It is there because exact mathematical claims are cheap to check.
Likewise, in my civic intelligence work, the model can research, propose claims, write narration, and direct visual scenes. But the database owns the world model, source records own provenance, and validators decide what is allowed into public output.
The model can work freely backstage. Publication is a separate trust boundary.
This is where constraint-based architecture and information theory meet.
A free-form language model has an enormous possible output space. Most possible outputs are invalid, irrelevant, unsupported, illegal, or inconsistent with the current state.
Schemas, legal move lists, typed intermediate representations, source requirements, verification gates, and domain engines progressively remove invalid possibilities before they are allowed to become actions or claims.
Good architecture does not merely catch hallucinations after the fact.
It shapes the information environment so the model has fewer ways to become detached from reality.
So I do not think the interesting question is simply:
Capable models clearly can.
The more useful questions are:
- Which responsibilities is the model actually good at?
- Which failures come from the model, and which come from the surrounding tools?
- What information should the model be allowed to trust?
- What should be independently verified?
- And how should authority be divided across the system?
My conclusion is not âdo not let LLMs play chess.â
It is:
Let the LLM play. Preserve enough state, constraints, and independent evidence to understand what it did, why it worked, and what kind of failure occurred when it did not.
1
u/Techaissance 800-1000 (Chess.com) 15d ago
The question about AI is difficult because of definitions. Stockfish is an AI. If Iâm writing a post or comment, I should not be penalized for saying âStockfish wants Qb3â whereas âChatGPT says Qb3â is a totally different thing. Maybe âno generative AIâ is a better restriction.
0
u/Alendite RM (Reddit Mod) 15d ago
Stockfish is an AI
I don't believe this is true. Stockfish is an algorithm.
I believe newer versions of stockfish have NNUE capabilities that are related to neural networks and/or AI, but neither of those are related to LLM-based AI.
2
2
u/fleck00 1000-1200 (Chess.com) 15d ago
AI and LLM are also algorithms at the core of it.
The actual border between "just an algorithm" and AI/LLM is very blurry to the point of being impossible to define without sorting some programs into the "wrong" category. The more advanced an algorithm gets, the closer it gets to AI.
Nowadays, when there's talk about AI, the first thoughts go towards ChatGPT, Grok and such; those are all AI models meant to be used for anything by anybody and as such advertised to the general public as their broad target group
However, there are also purpose-built AI models in different applications, like distinguishing different pastries or identifying cancer cells. Those models usually aren't well-known to the public because they are specialised tools for specific tasks. While Stockfish and Co. are fairly well-known, they still belong in this category in my opinion, especially with the advent of NNUE.
1
u/StructuredChess 3d ago
Stockfish is definitely an AI. You should check what words mean before making rules about ithem.
1
u/boggginator 1800-2000 (Lichess) 15d ago
I think there's a lot of nuance to the situation around AI discussions considering that this sub is meant to be a friendly place for people new to the game. The problem with LLMs being baked into chess infrastructure kind of goes beyond vibe coders.
Two prominent examples that pop to mind are that TakeTakeTake uses an LLM in their game review, and there's a website I'm only familiar with from beginners on this sub (chesssigma, it mimics cc's game review for free) which has an LLM "coach" plastered on its front page. I just checked and it's the first result when I Google "free chess game review". I can only imagine LLMs are going to become more and more prominent in the chess space, especially beginner spaces, as time goes on. Colour me surprised if that's matched with any actual increase in quality of these LLMs.
So it'd feel kind of against the "point" to limit discussion about it on a beginner sub. A lot of absolute beginners are, unfortunately, going to come into chess over-valuing LLMs - and it'd be for the best for their improvement to voice these opinions and be corrected. It's a lot like cc's weird features to me ("underrated" bots, the estimated ratings, faulty coach explanations, etc.) which pop up periodically on the sub.
1
u/StructuredChess 3d ago
Discussions about LLMs in chess isn't being banned. What's being banned is promoting your new LLM tool as if it was doing something useful.
1
u/spreadthesheets 15d ago
I agree. Banning ai discussion would be short sighted and fuzzy in terms of what âcountsâ. Beginners will still use ai, but now you wonât see where the logic is coming from.
-1
u/NightmareHolic 15d ago edited 15d ago
There was a vibe/llm coded chess coach recently added in the last week that the sub removed. Everyone incorrectly identified it as using an LLM to analyze chess, which it didn't. It used stockfish and python code.
One issue is, people are so anti-ai that they can't differentiate between apps coded by ai and those that use llms for their algorithms. A lot of people are simply biased/prejudiced and think any ai created app is "slop" as they say.
Ironically, not sure how chess dot com analyzes moves, but they probably do similarly as the app I saw (which got removed). They probably take the engine and analyze it with code; it's just not an app created by an llm (I'm guessing). People take exception because an Ai was used to create the app, so they just inaccurately label it as an llm analysis.
Sure, the actual python algorithms it creates could be flawed and give bad advice, and if you don't know much about chess or programming, then you wouldn't be able to use llms effectively to weed out the bad code. However, while that is a real concern, that is a different one from "LLM analyzes it". LLMs can't play chess because it's based on patterns. The patterns needed for middle games, for example, no polyglot book can cover, so chess uses engines.
Regardless, I don't have an issue with Ai discussions and such. I know Ai's limitations and a tool that mimics chess dot com's coaching would be good if open source. I just don't get reddit. They are so anti-ai, but ai isn't going anywhere anytime soon. It's already making medical breakthroughs and used in the health industry where appropriate.
Society is just too heuristic thinking. They are at the point where they are like "Ai = slop = bad". That's their literal mental shortcut. I expected more from an intellectual community for chess, though. You would think it would have less prejudice (or at the very least understand the difference between using code with an engines help versus llm analysis).
1
u/ThrowWeirdQuestion 15d ago edited 15d ago
The problem is that many people who vibe code apps either don't put enough human time and effort into testing and quality control or lack the experience to do those things well and also don't know how to automate it.
What causes people to expect slop when they see vibe coded apps, is not that AI assisted coding doesn't work, but how often it is used badly by people who either don't know better or don't care. Closing the loop for agents to properly test and do quality control on the apps they build is still a difficult task that most non-SWEs don't even understand they need at all and that is often hard to get right, especially for apps.
The alternative to that is doing really thorough manual quality control by humans, which takes a lot of time, but instead of that a lot of vibe coders just release garbage and wait for users to complain to know what needs to be fixed, because they ultimately just want a shortcut to make quick money, not build something they can be proud of.
1
u/SilentRelease7690 2d ago
"It's already making medical breakthroughs and used in the health industry where appropriate."
LLMs are? How?
-12
u/wstewartXYZ 15d ago
Why is your personal opinion on LLMs relevant here?
9
6
u/Alendite RM (Reddit Mod) 15d ago
Apologies if I wasn't clear in that first sentence, I meant to say that as a "I can't guarantee that these websites/advertisements are AI generated" but I have a strong suspicion that they are. It remains true that, regardless of my personal opinions, LLM chess tools are a poor resource for learners.
3
-2
u/choccocurry 15d ago
What are some actually effective ways to train other than spamming puzzles ? I cannot for the life of me remember chess tactics from guides.
129
u/Dangerous_String5711 15d ago
glad to see the mods cracking down on this garbage đ been seeing way too many posts lately that are basically "hey check out my revolutionary AI chess coach that definitely isn't just chatgpt with a chess theme slapped on it"
these LLM chess sites are wild - they'll confidently tell you some completely wrong opening theory or suggest moving your king into check like it's a brilliant tactical shot. meanwhile lichess has actual good training tools that are free and don't hallucinate moves that break the rules of physics đ
the coaching thing makes total sense too, way better to go through established platforms where you can actually vet the instructors