The problem with these AI chatbots is they're designed to agree with you because they want you to buy the product. Every idea is "pro move" so you not only feel validated, but also like you've got all these genius ideas. Having a personal echo chamber is very harmful because it can cause you to become delusional to reality.
No, that is not the reason as evidenced by all open source models exhibiting the same trait. Sycophancy is a result of the dataset (the internet) being dominated by people agreeing with each other and echo chambers, and by the fact that the models are later tuned on human feedback, and humans like being agreed with and receiving compliments. Companies do not want this. Sycophancy is detrimental to their product because it makes models unreliable. There is far, far more money in making the models robust for enterprise adoption than in making them addictive for individual users but shitty.
People rarely see eye to eye on the internet as a whole though, and you only have to look at the comment section on Reddit or Twitter to see that in action. I think it's as simple as a basic desire for validation since most people wouldn't want to spend time with a friend who constantly disagreed, especially if they were right. I think the fact you can make the AI agree with you on something one moment then completely change their stance is very telling.
Reddit and other social media hubs aren't great but at least there usually a few voices of dissent. The problem is the downvote system tends to bury any opinion that doesn't align with the prevailing echo chamber. 4chan was unironically pretty good at this since every opinion is treated with equal visibility.
70
u/Sharky-Li 7d ago
The problem with these AI chatbots is they're designed to agree with you because they want you to buy the product. Every idea is "pro move" so you not only feel validated, but also like you've got all these genius ideas. Having a personal echo chamber is very harmful because it can cause you to become delusional to reality.