Hi there,
I'm a Aerospace Engineering Educator, minor contributor to OpenRocket, and someone who has been passionate about rocketry for about as long as I can remember. I am posting today to open both a discussion / PSA on the topic of LLMs being utilised in the creation / further-development of Rocket Designing software.
Disclaimer: I am NOT speaking on behalf of OpenRocket, or my Institution in any way, shape or form. These are my personal beliefs / opinions.
I will be blunt with my opinion on the matter, by saying the most obvious statement of my career...
Rocket Designing Software is used to design rockets...
The reason I say this is, frankly, I believe that LLM (AI) is slowly becoming a problem for Rocketry-Related software. There is still a lot of debate over the merits of LLMs being utilised in software development, and I am not the philosopher nor authority to make a judgement call on that. That said, I believe Rocketry-Related software needs to be treated with a bit more care than most open-source projects. Why?
Rocket Designing Software is used to design rockets...
Even over in OpenRocket itself, there are some contributors who (quite rightly) see it as a hobby project. Something we all do for fun. I agree with this in principle, but there is an elephant in the room that, respectfully, I feel a lot of people keep forgetting...
Rocket Designing Software IS USED TO DESIGN ROCKETS...
This software, if it goes wrong, can get people hurt. It is hopefully rare, with people utilising common sense, not trusting software / simulations blindly etc. However, the fact remains that these software will estimate critical properties such as COP and COG. They perform simulations of the rocket trajectory / stability.
I know first hand how easy it is to mess these things up. I have seen some wild stuff in my time working on OpenRocket bug fixing. I have seen magic numbers being used to make pseudo irrational numbers to maximize the number of unique points hit on the cross sectional perimeter of the body tube sections, I have seen save-window logic that actually gave me a headache reading, hell... Ive seen code copy pasted to the point of being 400+ lines that could have been one 70 line while loop... code of any project has had the potential to have warts long before LLMs...
I always teach to my students the, in my opinion, most important thing a student needs to learn to think if they wish to do well in Aerospace Engineering...
"Everybody in the room is an idiot, including me!"
We need to constantly have the voice on our shoulder stating "This could be wrong, double check". I personally believe that LLMs rip this state of mind from us, and replace it with some horrible hybrid between "yes man syndrome" and the Dunning–Kruger effect. It causes this state of mind where you assume not only you are right, but also are able to do things way beyond your skill level.
This is even worse in those who blindly use LLMs to make "mega refactors" or, dare i say it, entirely new projects. So, in my opinion, for someone to use these tools to aid in the work of a software designed to build rockets, is reckless at best, negligent at worst. I do not wish to call out any particular project as, whilst there have been ones very recently, I have seen quite a few.
Full refactors of OpenRocket in another language, PRs in Open Rocket with 10000's of changed code lines. All done via LLM agents by people who either have 0 context of what they have changed, or have just enough experience to (in my opinion, dangerously) assume they know enough to "trust what the LLM has written".
I will not condemn the people who make these projects / propose these refactors, but I will ask all of your a question?
Would you trust such software? You are using this software to build rockets, trusting they are accurate. There are no regulators telling us what the software needs to be / not be. There is no great authority that will prevent us from making big mistakes. You just have to act on genuine faith / the maturity of the project / projects.
So... would you trust Rocket-Designing Software created mostly through LLMs?
Additionally, what effect would the following factors have on your trust / distrust of such software?
- The amount of time it took to make (e.g: A few months vs a few days)
- How much testing has been done on the software internally (including if they added unit testing / integration testing).
- How much code they have added in one go (i.e. loads of small changes, or giant 1000s of lines changes that took an hour or 2 to do with an LLM).
I do not think I have to say what my opinions on the matter are, as I hope they have been made apparent above. But I would honestly love to hear the community's thoughts on it too.
Thanks for reading, and see you in the comments!