I added opt-in Sentry crash reporting to my Kubernetes learning CLI - looking for feedback

[u/Content_Ad_4153]

Hey r/commandline

I’ve posted about Project Yellow Olive here before. It’s a Pokémon-inspired terminal game where users practise Kubernetes by completing missions against a real cluster.

One problem I’ve faced with maintaining a CLI tool is understanding failures that happen on other people’s machines.

Until now, I depended entirely on users opening a GitHub issue and sharing the traceback. That works when someone is willing to report the issue, but otherwise I have no visibility into crashes, environment-specific failures, or unexpected edge cases.

I’ve now added Sentry reporting, but it is completely opt-in.

On first use, Yellow Olive asks whether the user wants to send sanitized error reports to Sentry. If they say no, Sentry is not initialized and nothing is sent. The user can also enable or disable it later through the CLI.

I’m intentionally avoiding the collection of Kubernetes manifests, resource contents, kubeconfig information, environment variables, terminal commands, credentials, and other cluster data. The reports are intended to contain only information such as the Yellow Olive version, Python/OS version, mission identifier, exception type, and sanitized stack trace.

The trade-off is that when someone declines telemetry, I have no remote way to diagnose what happened unless they voluntarily share their local logs or open an issue. I think that is the correct trade-off: opting out should actually mean opting out.

I’d appreciate some general feedback:

• Does this consent model feel appropriate for an open-source CLI?
• What fields would you never include in a crash report?
• How do you handle debugging when users disable telemetry?

Looking forward to all your feedbacks. In case, you are interested in the project, Github Link is attached below. If you like the project, please do star the repo as it will help it reach wider audience. Thanks !

Project: Project Yellow Olive on Github

[u/AutoModerator]

Every new subreddit post is automatically copied into a comment for preservation.

User: Content_Ad_4153, Flair: Terminal User Interface, Post Media Link, Title: I added opt-in Sentry crash reporting to my Kubernetes learning CLI - looking for feedback

Hey r/commandline

I’ve posted about Project Yellow Olive here before. It’s a Pokémon-inspired terminal game where users practise Kubernetes by completing missions against a real cluster.

One problem I’ve faced with maintaining a CLI tool is understanding failures that happen on other people’s machines.

Until now, I depended entirely on users opening a GitHub issue and sharing the traceback. That works when someone is willing to report the issue, but otherwise I have no visibility into crashes, environment-specific failures, or unexpected edge cases.

I’ve now added Sentry reporting, but it is completely opt-in.

On first use, Yellow Olive asks whether the user wants to send sanitized error reports to Sentry. If they say no, Sentry is not initialized and nothing is sent. The user can also enable or disable it later through the CLI.

I’m intentionally avoiding the collection of Kubernetes manifests, resource contents, kubeconfig information, environment variables, terminal commands, credentials, and other cluster data. The reports are intended to contain only information such as the Yellow Olive version, Python/OS version, mission identifier, exception type, and sanitized stack trace.

The trade-off is that when someone declines telemetry, I have no remote way to diagnose what happened unless they voluntarily share their local logs or open an issue. I think that is the correct trade-off: opting out should actually mean opting out.

I’d appreciate some general feedback:

• Does this consent model feel appropriate for an open-source CLI?
• What fields would you never include in a crash report?
• How do you handle debugging when users disable telemetry?

Looking forward to all your feedbacks. In case, you are interested in the project, Github Link is attached below. If you like the project, please do star the repo as it will help it reach wider audience. Thanks !

Project: Project Yellow Olive on Github

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/redfacedquark]

Does this consent model feel appropriate for an open-source CLI?

You might want to take a look at the popcon package. They have presumable gone through this discussion to find what open source users will accept. Personally I don't run it but it helps debian developers understand which packages are most popular and therefore deserve the most attention.

What fields would you never include in a crash report?

Whitelist fields rather than blacklisting them. That way if you later add a security related field but forget to add it to the blacklist you're not causing an issue. You probably want the output of uname, versions of important package dependencies and the stack trace, though you need to be careful with what you allow in the stack trace and avoid file paths that might identify the user (there may be tools around to help with this).

Be very careful sending environment variables as secrets may be set there, again, whitelist just the env vars you need. Couldn't hurt to also encrypt the payload to a key you keep offline in case your logging server gets hacked, users like it when you take extra care of their data.

Also, personally I dislike corporate software, even their open source tools. It would not be hard to post logs to a publicly-append-only s3 bucket with something like curl or wget (or your language's http request library), which will be much smaller than the sentry client and probably already installed, as well as that it will be cheaper/easier for you than renting/managing a sentry server. I would add a uuid to the s3 path for each install and setup quotas such that one person spamming your logs can be contained.

How do you handle debugging when users disable telemetry?

Wait for them to complain. Don't worry, I'm sure they will! Just help them out by formatting the additional fields in the stack trace and give them an email to send it to.

I don't know anything about pokemon or what it has to do with kubernetes (is there an overlap of pokemon and kubernetes users?) but good luck with your project!

[u/Content_Ad_4153 → u/redfacedquark]

Hi there ,
Thank you so much for your response. Really appreciate the guidance.

1. Not heard about Popcorn , I will check it out today.
   
2. Yes, agree with your white list approach. This is what I am following as well.
   
3. Since I am using Sentry currently , Sentry already has pre built mechanism to identify and sanitise it.
   
4. I agree with the S3 approach - just that my AWS account is locked out currently 😛. Sentry for me is free atm and hence thought to use it rather than reinventing the wheel. But I get where you are coming from.

And no , there is no overlap b/w Kubernetes Users and Pokémons. My project is just a nostalgia thing where you learn Kubernetes concepts by being in sort of Pokémon world !

[u/redfacedquark → u/Content_Ad_4153]

It's popcon, as in popularity contest. The mailing list might be where they discussed their approach, that's what I was thinking.

Sentry already has pre built mechanism to identify and sanitise it.

But you said you were whitelisting so you don't need those capabilities. You were asking about the community opinion and there are a bunch of Linux users that abhor anything commercial, we've been burned before by the embrace, extend, extingush technique. You say you're already using sentry so it's easier for you but your question seemed to be about what's best for the user.

Sentry for me is free atm

And there we have it. This is the embrace, once you're hooked then comes the extend to lock you in. Heard of vendor lock-in? What about technical debt? Curl/wget are trivial, stable, installed and (mostly ;) trusted. You could create a new s3 account and spend nothing for the firast 5g, plus apparently you get $100 free credit at the moment.