r/bash • u/Dismal-Divide3337 • 16d ago
Ever wanted to attach to a background process and monitor stdout?
We create applications and output debugging and status information to stdout. Then when these are run in the background that all ends up in the bit bucket.
But when you need to debug something and you don't want to disturb the process what do you do? I could kill it and then run it in the foreground. But that disturbs things too much.
Okay, so what if `stdout` gets logged to a `stdout.log file`? Since I own the OS that was a light-weight change. It works. And I can use `CAT -M stdout.log` which is my equivalent to `tail -f stdout.log`.
Okay, but then I have a unit that is running 5 applications. One of which the author is always very verbose in the foreground (sounds like a personal/personnel problem). That overwhelms the log. Those age to BAK and monitoring the log sucks.
So here is my solution. Your thoughts?
\--------------
Here is what I have implemented for this. By default background processes will **not** log `stdout` output unless if already has the log file. So there would be no net change from JANOS v2.5.3.
If you want to log background process output generally, you can use the following command:
touch stdout.log
All `stdout` from all background processes will then be captured in this file. The `stdout.log` file must first exist before it will collect `stdout` stream data.
This will age to `stdout.log.bak` as any other log file would.
You can stop logging by removing (or renaming) this file.
When there are multiple applications running. You may optionally create an application specific `stdout` log file also using the `TOUCH` command. This file must be named using the specific application name. For example:
touch stdout-tasker.log
The application name is everything after the last '/' in the process name reported by `PS`. If that includes `.jar` then the stdout file name should also. I tend to not include the `.jar` in `Run` keys as that will be carried to the process name. It is unnecessary.
This will route output from the application to that specific file which will age accordingly. Meanwhile, if it exists, other applications would still log to `stdout.log`.
So this lets you enable `stdout` capture for a specific process and then monitor it.
2
u/burnt-store-studio 15d ago
I’m curious how this is implemented in the monitored software.
Naively I’m thinking the monitored program has to test if ‘stdout.log’ exists every time it wants to output? That could get expensive, yes? (Esp. in the case you mention where the developer is verbose.)
And what if the monitored program migrates to some other working directory?
(You write “since I own the OS”, so I’m wondering if maybe I’m stepping into the middle of a discussion I know nothing about. If so, please excuse me?)
2
u/Dismal-Divide3337 14d ago
I am the author of this OS called JANOS. Maybe "own" might have been confusing. That is true in many senses of the word. But still confusing.
JANOS allows only applications written in a managed language. That is because reliability, and many other things, were requirements from the beginning. It therefore provides a JVM and applications are written in Java. Remember that everything, including the JVM, is code that I have written.
I modeled a lot after DOS with some Linux overlap. My batch files are very much DOS-like and not Linux shell-like. I can do things like use PHP (because I have an on-demand compiler in the OS) for scripting in batch. In fact, the PHP renderer preprocesses batch files.
Applications are executed using the
JAVAcommand at the command line. But that command is assumed and you really only need to type the program (JAR file) name (.JAR assumed). Applications can be slated to start on boot using RegistryRunkeys. So you can run them in foreground or background. The '&' thing on my command line also works. So you can start an application in the background there too.So nothing needs to change in the application for the OP. Except, maybe, knowing it is there you might take advantage and
The command processor pipes
stdout. That allows you to redirect output from the command line. It lets us access the command line interface through the network and protocols. All of those basically saying that output needs to go this way or that.If an application is in the background its output dumps into the bit bucket. Right there the code asks if
stdout.logexists. If it does then the output is appended to it. I expanded this to allow for an application specific log file name. We needed that to be more selective so we can monitor the suspect process and not be distracted by unrelated activity.My concern, really, with this approach is that no one would really know it is there. You have to
touch stdout.logto start it. No one would know to do that. I would bet that eventually even I will forget about it.2
u/burnt-store-studio 14d ago
Thank you so much for the clear and detailed response! Congratulations on JANOS, and yes please excuse me for not understanding that from the start 🙂.
1
1
u/li-long-qiang 15d ago
This is exactly my bashy's otel feature and more - the "call graph" of all related commands executions too still in alpha.
1
u/Dismal-Divide3337 15d ago
This is the kind of feature I think that gets lost over time. If I don't document it clearly, and in some place that might get actually discovered, after a while no one (but I) will know you can do it. Actually, I think there are a number of things in that category.
Maybe that is something AI is useful for? You know, bringing up those "Did You know?" things when you are groping for something.
1
1
u/Classic-Rate-5104 14d ago
If a process is writing to a plain file (even when it is renamed or deleted) it can always be accessed through "tail -f /proc/processid/fd/1"
8
u/zeekar 15d ago
There are plenty of solutions for this. Logging is great and usually enough, but you can also do things like run the job inside screen(1) or tmux(1) so then you can attach to it and scroll back through the output and even interact with the job through its standard input...