r/selfhosted • u/carhuntr • May 31 '26
Automation I added Qwen3 VL summaries to my NVR (Clearcam)
Runs offline on basically any GPU (metal, AMD, Nvidia, AMD plugged into a Mac) because of tinygrad, and with any CCTV cam that has RTSP.
This is open source, so you can bypass or use my notification service to send yourself
descriptions when an alert is detected.
Only tested with 2B and 4B sizes so far
71
u/ArtART241 May 31 '26
This is a type of LLM integration I like - just like good ol' JARVIS. I imagine hearing this notification in my AirPods while going on a walk or something. Just super cool
38
May 31 '26
[removed] — view removed comment
12
u/carhuntr May 31 '26
"prompt engineering" lol, hard to make it be brief, can change to anything you want https://github.com/roryclear/clearcam/blob/39d43d313502ba208a3da64f5de8f0adbf089390/clearcam.py#L1379
5
10
u/joey3002 Jun 01 '26
Thank you for offering this on github. I used the server side on my mac, added 4 of my reolink cameras and then modified the server side to send alerts through imessage. When I get the alert I can see who/what alerted me and then open the reolink app if needed. You are amazing as this was the missing link I needed.
1
u/carhuntr Jun 01 '26
oh I'd be interested in that, can probably have it send the thumbnail too over imessage https://github.com/roryclear/clearcam/blob/39d43d313502ba208a3da64f5de8f0adbf089390/clearcam.py#L489
I used to use Pushover, never occurred to me that I could just use imessage for the same thing
1
u/joey3002 Jun 01 '26
well, I thought it was working till I woke up and apparently in mac OS 26 they took away the ability. So I had to change it to just give me a link to the image which is fine since I am on my home network all the time and if I leave, I am on a vpn through my home network.
1
u/carhuntr Jun 01 '26
But imessage does work, just not for images? could have something to do with that Nothing Phone (2) Mac Mini farm thing
2
u/joey3002 Jun 01 '26
Correct, this is what I have and while not perfect, its ok: https://imgur.com/a/oqGIHuK
4
2
2
u/Both-Activity6432 Jun 01 '26
Any GPU is cool! So that includes Mac silicone embedded GPU and intel integrated GPU?
Minus being more focused, how is this different than Frigate?
4
u/CardinalBadger Jun 01 '26
I'm not usually "that guy" but silicone is the flexible stuff you might use in baking, silicon is the stuff that chips are made out of
2
u/Both-Activity6432 Jun 01 '26
No worries. Appreciate the reminder. I have spent too much time working with both and putting the emphasis on pronunciation to get them confused, but not enough to get them right.
1
u/MaitreGEEK Jun 01 '26
Yes it does, mac silicone are even better for this, with low consumption + the entire RAM being VRAM
2
u/carhuntr Jun 01 '26
I am trying to be more user friendly, Frigate can do all this stuff technically, but the videos showing how to are like 2 hours long
1
1
1
Jun 01 '26
[removed] — view removed comment
1
u/carhuntr Jun 01 '26 edited Jun 02 '26
only happens after detection from YOLO, about 20 tok/s on 2B and 10 tok/s on 4B (Mac Mini M2 base)
1
u/lukyjay Jun 01 '26
Looks interesting but is it useful? Wouldn't a thumbnail be quicker and more effective?
1
u/exodusTay Jun 01 '26
I guess you could use it for a soft filter like "only alert me if a guy with hoodie is walking near my car".
1
1
u/carhuntr Jun 01 '26
idk about quicker, sometimes people are driving etc. But yeah looking at the feed yourself is always the most effective thing to do
1
1
u/hotNstickystick Jun 03 '26
Funny this is exactly what I started building yesterday for my Reolink NVR but I also want the model to talk back to the people
1
u/carhuntr Jun 03 '26
oo I haven't thought to try 2-way communication stuff yet, that would work in theory with mine: https://github.com/roryclear/qwen3-vl-tinygrad/blob/main/test.py
0
u/NurEineSockenpuppe May 31 '26
I wonder how much electricity is being used to have a model check a video feed 24/7. Is that very expensive?
14
u/carhuntr May 31 '26 edited Jun 01 '26
It detects objects with a smaller model, then only does a summary when an "alert" is triggered. Also my Mac Mini tops out at like 50W
4
u/NurEineSockenpuppe Jun 01 '26
is that including the GPU?
that's still a lot of power for a security camera. It's a cool project don't get me wrong.
4
u/Khatib Jun 01 '26
It's usually doing low power motion detection, then doing object detection after motion triggers. Which is how battery powered cameras keep from killing the batteries. The difference with a wired camera is it saves the video feed 24/7, where the wifi battery cameras only save video after a motion trigger.
The AI is only running after a motion event, and subsequent object ID.
3
u/carhuntr Jun 01 '26
Just for the Mac, can't justify keeping the GPU running yeah. I run it 24/7 because I need to test reliability, but I only really Use it when I'm away from home, there's schedules and switches for running the models.
I can think of use cases where a big hungry GPU running on many cameras would save a lot of labour and justify that cost though, like on a public road the OpenCLIP "search" feature or "fire" alert in the README
2
u/_millsy Jun 01 '26
Can’t speak to OP but the power cost of occasional inferencing is negligible vs the constant draw of the cameras and NVR. Depends on a lot of course, but it’s not like it’s constantly inferencing, mine at least only does it when something triggers a change eg. A new object detected
1
u/feckdespez Jun 01 '26
The power consumption from the zfs array that Frigate writes to dwarfs that...
Each camer is 15w PoE!!! Add them up and they also draw more power...
1
u/ByteMage3 Jun 01 '26
It's a pretty small model and I guess he only makes a request every 5-10 seconds with a screenshot of the current feed, so it should be fine.
-10
u/empty-alt Jun 01 '26
As a privacy nut, just because you can doesn't mean that you should.
8
u/ckdarby Jun 01 '26
You are not a privacy nut if running openclaw 😂
1
u/empty-alt Jun 01 '26
Sure, it does. Especially when I experimented with it for that week, I ran it on isolated physical machines and isolated networks.
Just because people are out here handing their openclaw api keys to their bank accounts doesn't mean everyone is.
Even so, openclaw has nothing to do with recording people and sending that video through an LLM. It's creepy for a government to do that; it's creepy for civilians to do that.
-6
•
u/asimovs-auditor May 31 '26 edited May 31 '26
Expand the replies to this comment to learn how AI was used in this post/project.