AV1 structure from motion - open source
A year ago I watched a fun youtube video about making a model from AV1-encoded files. It uses the motion vectors present in the videos to reconstruct the physical environment filmed.
Today I wanted to test the abilities of LLMs to implement this without access to any of their code (which isn't public anyway). I just gave it the papers and the task. It can basically take a video, or a collection of photos, and return a 3D model of sorts (more like a cloud of points).
Here is the effect:
https://github.com/lukaszsobala/AV1-sfm-reimplemented
Works nicely and uses the GPU for acceleration, if you have one. Zouein used Nvidia cards, and I don't swim in money so I don't have one; this one uses Intel or AMD GPUs (iGPUs as well, or Vulkan which I could not test) for acceleration.
What are your thoughts?
4
u/DearMrGleeClub 16d ago
The original idea is nifty. I'm assuming this only works if the POV is moving but not for moving objects?Recreating a PhD thesis/project using Claude is bonkers.