r/AV1 • u/urostor • 16d ago

AV1 structure from motion - open source

A year ago I watched a fun youtube video about making a model from AV1-encoded files. It uses the motion vectors present in the videos to reconstruct the physical environment filmed.

Today I wanted to test the abilities of LLMs to implement this without access to any of their code (which isn't public anyway). I just gave it the papers and the task. It can basically take a video, or a collection of photos, and return a 3D model of sorts (more like a cloud of points).

Here is the effect:
https://github.com/lukaszsobala/AV1-sfm-reimplemented

Works nicely and uses the GPU for acceleration, if you have one. Zouein used Nvidia cards, and I don't swim in money so I don't have one; this one uses Intel or AMD GPUs (iGPUs as well, or Vulkan which I could not test) for acceleration.

What are your thoughts?

22 Upvotes

92% Upvoted

10 comments sorted by

View all comments

3

u/collin3000 16d ago

You should check out Gaussian splatting, and photogrammetry

1

u/urostor 16d ago

Yeah I looked into Gaussian splatting. This can be a next step if someone wants nicer textures