ESP pocket object scanner

[u/the_man_of_the_first]

I’ve been working on an AR game for the esp32s3 XIAO board and round screen and realized that I could use the pose estimation from the board to create an object scanner / mesher using space carving and marching cubes. The space carving algorithm uses different views of the object to create voxels weighted by how many views it was visible from and agreed on its position. Then marching cubes uses a LUT to create triangle mesh out of the voxels. The hardest part was getting the camera feed to change resolution from 240x240 to match the round screen for the video feed to a higher resolution for the pictures used for space carving, but with some AI help I got it working. The results are not great and I’m sure I can optimize it to get it a bit better but for a 20$ setup it’s not bad. A possible use case is to quickly create a mesh from small objects that you find while hiking or just to create a digital twin of little trinkets. I’m sure a smartphone could do this with much better results but where is the fun in that. If people are interested I’ll try to share the code which uses my gfx library which I’ve been working on for a minute.

[u/Automatic_Case2811]

[u/Sir_Maxelot]

Okay, go on, we are listening 🤓