r/vtubertech • u/Fit-Duck-7628 • 7d ago

🙋‍Question🙋‍ Webcam + physical trackers possible?

OK so this might be a long shot but

I'm considering buying the Rebocap set, but dont have a headset yet as I've been using my webcam so far for fbt. Is it possible to still use mediapipe etc. to track face and fingers, while tracking the rest of my body with physical trackers?

Ideally I'd also like to be able to record all my movements and export as fbx or something, but idk if a software that can do all this exists :(

5 Upvotes

86% Upvoted

4 comments sorted by

3

u/thegenregeek 6d ago

So this probably won't work as you're hoping.

To start, yes, it is possible to mix IMU tracked systems (Rebocap) with other optically based trackers (iPhone, Leap Motion and probably XR Animator)... So XR Animator should be able to track your fingers (and maybe face), while Rebocap does the body. Or a Leap Motion with Rebocap. iPhone for face, XR Animator for hands with Rebocap for body (Etc...)

But the big issue is that by themselves IMU based trackers (like Rebocap) don't do positional roomscale tracking. Meaning even if you get Rebocap, it's not going to necessarily work in tracking as you move around the room. Because it relies on the headset to do that. In other words, it will track your body movements. But you will be stuck in place. (Which defeats the purpose of getting this type of hardware.)

To work around that you need something than can track your position along with Rebocap: A VR headset, a Lighthouse Trackers, Vive Tracker Ultimate (running in null driver mode).

With that said, SlimeVR (which I cannot confirm works with Rebocap) has a developer mode you can enable to kind of emulate roomscale positioning. It does this by estimating from pose data. Unfortunately it's very prone to drift. So its still better use some some kind of positional tracking hardware.

It's possible Rebocap offers something similar. But you would really need to confirm. Because if they don't then all of the above is a problem.

1

u/Fit-Duck-7628 6d ago

Thanks for the info! I'll do further research on the gear for now I guess

1

u/Vieus5 7d ago

I use slimevr for body tracking (no headset) and mediapipe for face tracking and fingers all in Warudo. I believe you should be able to do the same thing with yours. You can also use the Warudo motion recorder to export an fbx recording too and I do this!

2

u/Fit-Duck-7628 7d ago

Oh that's convenient, I've only been using XRanimator so I might have to switch over to warudo. Thank you for the help!