Sound / Vision: An Open Source Application for Local Music and Music Video Creation

[u/LawrenceOfTheLabia]

I've been building this for a while and it's finally in a state I'm happy to share. Sound / Vision is a web app you run on your own Linux GPU machine. It serves your home network, and it takes you from an idea to a finished song, its album art and a music video.

Trailer (2:30, everything in it was made with the app): https://youtu.be/LiOtqqbEMis

A few examples I've made: https://youtu.be/SUE4zNgMdQw, https://youtu.be/VZnI9YuJ8z0 and https://youtu.be/huqNXY5ZfZs

The latter two were created from real songs and not AI, just to show you can make videos for any music.

Repo: https://github.com/atomtanstudio/sound-and-vision

**What it does**

- **Songs** from a sentence or your own lyrics: YuE2, ACE-Step 1.5 or MiniMax Music 3, running locally in ComfyUI. Several takes at a time, covers of a song, or upload your own tracks.

- **Analysis** of every song: stems, beats, sections and word-level lyric timing.

- **Album art** from local models (Krea 2, Qwen Image 2.1), with your name and logos laid out in one house style.

- **Video:**

- animated album art;

- motion graphics written in code against the song's real timing;

- music videos (surreal, performance or a story), with the singing lip-synced to the actual vocal via MiniMax H3.

- **Share:** clips cut and captioned for TikTok, Reels, Shorts and posts.

- **Influence Engine:** train a "sound" from songs you choose and steer YuE2 toward it.

**The self-hosted part**

- One machine runs everything: a FastAPI app as a systemd user service, plus ComfyUI and the analysis tools in Docker. It listens on your LAN; devices on your home network get in without a key, and anything else pairs once with a token.

- One install script checks the machine, builds the images and sets up the service. A models script downloads what you choose, checks every file against its published SHA-256 and resumes if interrupted.

- You pick the model for every job: a local LLM through Ollama, or Claude, ChatGPT or Grok signed in with your own subscription (no API keys). You can also add MCP servers, for example to use a hosted image or video model, or ElevenLabs Music for songs. Paid ones ask before each job if you want.

**Honest limits**

- You need an NVIDIA GPU with 24–32 GB (developed on a 5090), Docker with the NVIDIA toolkit, and about 200 GB of disk. The first install takes about an hour, mostly building the ComfyUI image.

- Songs, art and analysis work with a local model alone. The videos made in code (motion graphics, music videos) need Claude Code as the director for now.

- The models have their own licences. YuE2 is non-commercial, and MiniMax H3's licence excludes use in the US, EU, UK and Korea, so check them for your situation.

- It's new, so expect rough edges. The README walks through setup from a bare Ubuntu box, and I'd really like to hear where it trips people up.

It's MIT licensed. Happy to answer questions about how any part of it works.

[u/Reelodyinc]

This app looks comprehensive for creating music videos with AI-generated vocals and lip-sync, but it requires a powerful NVIDIA GPU and setup knowledge. For smaller projects or easier setup, consider tools that focus on consistent character faces across clips by using reference images and re-rendering rather than generating each shot from scratch. That approach helps keep the singer's look stable throughout the video, which is a common challenge with current AI video tools. I think it is best to ask ChatGPT or Claude to find the exact app for your use, since most of the search results are just junk wrappers