What Audio Separation Actually Is
Audio separation - also called stem separation or source separation - is the process of splitting a mixed audio track into its component parts. Pull the vocal track out from a song while keeping the instrumental. Extract the background music from a video while preserving the speech. Remove a persistent noise from a recording while leaving the primary audio intact.
None of this was practically possible for regular users even five years ago. The traditional approach required expensive software, studio experience, and multitrack recordings where each instrument had been recorded to a separate channel. If you had only the final mixed file, the components were considered essentially unrecoverable. The mix was the mix.
AI changed that. Neural networks trained on enormous datasets of paired mixed and isolated audio learned to recognise the spectral and temporal signatures of different sound sources - what a human voice looks like in a spectrogram, what a drum kit looks like, how they overlap and interfere with each other. The models can now separate sources from a single mixed file with accuracy that would have seemed implausible a decade ago.
What It's Actually Good For
Removing background noise from video is the most common practical use. AC units, street noise, crowds, wind - these have consistent frequency signatures that the models have been trained to identify and separate. The quality of the result depends heavily on how dominant the noise is relative to the speech you want to keep, but for moderate interference the results can be striking.
Extracting vocals from music is what most people think of first. DJs do this for mashups. Cover artists do it to create backing tracks. Researchers do it for music analysis. The quality has gotten to the point where stem-separated vocals from mainstream music releases are genuinely usable for production purposes - not perfect, there are often artifacts in complex musical passages, but usable.
Creating karaoke versions is the obvious downstream application of that. Upload a song, separate the vocals, keep the instrumental - done. Several streaming services have used AI separation to create karaoke versions of songs that were never specifically mixed for karaoke release.
Podcast and interview cleanup is where I've found it most practically valuable. Remove the hum from a poorly recorded interview. Pull a guest's voice out from a recording where the levels weren't balanced properly. The results aren't always broadcast quality but they're often significantly better than the original, which is sometimes good enough.
The Free Tools That Are Worth Trying
LALAL.AI is the most polished of the free options. Upload a file, pick a stem type (vocals, drums, bass, piano, guitar, or general background), and it processes and returns the isolated track. The free tier gives you a limited number of minutes per file, not a fixed number of uses, which is actually more practical than most competitors' free tiers. The audio quality on clean studio recordings is impressive. Background noise removal from speech also works better than I expected.
Demucs is Meta's open-source model and it's the technical quality leader for music separation, particularly for complex multi-instrument tracks. It runs locally, costs nothing beyond compute time, and produces genuinely excellent stem separation on well-produced music. The catch is setup - you need Python and some comfort with running things from a command line. It's not a consumer app. But if you're separating a significant volume of audio and want no file size limits, no usage caps, and no privacy concerns about uploading your audio to someone's server, Demucs is the right answer.
Moises has a generous free tier and a very clean interface. You upload audio, select stems, and it handles the processing. Quality is good for vocals and instruments, somewhat weaker for general background noise removal compared to LALAL.AI. The collaborative features (multiple people working on the same session) make it useful for bands or production teams. For personal one-off separation tasks the free tier is plenty.
Spleeter is Deezer's older open-source model, now somewhat superseded by Demucs in quality but still widely used because the documentation and community support are extensive. Many third-party tools and web interfaces are built on top of Spleeter. If you see a web tool offering "stem separation" without naming the underlying model, there's a reasonable chance it's running Spleeter on the backend.
What It Still Can't Do Well
Field recordings with complex, overlapping noise are the hardest case. Multiple people talking simultaneously over background noise, or a speaker in a venue with significant reverb, often comes out worse after separation than before - the model tries to separate sources it can't cleanly identify, and introduces artifacts in the process. The wedding HVAC recording I mentioned worked because the AC noise was consistent and the speech was prominent. A crowded room with multiple speakers would not have cleaned up as well.
Low-quality source audio produces poor separation results. Heavily compressed audio, recordings with clipping, or files that have already been through multiple generations of lossy encoding don't give the model enough clean signal to work with. Garbage in, garbage out applies here more than in most contexts.
And stem separation is not the same as audio restoration. It splits sources - it doesn't enhance the quality of any individual source. If you separate vocals from a muddy mix, you get the vocal stem, but it's still the muddy version of the vocal. For actual restoration (removing degradation, improving clarity, reducing noise within a single source), you need different tools. Adobe Podcast's enhance feature and the speech enhancement in iZotope RX are designed for that purpose rather than separation.
The Best Workflow for Video
If you're trying to clean audio from a downloaded video rather than a pure audio file, the cleanest workflow is: download the video in the highest quality available, extract the audio track as a WAV or FLAC file (HandBrake or FFmpeg handles this), run the separation on the audio file, then recombine the cleaned audio with the video using HandBrake or DaVinci Resolve.
Keeping everything in lossless formats until the final export matters more than most guides suggest. Running separation on an MP3 that was already compressed introduces artifacts faster than working with uncompressed audio. If the source video has decent audio, extracting lossless and working in that format preserves more of what the AI model can work with.
MyVideoCity downloads video from TikTok, Instagram, Facebook, X, and Vimeo in the best quality the platform offers. For audio work, that highest-quality download is your starting point. See also the AI video enhancement guide for the visual equivalent of this process, and the video formats guide for understanding what quality you're actually getting from different platforms.
The technology isn't magic. But it's gotten to the point where "I have this one recording that matters to me and I want to rescue it" is a solvable problem instead of an impossible one. That's a genuinely good thing.