Segment Anything 2

Segmentation models can detect and separate out objects or elements within an image, and with Segment Anything 2, Meta have developed a model that can do this at a very high level, quickly and with video! There’s also a demo so everyone can give a go for themselves online!

Here we can see three objects within the scene have been segmented, a person, a dog, and a football, and those segmentations are tracked across a video clip. Each segment stays intact even when broken up by another object, like how the dog segment is maintained even when the person’s leg gets in the way.

In the demo Meta provides, you can simply select the areas to segment and apply different effects to them or the background. The model appears to do a decent job at rotoscoping each element out too.

Being able to quickly track and mask different elements within a shot could be a powerful tool for VFX artists, but I could see this model forming part of a wider VFX toolkit.