Product Update: Panoptic Segmentation Masks

Pixel-accurate panoptic segmentation masks replace COCO polygons in Bifrost dataset downloads, now delivered as high-bit-depth TIFF files.

Panoptic segmentation mask visualized alongside its rendered frame

Polygons approximate. Masks don't.

COCO segmentation polygons trace object boundaries as vertex lists, which means fine structure - rigging, railings, antenna masts, anything thin or concave - gets simplified away. We've replaced them with panoptic segmentation masks that map every pixel in the frame to an annotation ID. Your segmentation labels are now exactly as accurate as the render itself.

What's New

Panoptic segmentation masks now ship in every dataset download, replacing COCO segmentation polygons
Each mask file is mapped per image in the COCO JSON, so your loaders can find them programmatically
A sample notebook visualizes panoptic masks and converts them to semantic masks if that's what your pipeline expects
Still need COCO polygons? A conversion script turns the panoptic masks back into polygon format

Reading the masks

Every pixel in a panoptic mask stores an annotation ID rather than a color, so the file looks black in a standard image viewer. That's expected - the data is all there. Load it as an array and the IDs map straight back to your COCO annotations:

import numpy as np
from PIL import Image

mask = np.array(Image.open("panoptic/frame_0001.tiff"))
annotation_ids = np.unique(mask)  # every annotation present in the frame

The sample notebook included with your download renders the mask in color for inspection and handles the semantic-mask conversion for you.

Panoptic mask IDs rendered as a color-coded visualization

Update: TIFF masks for larger datasets

We've since upgraded the mask format itself:

Panoptic masks are now delivered as TIFF instead of PNG, supporting bit depths up to uint32 - enough ID headroom for larger, more complex datasets
Very large downloads may be slightly bigger on disk; smaller downloads stay about the same size
Packaging is faster, so your downloads are prepared sooner
The COCO metadata field bifrost_segmentation_path points to the TIFF file for each image

TIFF panoptic masks referenced from the COCO JSON metadata

No pipeline changes needed beyond reading TIFF where you read PNG - the snippet above works unchanged.

Pixel-accurate labels, no relabeling

Segmentation quality caps out at annotation quality. If polygon approximation has been the ceiling on your model's mask accuracy, this update removes it.

Get in touch