research area Computer Visionconference ICCV
content type paperpublished December 2025

Unified Open-World Segmentation with Multi-Modal Prompts

AuthorsYang Liu†, Yufei Yin‡, Chenchen Jing§, Muzhi Zhu†, Hao Chen†, Yuling Xi†, Bo Feng, Hao Wang, Shiyu Li, Chunhua Shen†

View publication

Recent years have witnessed the rapid development of open-world image segmentation, including open-vocabulary segmentation and in-context segmentation. Nonetheless, existing methods are limited to a single modality prompt, which lacks the flexibility and accuracy needed for complex object-aware prompting. In this work, we present COSINE, a unified open-world segmentation model that Consolidates Open-vocabulary Segmentation and IN-context sEgmentation. By framing open-vocabulary task and in-context segmentation task as promptable segmentation tasks, COSINE supports diverse modalities of input, such as images and text. Containing a model pool and a segdecoder, COSINE makes full use of the representation capability of foundations models and is able to accurately segment specific concept based on diverse modalities of input, such as images and text, offering powerful open-world perception capabilities. Experiments on various segmentation tasks show the effectiveness of the proposed method.

  • † Zhejiang University
  • ‡ Hangzhou Dianzi University
  • § Zhejiang University of Technology

Related readings and updates.

Segmental Attention Decoding with Long Form Acoustic Encodings

July 6, 2026research area Methods and Algorithms, research area Speech and Natural Language Processingconference Interspeech

We address the fundamental incompatibility of attention-based encoder-decoder (AED) models with long-form acoustic encodings. AED models trained on segmented utterances learn to encode absolute frame positions by exploiting limited acoustic context beyond segment boundaries, but fail to generalize when decoding long-form segments where these cues vanish. The model loses ability to order acoustic encodings due to permutation invariance of keys and…

Read more

On-device Panoptic Segmentation for Camera Using Transformers

October 19, 2021research area Computer Vision, research area Methods and Algorithms

Camera (in iOS and iPadOS) relies on a wide range of scene-understanding technologies to develop images. In particular, pixel-level understanding of image content, also known as image segmentation, is behind many of the app’s front-and-center features. Person segmentation and depth estimation powers Portrait Mode, which simulates effects like the shallow depth of field and Stage Light. Person and skin segmentation power semantic rendering in group shots of up to four people, optimizing contrast, lighting, and even skin tones for each subject individually. Person, skin, and sky segmentation power Photographic Styles, which creates a personal look for your photos by selectively applying adjustments to the right areas guided by segmentation masks, while preserving skin tones. Sky segmentation and skin segmentation power denoising and sharpening algorithms for better image quality in low-texture regions. Several other features consume image segmentation as an essential input.

Read more