Visual Analytics and Imaging Laboratory (VAI Lab)
Computer Science Department, Stony Brook University, NY

What is the Color of Serendipity? Investigating the Use of Language Models for Semantically Resonant Color Generation

Abstract: Humans inherently connect certain colors with particular concepts in semantically meaningful ways that facilitate visual communication. These colors are known as semantically resonant colors. For instance, we associate “sky” and “ocean” with shades of blue, and “cherry” with red. In this paper, we investigate how language models, including Word2Vec, RoBERTa, GPT-4o mini and the vision language model CLIP generate and represent nuanced semantically resonant colors for diverse concepts. To achieve this, we utilized a large dataset of color names and concepts, tailored models for the structure of each language model, and developed an interactive web interface, CONCEPT2COLOR, as a use case. Additionally, we conducted experiments and a detailed analysis to assess the ability of these models to generate meaningful colors. Through these experiments, we examined how factors such as model design, training data and context affect the color output. Our findings reveal the capabilities and limitations of language models in processing and generating semantically resonant colors for concepts, thus contributing insights into how they depict semantic color-concept connections. These insights have implications for data visualization, design, and human-computer interaction, where leveraging effective semantic color generation can enhance communication and user experience.

Teaser: This image shows three examples of visualizations with semantically resonant colors identifed with our language model powererd algorithm:

Teaser image

The composite image shows a bar chart (A), a pie chart (B), and an infographic (C) along with the colors returned by the Concept2Color interface. The bar chart, piechart, and infographics color were generated by CLIP, RoBERTa, and GPT-based models, respectively. Colors for (A) and (B) were directly generated from the input categories, whereas, the infographic colors are part of a cohesive palette returned by GPT 4 based on the theme “Mindfulness”.

Video: Watch it to get a quick overview how a user would use the CONCEPT2COLOR system to find semantically resonant colors for any word and concept:

Paper: S. Salim, T. Pial, K. Mueller, “What is the Color of Serendipity? Investigating the Use of Language Models for Semantically Resonant Color Generation,” IEEE Transactions on Visualization and Computer Graphics, 2026. PDF

Funding: This research was funded in part by a Stony Brook University seed grant.