Visualizing Identity
Machine Learning, Identity Data, Data Visualization
A text-to-image Machine Learning project attempting to bridge the perception gap between social identities through AI-made visuals, formed by identity data. These visuals will serve as stand-ins for individuals’ isolated perceptions of experience, and references for better understanding the formation of identity, how identity impacts experience, and how these experiences compare and contrast between social groups.
Visualizing Identity (2019)
In 2018, I came across a peculiar philosophical term, “qualia,” that describes the internal and subjective component of sense perceptions, arising from stimulation of the senses by phenomena. These are things like seeing the color red, tasting a glass of wine, and smelling a flower. Qualia are widely experienced, but there’s no guarantee that they are experienced the same way. For example, if someone describes the color red to an audience, there’s no way to know if the shade of red I imagine is the same shade of red you imagine, and if someone has never seen red before they can’t imagine it at all. Is this a limitation of human language, or a limitation of human consciousness? Philosophers aren’t sure, but it creates an interesting conundrum.
What’s the most interesting to me about qualia is how closely it resembles social identity and experience. Although I may try to explain to someone how it feels to exist at my social intersections, unless they exist at my same intersections (and honestly, even if they do) they may never be able to understand and visualize my experiences the way I do. This presents an obstacle for social change and collective human understanding, because it’s difficult to bridge a gap you do not fundamentally understand.
So I thought to myself, since human consciousness and language are limited, what would happen if I left these visualizations up to an AI instead? And could these visualizations serve as stand-ins for these things we visualize and experience, but can’t quite explain?
“I feel like a plant in the sun.” - An example input and output generated from the AttnGAN model.
Procedure
Gathering data was done using a self-designed oral survey covering 6 different aspects of social identity: age, race, ethnicity, gender, sexuality, and ability, and how each individual feels existing at those intersections. The results of each individual’s survey were then transcribed, and used as input for the AttnGAN model which produced a unique, AI-made “portrait” representing each individual person and their experiences.
The “portraits” are available below for people to observe and draw conclusions on how identity is formed, and the similarities and differences in experiences between social identities.
Model Overview
The idea was to interview people about their identity and how it feels to exist as their identity, transcribe it into text, and then feed these responses into a Machine Learning model to output an image. This lead me to the beautifully complex AttnGAN model (Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, Xiaodong He). In simplified terms, the way the model works is by intaking a text caption and sending it through a text encoder. The caption is then analyzed at the sentence level, but also at the word level using attention. Attention, in essence, assigns vector scores to each word, and compares those scores in relation to all other words in the sentence. The higher the score, the higher the importance of the word. The top-5 most attended words are then used to generate fine grained details in two different levels (generating two low-res images), and then the details from each layer are compiled into one final 256 x 256 px image. You can get into the nitty gritty of the model here.
“Being gay feels like flowers and rain.” - An example input and output generated from the AttnGAN model.
Results
Notes
The process for this project seemed simple enough. Collect identity data, feed it to the machine, get a visual then boom, done. Simple enough. Except nothing about identity is simple.
Trying to quantify something as abstract as identity loses a lot of nuance, and discussing social classifiers such as race and gender naturally incurs some bias. I did my absolute best to minimize bias and preserve nuance in this project; however, I acknowledge that fulfilling either goal 100% was not possible.
Also, it is important to acknowledge and note that identity is comprised of many facets. Therefore, only analyzing age, race, ethnicity, gender, sexuality, and ability is not an all-encompassing picture of someone’s identity. I’m hoping to conduct future iterations of this project with additional social aspects included to go deeper into understanding identity and experience.
Project Acknowledgements
Special thanks to Riley Wong, Colin Rothfels, and Charlie Lu for assisting with the GAN implementation, and Dr. Angie Luvara for survey development.