AditWork / About Me 



INDIVIDUAL PROJECT - 2019 ROLE: Designer, Research, Technologist
Case Study: Unstable Label

Design discovery through prototyping


Unstable Label is a project developed as a part of my MFA thesis exploring Machine Learning datasets in a civic context. The end outcome of the project was a set of provocations, ideas, and experimental approaches for machine learning presented in the form of a speculative data labeling application.

This case study gives an overview of the process of developing the concepts at the core of Unstable Label, showing how open-ended prototyping can be a productive tool for early design discovery.  





Design Process Overview


Typical design processes place prototyping and making in a concept development phase that follows an exploration phase that includes research and synthesis. Prototyping only comes into play once you know what to build, at which point you’ll go through an iterative cycle of designing and building that thing.

In this project, I use prototyping and material exploration as a method in the early design discovery phases as a way to think through ideas. Throughout my design process, these prototypes take different forms and fidelities, from rough cardboard models to scenarios and speculative software. What remains in common amongst these prototypes, however, is that they were not developed with an outcome in mind. Instead, they were built as a way to explore, communicate, and make sense of machine learning in the civic context.





 Research & Sensemaking 


The first phase of work involved amassing research into the space of machine learning in the civic context through prototyping, getting involved with the local community, and conducting secondary research.

Protototyping


Prototyping and making in this stage took the form of cardboard models (sometimes with simple electronics), simple web applications, and diagramming. These prototypes often were responding to other threads of research, as well as my own intuition.


In some cases, like with the “Become a Hotdog” web application, I was materially exploring ideas around how we “perform” for algorithms.

Some ideas initially conceived at this early stage of prototyping would show up in different forms in later stages of the design process.


Secondary Research


Secondary research was an extremely important part of this early design work, looking at existing design work that engaged machine learning as a subject matter, as well as critical work happening in Science & Technology Studies, Anthropology, and New Media Studies.

  1. into the black box - broadly focused on critical perspectives on algorithms, ML and AI
  2. data/labeling/information - research somewhere at building datasets for ML, data labeling, making information, and contestibility

Primary Research in Context


In addition to working the work in the design studio, I also spent time working to understand how machine learning was being operationalized in my local community, specifically with predictive policing. There is an active community of organizers in Los Angeles working on dismantling predictive policing as well as policing as a whole whose work I participated in, and learned from.


The learnings from this work greatly informed my design work, as well as provided the foundational values upon which my machine learning speculations were built from. Some particularly important takeaways for me were:

  1. Data driven policing highlights in an explicit way how power dynamics are embedded within algorithms.
  2. Thinking about algorithms from a much broader perspective, beyond the code, into the ways that algorithm is operationalized, how it was developed, and more.
  3. Centering the impacts on community in our analysis of algorithms, especially when those people aren’t always considered “users”

For more on the images here, see Algorithmic Ecology, and PredPol Is...


Sensemaking through Diagrams


Creating diagrams are an incredibly useful way for me to make sense of research. While creating these visual metaphors and flows can at time be a simplified view of what is going on, they also help to focus our view into a particular aspect of the process.


These diagrams were not built to be communication tools (though they could be). Instead, they were created a means for me to give material form to the ideas I was researching. The process of making them help me wrap my mind around the subject matter.



 Developing Scenarios 


The next phase of work was to do some synthesis of the broad range of research by creating specific scenarios that would put some of those initial explorations into particular contexts as a way to dig deeper into the subject matter.

From the initial research, I chose to focus on research threads on the process of data collection for machine learning as well as themes of surveillance, safety, and policing from my experiences working on dismantling predictive policing.


The scenarios I developed responded to the prompt:

What would data collection devices that collected data on fear look like? And how might you intervene in such data collection systems?



To address this brief, the research effort became more focused on mechanisms for collecting data. In this case, the research involved prototypes as well as secondary research on the material qualities of various data collection devices.


The development of the scenarios emerged through the prototypes I created. As the prototypes gained fidelity, I also began “prototyping” stories of the near-futures where these devices would exist, filling in details of the entire system of data collection.


Scenario 1: Civic Data Collection


The first scenario imagined a municipal “Data Surveyors” program, where residents participated in a “fear mapping” of their neighborhoods, which would impact where resources were deployed.

It posed questions about the subjective nature of data representing “fear” as well as questions about whose data would be collected in a program like this.


Scenario 2: Corporate Data Collection


The second scenario was focused on corporate data collection, imagining a system where a big tech company allowed users of their home security technology to train their own algorithms to automate their cameras.

This scenario imagines people intervening in such a consumer system by “re-labeling” the default categories provided.


To see more detail on these scenarios, take a look at the Machined Data project.

While these scenarios largely just provide critique of machine learning systems, elements and themes developed for these scenarios were key parts of imagining entirely new, more experimental approaches to machine learning.



 Moving from Research to Concept 


Up until this part of the process, the design work was largely about exploring and critiquing existing machine learning systems. The next step was to pull together all the insights from those explorations into new concepts for machine learning approaches


I began this exploration from an insight from the scenario development: that machine learning is a process of worldbuilding facilitated by the labeling of data, which meant that feeding an algorithm fictional data could be used to generate aspirational worlds.


This initial prototype sketched out an interface for users to create imaginary data ontop of google street view.



Developing Variations


The challenge became how do you prompt users of this tool for imaginary data? How do you spark their imagination? The following are variations developed to try to answer that question.


The most promising and intriguing approach was using the idea of relabeling, a theme that showed up throughout my design process.


First Prototypes


The first prototypes of this data labeling application designed around relabeling are shown below. For a user to label data, they had to create their own categories, instead of using ones that were pre-defined for them. They created new categories by relabeling existing ones.


This prototype was used as part of a physical installation that illustrated the re-labeling concept at the ArtCenter Media Design Practices Thesis Show.


Testing the Experience


The prototype, gallery show, and thesis presentation allowed me to test the concept.


The big insight from testing was that more than the new labels or images, the “description” was the real data, representing the stories behind why a user was relabeling.


Detailed Design


Following the gallery show, and testing of the prototype, I began to develop the next iteration of the app focused on the visual aesthetics of the software. How would I represent the ideas at the core of the concept through the visual aesthetic? And how could I enhance the main features to highlight the stories behind the data?


Some of the design explorations I went through can be seen in the sketches as well as Figma mockups illustrating some of the flows I explored.



Final Prototype & Design


The final app can be found at https://unstable-label.glitch.me, and the final designs can be seen below.


Some of the key changes/additions made in the final design were:
  1. Representing the “model” as a network diagram showing the branching nature of the categories used
  2. Allowing users to draw bounding boxes freehand instead of resizeable rectangles.
  3. Using lots of overlapping UI elements to represent the real messiness in developing categories for a machine learning algorithm.

This resulting application represents an experimental approach to machine learning. It serves as a provocation and represents a set of critiques of contemporary machine learning’s approach to data collection.