Bits of Sorrow is a virtual reality experience that uses data from cases of forced sterilizations in Peru performed under the guise of a poverty eradication program between 1996 and 2000. By modifying the variants of TopoSound, a GIS-based system in development, the experience allows the viewer to navigate Peru’s geographical characteristics in a virtual space. The red overlay in some areas shows how this sterilization program targeted people. When the viewer migrates from one place to another, a circle of pillars surrounds them, and musical notes play. The notes respond to the site’s altitude, showing how the land’s geographical characteristics influence indigenous people’s distribution in Peru. Simultaneously, these notes’ volume responds to the cases’ intensity, producing an auditory overload and discomfort.
VISUALIZATION
For the visualization portion of our project, we used the information available through the statistics portal of the Aurora National Program. The collection of this information is ongoing, so our data reflects the cases collected from 2016 to 2019. The data was processed to reflect the total cases by each department. We then used Mapbox Studio to create a render of Peru’s topography. This information was connected to the Maps SDK for Unity, which allowed us to explore a detailed
3D render of Peru that included digital elevation data. We used a red transparency layer as an encoding representing the number of cases in each department affected by the sterilization program. This layer’s opacity depends on the number of cases, having the number of instances tabulated to an opacity range from 10 to 60 percent. Additionally, we used TopoSound, a GIS-based system project currently in development, to add a visual representation that responds to the piece’s sonification. We set light pillars on the ground that surround the viewer. All pillars are the same size to show the elevation of the terrain. Once sonification starts, the pillars light from blue to their relative height encoding.
3D render of Peru that included digital elevation data. We used a red transparency layer as an encoding representing the number of cases in each department affected by the sterilization program. This layer’s opacity depends on the number of cases, having the number of instances tabulated to an opacity range from 10 to 60 percent. Additionally, we used TopoSound, a GIS-based system project currently in development, to add a visual representation that responds to the piece’s sonification. We set light pillars on the ground that surround the viewer. All pillars are the same size to show the elevation of the terrain. Once sonification starts, the pillars light from blue to their relative height encoding.
Image description: Stereoscopic image of Bits of Sorrow, generated using the Unity Recorder
SONIFICATION
TopoSound employs the sonification of topographical data to produce an auditorial experience that becomes informative. For this project, we set the system to approach sonification by synthesizing the Andean instrument’s musical notes, specifically the quena and Andean panpipes’ sounds. The instrument notes respond to the terrain’s elevation around the viewer’s geospatial location in the virtual environment. The terrain elevation is mapped to three octaves in the chromatic scale, allowing for granularity in discerning the data. To reference the number of cases by each department, we employed a formula that involved the weighted average of the different RGB channels in determining how red the area is. The formula used gave more weight to the red channel in an effort for it to better represent redness. Depending on this value, the sound’s gain changes from 35% to 1%.
Image description: TopoSound shows the default sonification accompanying visualization over the mountains of Banff, Alberta.
The planned experience is for people to use Bits of Sorrow in a VR or augmented reality setting. People will be able to move around an area while seeing a layered map of Peru. As the person stays still, the light pillars light up, and the sonification process starts. We encourage people to walk around and explore the map physically.
Due to the current inaccessibility of virtual reality experiences, a second version of the project is available as a 3D experiential video. Using a mobile phone, viewers can experience the
piece as they are guided through the virtual space. While this version of the work takes away the power of moving freely in the virtual environment, it allows the viewer to search around the locations where they are taken freely.
Due to the current inaccessibility of virtual reality experiences, a second version of the project is available as a 3D experiential video. Using a mobile phone, viewers can experience the
piece as they are guided through the virtual space. While this version of the work takes away the power of moving freely in the virtual environment, it allows the viewer to search around the locations where they are taken freely.
ACKNOWLEDGEMENTS
We would like to extend our heartfelt thanks to Dr. Izabella Pruska-Oldenhof for her insights, sources, and recommendations that were instrumental in shaping the direction of our project and will undoubtedly inform our future implementations. We are deeply grateful for her generosity in sharing her expertise and for her unwavering commitment to advancing scholarship in research-creation.
We would also like to thank Mathias Lacroix for his assistance in sourcing the musical base for our sonification. We are grateful for his dedication to his craft and for his willingness to share his talents with us.