DOI: 10.14714/CP101.1767
© by the author(s). This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0.
Harrison Cole (he/him), The Pennsylvania State University | harrisoncole@psu.edu
Conventional visual maps present significant accessibility challenges for blind or low vision users, leaving them with few or no options for interpreting spatial data. This need not be the case: tactile maps, designed to be read through touch, have been published for more than a century. But they have most often been categorized as a navigation tool, or mere “tactile graphics” (i.e., not as expressly spatial documents). Tactile maps that allow their users to explore and synthesize thematic spatial data are rare, as are studies evaluating them. As our world continues to face existential threats that are spatial in nature—pandemics, supply chain disruptions, floods, etc.—maps will continue to provide critical information in ways that other media are unable to match. In the absence of accessible thematic maps, blind people will not only be left out of the loop, but their capacity for contributing valuable input will be severely diminished. In response, I describe here a study that evaluates the potential of thematic tactile maps for providing blind users an accessible means of analyzing spatial data when working in collaboration with sighted partners. Findings indicate that while the maps did not prove to be useful tools on their own, they did facilitate collaboration between blind or low vision participants and sighted participants. This suggests that, with some refinements, similar maps could be feasibly distributed as a means for people with visual disabilities to meaningfully participate in an otherwise inaccessible process that requires the synthesis of thematic spatial information.
KEYWORDS: thematic map; design; usability; user experience; blind; low vision; disability; disaster; hazard; mitigation planning; flood
Especially over the past 50 years, advocacy and research has resulted in a variety of accessibility standards for visual media that are intended to benefit people who are blind or have low vision (hereafter B/LV). For example, Section 508 of the Rehabilitation Act provides important technical guidance for creating accessible technology and media (section508.gov/manage/laws-and-policies), while the World Wide Web Consortium has compiled Web Content Accessibility Guidelines (w3.org/WAI/standards-guidelines/wcag). These efforts provide a useful checklist for improving accessibility for web users with disabilities, covering improvements such as alt text, prerecorded audio, and gesture-based interaction (Kirkpatrick et al. 2018), and are ostensibly meant to extend to web maps as well; however, they do not address the particular qualities of maps themselves, instead categorizing maps as a type of generic image or widget rather than specifically as documents/tools for representing space. This is characteristic of most accessibility guidelines that cover maps. With the exception of region-specific guidelines such as the Australian National Specifications for Tactual & Low Vision Town Maps (Goodrick 1984), comprehensive and widely-adopted accessible cartography standards have yet to be developed (for further discussion, see Hennig, Zobl, and Wasserburger [2017]). In response, my research contributes two important perspectives: (1) a multivalent focus that considers the dimensions of accessibility that involve people’s lives outside of the technology itself; and (2) a focus on accessible cartography, specifically, rather than accessible media in general. The importance of this latter point to the field of accessibility research is illustrated below.
The term “accessibility,” is often used to refer to a technology’s usability, especially by people with disabilities. But “accessibility” can more broadly refer to a technology’s availability, geographic proximity to potential users, institutional support, reliability, operability by a wide range of users, and so on. Taken together, these goals will be referred to in this article as “multivalent accessibility.” In short, “access” here is not synonymous with “accessibility” (Brown 2009). While multivalent accessibility can be examined for any technology, the study described in this paper explores it in the context of mapping.
Current guidance for creating “accessible” maps on the web is to append a type of data to static map images called “alt text,” which is essentially a written description of the map that is readable by accessibility software. Alt text is usually a suitable method for increasing the accessibility of simple photographs or diagrams, and it may be appropriate for extremely simple maps (e.g., walking directions between buildings on a university campus). But given the dynamic and scalable nature of vector data, alt text quickly becomes too cumbersome a tool for web map accessibility, and although research is underway to determine how alt text could best be applied to interactive web maps (Hennig, Zobl, and Wasserburger 2017), these approaches are currently largely experimental.
Alt text is not the only tool available for making maps more accessible. Tactile maps, which use physical volume and texture to represent data, present an alternative option for B/LV people. This is a technology that has existed for over a century (e.g., Rumsey 2015), and has proven to be effective for communicating at least basic spatial information. However, though many efforts have been made to improve on this method by way of digital and other electronic components (Cole 2021), high-tech tactile media can be expensive, complicated to use, or both. As a result, more complex maps and/or maps that use more advanced technology tend to be available mostly in settings where institutional funds and resources are available; thus, thematic tactile maps are used primarily by school-age users, while tactile map use among adults is primarily for the purposes of wayfinding (Aldrich and Sheppard 2001; Cole 2021).
In the interest of keeping with the goals of multivalent accessibility, I have approached the design of maps in this study by eschewing cutting-edge (and often expensive) technology in favor of designs using cheaper, proven technology, with the notion that the maps presented here will be deployable more widely and sooner than if research-grade equipment was used.
The maps in this study will represent data about floods. In the United States, the Federal Emergency Management Agency (FEMA) distributes Flood Insurance Rate Maps (FIRMs) in order to help property owners determine whether their property overlaps with a flood zone—in which case they would be required to purchase flood insurance. But FIRMs are also frequently used in the development of natural hazard mitigation plans (NHMPs), which are documents that communities produce by analyzing their vulnerability to various natural hazards and identifying steps that they can take to mitigate damage from those hazards, after which the NHMP can be submitted to FEMA for grants to help carry out the plans. However, FIRMs are visual maps, meaning that B/LV community members are effectively excluded from contributing to major portions of the NHMP process. These are the circumstances into which the research presented here intervenes.
In this article, I will use terms such as “B/LV people/users” or “disabled people/users.” This is called identity-first language (IFL; Dunn and Andrews 2015), contrasting with what is known as people-first language (PFL), which encourages the use of phrases like “a person who is blind” or “a person with a diagnosis of blindness,” intending to foreground that person’s humanity rather than their disability/diagnosis (Snow 2007). An extensive debate surrounds the use of these terms and will not be covered here (cf. Botha, Hanlon, and Williams 2023; Muredda 2012; Streeter 2010). Instead, I defer to the official position of the National Federation of the Blind (NFB), which rejects PFL in favor of IFL, and has resolved thus: “We believe that it is respectable to be blind, and although we have no particular pride in the fact of our blindness, neither do we have any shame in it. To the extent that euphemisms are used to convey any other concept or image, we deplore such use” (Resolution 93-01, 1993). Because NFB members constitute the majority of this study’s participants and consultants, they and the population that they represent in this study will be referred to using IFL unless grammatically cumbersome.
In this study I aim to meet two goals. The primary goal (Goal 1) is to evaluate the use of tactile maps by B/LV users to analyze flood risk. This study does not introduce any novel technologies, but rather evaluates how existing resources can be applied to a novel context; specifically, tactile maps being used for flood risk analysis. In terms of a research question, Goal 1 is meant to answer, “Can existing resources be used to create accessible flood maps?” Secondly, map use will be evaluated in a collaborative context with B/LV participants working with sighted participants in order to simulate the type of environment that might be encountered in a real-world NHMP process (Goal 2). Goal 2 address the question: “Can accessible flood maps foster collaboration between B/LV and sighted community members?” This study is a formative assessment (Buttenfield 1999), meaning that it is one step in a series of design iterations, as opposed to a summative assessment that compares a new design against an existing one.
In the following sections, I first present a brief overview of research on tactile maps, collaborative decision-making with maps, and accessibility in hazard mitigation planning. Next, I detail how the maps used in the experiments were designed. Then, I explain the study methodology and results. Afterwards, I discuss the implications of the results, study limitations, and challenges that I encountered—specifically as they pertain to certain accessibility issues. I then conclude by discussing how this study can inform future research in accessible hazard mitigation planning, but also accessible cartography more broadly.
In terms of scholarly research, tactile maps appear most frequently in the context of orientation and mobility (O&M) studies, whether researchers are evaluating design (Engel and Weber 2021; Jehoel et al. 2006), production methods (McCallum et al. 2005; Rowell and Ungar 2003; Shi et al. 2020), or how those factors affect wayfinding (Toyoda et al. 2020). Especially within the last 20 years or so, it has become much more common to investigate tactile O&M maps that augment their volumetric/texture features with audio feedback (Papadopoulos, Barouti, and Koustriava 2018), haptic feedback (Katzschmann et al. 2018), or interactive multimodal devices, which are often modified touchscreen devices (Giudice et al. 2020). Regardless of the modality, wayfinding is by far the most common use case in tactile map research, which is in stark contrast with visual cartography research that has produced decades of scholarship on the social and scientific/analytical dimensions of thematic and reference maps in addition to wayfinding research.
Because this study’s design is in many ways rooted in analysis of the social dynamics of disability, carefully considering exactly whom the tactile maps will be made for was foundational. To address this question, a number of researchers and practitioners have adopted the ethos of “universal design” (UD) when conducting tactile map studies (e.g., Coughlan and Miele 2017; Hasan and Gjøsæter 2021). UD advocates that design (of products, environments, media, etc.) should be accessible to everyone regardless of ability, disability, age, size, gender, and so on (Lobben, Brittell, and Perdue 2015). Therefore, a map designed in line with UD principles would be usable whether or not the user is B/LV or sighted. A document that is printed entirely in braille, by contrast, would not follow UD principles. Critiques of UD point out that it does not necessarily address issues of justice or the fact that universally optimal design qualities do not, for the most part, exist in practice (Hamraie 2013); in light of these critiques, the study presented here was not designed to follow UD principles. But applying UD to future iterations of this research may prove fruitful.
Another changing research dynamic in the field of tactile map studies has been the rise of participatory research, which entails the subjects of scientific research guiding, to varying degrees, the research itself (this is not necessarily related to the move towards referring to subjects as “participants”). In short, it espouses working with rather than working on or for the subjects of research (Vaughn and Jacquez 2020). In tactile cartography research, this often takes the form of prototyping maps with guidance from B/LV people, O&M instructors, or people otherwise associated with B/LV communities (e.g., Ghodke et al. 2019; Thevin et al. 2019), as opposed to sighted researchers recording feedback from B/LV participants only as experiment data. It should be noted that simply eliciting design feedback from study participants is not, under most sets of criteria, sufficient for research to be considered “participatory.”
One of the driving principles of this study is that the contributions of B/LV people to NHMP processes will benefit not only the B/LV people themselves, but also the communities that they live in. Enabling these contributions will permit a larger number of people with a greater diversity of knowledge and experience to contribute their insights (Henly-Shepard, Gray, and Cox 2015), in addition to helping ensure consensus amongst community members (Jankowski and Nyerges 2001). For producing and communicating these insights, maps in particular can be of significant benefit, not only due to the spatial nature of disaster planning, but also because they facilitate exploration of data, evaluation of alternative solutions, identification of conflicts, and other crucial decision-making operations (Henstra, Minano, and Thistlethwaite 2019), all in a format that can, under certain conditions, be used simultaneously by multiple users (Koski et al. 2021; MacEachren 2001). All told, a collaborative map-based planning process, for hazard mitigation planning or otherwise, is frequently regarded as a “value-added” approach, especially in situations that involve public input (Pelzer et al. 2014).
Collaborative mapping methodologies are often filed under the heading of “participatory GIS” (PGIS), or “public participation GIS” (PPGIS); these are terms that are generally associated with increased “lay” participation in previously expert-driven processes involving maps, often involving the creation of spatial data in addition to its analysis (Sieber 2006). Some important PGIS research has examined these practices critically (e.g., Elwood 2006), questioning who the “added values” benefit, or who can, in practice, actually participate. Research by Gregory Brown (2012) suggests that advances in (P)PGIS research methods have not actually led to increased public engagement decision-making. There are a number of reasons for this, but the key takeaway is that simply involving more people in a planning process does not necessarily lead to unqualified benefits, but rather introduces additional dynamics that, while they may not be intractable, should be accounted for. The study presented here is meant to provide some initial daylight into these questions: what are some of the dynamics that arise, how are they connected to mapping, and what are some future trajectories? These questions will not be entirely answered of course, but will at least be broached.
While research on the experiences of B/LV people in disasters is sparse, we do know that disasters are especially challenging for disabled people in general relative to non-disabled people. Among other problems, people with disabilities experience greater difficulty obtaining information or notifications about disasters (Gerber 2009), higher injury and mortality rates, and more difficulty finding shelter during a disaster; they also tend to be neglected by support systems meant to aid disaster victims (Arnör 2014; Stough and Kang 2015). In light of these conditions, disaster risk reduction (DRR) researchers and policymakers have developed a set of guidelines referred to as the Sendai Framework, which aims in part to address these disparities by calling for increased, deliberate attention to the particularities of how people with disabilities experience disasters (UNISDR 2015). On the whole, most recommendations in the Sendai Framework are not especially complex or specific: many, such as maintaining more up-to-date databases or including more stakeholders in disaster planning, arguably benefit more than just disabled people and result in an overall more robust disaster planning process.
In addition to broad policy-level efforts, more targeted work has looked specifically at disability-inclusive DRR processes, and a subset of this work has looked at the role played by maps. Ronoh, Gaillard, and Marlow (2017), for example, address disaster planning’s frequent exclusion of children, and especially disabled children, as community stakeholders (see also Good 2015) by using mapping as a participatory risk analysis method. The maps in their study used a variety of materials and ultimately helped the children better articulate what they already knew regarding high-risk areas of their community—a level of collective knowledge that surpassed the expectations of those involved in the study. In a similar vein, Gaillard and Maceda (2009) examined 3D participatory mapping for community-based DRR in the Philippines and found that 3D maps—made from common materials like cardboard and push-pins—are better than conventional two-dimensional at representing a range of scales, are more intelligible to people outside of the planning group, and can communicate more data. This project did not address the participation of disabled people specifically, but, importantly, the collaborative and participatory methods used are also used in a number of B/LV-specific mapping studies.
I developed a collaborative experiment to evaluate the utility and usability of thematic tactile maps for decision making in the context of flood mitigation. I recruited participants to perform tasks drawn from FEMA’s Local Mitigation Planning Handbook (2013), which outlines the steps that a community needs to take in order to assemble a NHMP. These tasks required participants to use tactile maps that were provided to them, and the task instructions were given in the form of an online questionnaire. The tasks were all map-based, requiring simple assessment and evaluation, such as counting the number of buildings, or identifying areas of high flood risk. Some of these tasks required participants to work on their own, while others required participants to work collaboratively with a partner: B/LV participants were paired with sighted participants, and each session involved one pair of participants. Participant pairs were also asked to work together to produce a short summary of their “findings” after completing the map-based tasks. Finally, participants reported, using a Likert rating scale, how confident they felt while using the maps to complete the tasks. Because this is a formative assessment, this study was meant to determine whether tactile flood maps are worth pursuing in future research: affirmative results would demonstrate that the maps presented no substantial impediments to collaboration while also being legible to a degree that does not hinder collaboration.
This study was designed to contribute a user-focused perspective on tactile map design to the existing literature, augmenting a body of research that currently focuses predominantly on the maps themselves. While the technology used in this study has been in use for decades, it was not evaluated on its own, but rather in a novel configuration and applied to novel situations. In other words, this study was designed to provide a foundation for future research into tactile flood maps (or other types of accessible maps).
The questionnaire consisted of six main sections: demographic information, individual reference map tasks, individual flood map tasks, collaborative assessment questions, collaborative summary, and post-collaboration evaluation (full study instruments can be found in Appendices A and B).
Demographic information was meant to gather basic information about participants. Individual reference and flood map tasks asked the participants to locate, identify, and measure various features on the map in order to help ensure that participants became familiar with the map’s content. They also served as “quality control” questions, checking that participants were able to read at least some of what was being represented on the maps. These questions included counting the number of buildings on the map, counting the number of road labels, and measuring the east-west distance of the map using the scale bar. These tasks did not require comparison between the reference and flood maps.
Collaborative assessment questions were both subjective and objective, with some asking participants, for example, to identify the proportion of buildings overlapped by flood zones, while others asked participants to explain their reasoning for choosing a map quadrant that would be best for establishing an emergency meeting point. The collaborative summary section asked participants to develop a written summary of their analysis with regard to how the mapped community could take steps to reduce their vulnerability to flood damage. These collaborative tasks required comparing both reference and flood maps against each other, as well as each collaborator consulting the other. There was no information that was available to one partner and withheld from the other.
The post-collaboration evaluation asked participants to report their confidence levels while completing various parts of the questionnaire, both by themselves and with their partner. This section used a Likert scale to measure confidence and asked about each participant’s experience using the map as well as their experience working with a partner.
Blind participants were recruited through a nationwide email listserv maintained by the National Federation of the Blind, while sighted participants were recruited via postings to local listservs and message boards for the State College, Pennsylvania area. People who expressed interest were then paired with a partner based on mutual availability. In all, 20 participants took part in the collaborative experiments: 10 blind and 10 sighted. One participant’s questionnaire was left unfinished, so their partner’s results were discarded as well, leaving 9 viable sets of results, or a total of 18 participant responses.
Participants were divided into two groups: one using maps of Pasadena, Texas, and the other using maps of Quincy, Massachusetts. These two locations were chosen because they offer a similar variety of features (buildings, roads, etc.) but in different configurations. This division was made in order to reduce any bias that may arise if only one location was represented in the maps.
Design conventions for tactile maps do exist; however they are generally presented as guidelines for the broader category of tactile graphics, and not specifically maps. For the study presented here, conventions used to design the maps were drawn primarily from the Braille Authority of North America and the Canadian Braille Authority’s Guidelines and Standards for Tactile Graphics (2010) and Tactile Graphics by Polly Edman (1992). While both volumes include sections dedicated specifically to tactile maps, they do not present tactile maps as spatial information documents, and the conventions that are given exist primarily in the interest of legibility instead of articulating spatial data.
With that in mind, the maps I developed and used in this study were created through an iterative design process, incorporating guidelines from the above texts in addition to feedback from tactile graphics users, tactile graphics design professionals, and educators of B/LV students, all of whom reviewed pre-testing drafts of the maps. Then, the maps were used as test stimuli for individually based experiments, where participants worked with the maps alone (preprint available here). Early feedback from the individual experiments, along with further input from tactile graphics design professionals, resulted in a second revision of the maps, which were used as stimuli for the collaborative experiment sessions described in this paper.
When designing the maps, I made efforts to minimize my reliance on any technology with a relatively steep learning curve (I considered a 3D printer too complex for this study) or high initial costs (such as professional tactile graphics printers, which start at around $5,000 USD) in order to ensure that these maps, if widely distributed, would be printable in facilities that are not primarily set up to serve people with disabilities, or to produce any sort of specialized media. To that end, microcapsule paper was chosen as the printing medium for the maps. Also referred to as swell paper or swell-touch paper, this is a special type of paper with a layer of embedded chemicals that expand when exposed to heat past a certain temperature threshold. This threshold can be lowered by increasing the thermal conductivity of the paper, such as by applying ink with a high carbon content. Thus, when microcapsule paper with high-carbon ink is exposed to heat (by passing through a small device with a heat lamp called a “fuser,” which costs about $1,400 USD), the inked areas expand while the blank areas remain inert, resulting in a textured surface with edges that are well-defined enough to create readable braille cells (Figure 1). Studies have shown microcapsule paper to have high user satisfaction, durability, and tactile-graphical fidelity (Brittell, Lobben, and Lawrence 2018; Rowell and Ungar 2003).
Fortunately, typical consumer inkjet printer ink has a high enough carbon content to create functional microcapsule paper graphics, although using consumer-grade printers poses an issue as well: standard braille documents are usually 11 × 11.5 inches, and most home/office printers are only able to accommodate up to 8.5 × 11-inch US letter-sized paper. In the interest of reducing the amount of specialized equipment that one must invest in to create these maps, I chose to design the maps at a size of 8.5 × 11 inches. So while a typical FIRM (Figure 2) is meant to be printed on 24 × 36-inch paper, the maps used in this study provide a much larger-scale representation. Importantly, there is a precedent for this scale change: FEMA also produces what they call FIRMettes (Figure 3), which use the same data layers as a FIRM, but on a much larger scale and are sized to be printed on US letter-sized paper. FIRMettes served as the template for initial drafts of the tactile maps, but the layout was abandoned in later drafts.
Figure 2. A FIRM depicting flood risk for part of Suffolk County, Massachusetts. Orthoimagery is included in this FIRM but is not standard across all FIRMs, nor is any particular color scheme. Obtained from msc.fema.gov/portal.
Figure 3. A FIRMette showing flood risk for portions of the cities of Chelsea and Revere, Massachusetts. Obtained from msc.fema.gov/portal.
In subsequent designs, the layout template and road data layer came from TMAP, or Tactile Map Automated Production. This is a website (lighthouse-sf.org/tmap) developed by LightHouse for the Blind and Visually Impaired that creates on-demand tactile maps of a chosen area to be printed on microcapsule paper (Miele 2004). There are several advantages to using TMAP: (1) the map scale of 1:5,000 is fairly close to the scale of 1:6,000 used by FIRMettes, (2) maps are printable onto 8.5 × 11-inch paper, and (3) TMAP is being continuously improved, but is fairly well-established and reliable at this point (i.e., it is no longer research-grade software). TMAP maps include a tactile map along with a key on a separate page, along with versions of the map and key that use Latin script for sighted users.
Ultimately, study participants were provided with two maps: a “before” (reference) map that included roads, buildings, and hydrographic features, as well as an “after” (thematic) map that superimposed flood extents (derived from actual flood extent data, but modified for this study) on top of building and road features. Two different locations were chosen to be mapped—Quincy, Massachusetts and Pasadena, Texas—due to the variety of building sizes road types that they offered, and their proximity to flood zones. The features in flood zones were obscured, but users could still reference the “before” map to see what changed. These two maps were supplemented with two corresponding legends on separate pages, resulting in a packet of four ring-bound pages (Figures 4–5; 7). Each sighted participant received a PDF version of the tactile maps that the B/LV participants received (Figure 6), with the only differences being that all PDF text used the Latin alphabet rather than braille, and because less space was required, the keys were condensed to one page.
Figure 4. Collaborative experiment map set for Quincy, Massachusetts. Top row: reference map and key with roads, buildings, and coastline marked. Bottom row: flood map and key with roads, coastline and flood zones marked. Note that these maps include non-critical typographic errors.
Figure 5. Collaborative experiment map set for Pasadena, Texas. Top row: reference map and key with roads, buildings and coastline marked. Bottom row: flood map and key with roads, coastline and flood zones marked. Note that these maps include non-critical typographic errors.
B/LV participants were mailed a set of tactile maps while sighted participants were emailed a PDF of the tactile maps with the Latin alphabet replacing any braille text. I then called each pair of participants over the phone and began a conference call, allowing the three of us to speak to each other simultaneously. Participants then opened a link to an online questionnaire.
After joining the call, participants completed the first half of the questionnaire, then notified me when they were done, after which they began the collaborative portion. I gave them instructions for how to complete the collaborative portion, but otherwise gave no guidance to the participants except in the case of providing technical assistance. After the collaborative portion of the questionnaire was complete, participants then left the conference call and answered questions in private pertaining to their respective experiences with using the maps and, importantly, on working with their partner. Upon completion of the questionnaire, each participant was given $20.
Some prevailing demographic themes were present amongst participants (Tables 1 and 2). Most (>50%) participants in both B/LV and sighted groups identified as female, most were younger than 45 years old, most were college-educated, and almost nobody had any previous experience with community disaster planning. Most of the B/LV participants had spent all or nearly all of their lives as blind, and most of the sighted participants had little to no experience working with B/LV people in the past. All of the B/LV participants had at least some familiarity with braille, but several expressed general reservations regarding the use of tactile maps and graphics.
In the following elaboration of results, I largely don’t distinguish between the groups that worked with Quincy maps and the groups that worked with Pasadena maps. This is partly because there were no dramatic statistical variations between the two groups (nothing as striking as, for example, the Quincy group hypothetically answering the legibility questions 100% correctly and the Pasadena groups answering them 15% correctly), but also because the sample sizes are very small, thus any difference between the two groups could easily be attributable to individual variation.
For blind participants, the results of legibility tasks were highly varied. When asked to count the number of buildings shown on the maps, answers ranged from 5 to 64 for the Quincy group (the correct answer was 59), and 2 to 35 for the Pasadena group (the answer was 36). Similarly, when asked to use the scale bar to measure the east-west distance being represented by their map and to choose the correct answer from four options (both group’s maps used the same scale), participants chose response options between 800 and 1,250 for the Quincy group, and 128 to 1,250 for the Pasadena group. The range of responses given by sighted participants was much smaller: when counting buildings, the Quincy group answered between 54 and 60, and the Pasadena group answered between 26 and 35 (one participant’s answer was a single question mark). All sighted participants answered the scale bar question correctly.
When asked the identify the map quadrant with the highest building density, 3 out of 7 of the Quincy B/LV participants gave a correct answer, and 3 out of 6 of the Pasadena participants were correct. Of the sighted participants, 7 out of 7 in the Quincy group gave the correct answer, and 5 out of 6 in the Pasadena group. Nearly all B/LV participants correctly identified the amount of space taken up in the maps by flood zones, but answers were mixed for sighted participants.
When working together, participants talked through each question with their partner while also recording their responses in the online questionnaire. The collaborative portion was recorded with their permission. What follows is a collection of themes that emerged from these sessions, including data from both Quincy and Pasadena groups.
Communication: Most participant pairs did not have any significant communication challenges, and several participants explicitly noted that they had fun with the experiment. Pairs tended to work in a truly collaborative fashion: neither the B/LV nor sighted participants were more or less likely to “take charge” by dictating the pace of work, giving final approval for answers, or solving the majority of each problem. After coming to an initial conclusion for a question, participants would ask, “Does that sound right to you?” Or they would describe their own answer, then ask, “but what do you think?” Similarly, neither the B/LV nor the sighted participants contributed any more or less information than their partners. This is to say that insights—or lack thereof—into what the maps represented were shared between partners. Often, if one participant had no insights to contribute, or was confused by the question being posed, then their partner typically shared their sentiments.
Information: Even though most participants, both B/LV and sighted, expressed a desire for more information to work with, every pair was nevertheless able to produce a set of proposed measures that the mapped community could take to further mitigate flood damage. Some participant pairs would propose steps that did not reference data on the maps, like education and outreach campaigns, establishing a registry for disabled people, or establishing evacuation routes without referring to specific roads on the map. However, multiple responses did in fact use the maps to propose measures with specific spatial references, such as, “A meeting place could be established near [the] high density area in the southwest,” or “The building in the southeast should be accessible and used for a safe shelter,” or “Create more routes to get from the west side of the map to the east side where there is less flooding.”
Outreach: Three participant pairs, in their suggestions for community mitigation steps, advocated for the distribution of similar maps to the community. One participant wrote, “Tactile maps like the ones we are using should be made available to community members who experience blindness or visual impairment as printed maps are made available to community members with sight.” Also, “having tactile maps available . . . for blind and visually impaired people to assess the safety/potential damage of [surrounding neighborhoods] would be helpful.” Five of the remaining pairs either simply recommended that the maps be improved/expanded, or they did not mention maps specifically but did advocate for some sort of geospatial outreach. For example, participants suggested that “The community can help blind people or low vision people become more acquainted with their area or the geographical make up of streets and neighborhoods,” or “All the [property owners] need to disclose to their buyers/renters that they are within a flood zone.”
Assistance: Sighted participants had a somewhat better grasp on what the maps represented in terms of identification and location of features. This is reflected in their responses to the legibility questions, but it also held true during the collaborative portion. However, if B/LV participants were unclear as to the identity of a feature, their sighted partner would assist them, and afterwards the B/LV participant was able to generate insights about that feature.
Following the collaborative tasks, both B/LV and sighted participants were asked to give several ratings of their confidence while performing these tasks, using a 5-point Likert scale (Figure 7), with a rating of 1 representing “Not at all confident” and 5 representing “Extremely confident.” Overall, confidence ratings were largely medium-to-high, and the average confidence ratings of B/LV participants were on the whole higher than the sighted participants. In fact, several questions saw B/LV participants give an average rating over 4.0, while none of the sighted participants did (though none of them were below 3.0 either). All participants expressed the lowest amount of confidence in response to the question, “How confident did you feel while using the two maps to complete tasks?” Second-lowest, for both groups, was their confidence in other blind people being able to easily use the maps.
Figure 8. B/LV responses to confidence questions, in descending order according to average reported confidence.
On the other end of the spectrum, B/LV participants expressed the most confidence in learning the meanings of symbols on the maps and in answering questions with a sighted collaborator. Sighted participants had the highest confidence ratings for learning the symbols as well, in addition to using the maps to analyze flood risk. When asked what sort of partner each participant would prefer to work with, most participants chose “With any partner,” although the second-most popular response for each group was “With a blind partner.” Overall, no striking differences were present between the Quincy and Pasadena groups: both total average confidence scores were identical.
The results of this study strongly suggest that tactile flood maps are indeed worthy of further development and investigation. Both Goal 1 and Goal 2 were met, with some caveats. In the following, I elaborate on these determinations as they relate to the study findings.
Participants reported the lowest average confidence ratings for the first question asked of them: “How confident did you feel while using the two maps to complete tasks?” Significantly, though, their confidence was not negative—i.e., lower than 2.5. There are a number of possible reasons for this low confidence: (1) simple acquiescence bias, which was accounted for in the survey design but may be present regardless; (2) the question is the vaguest, and thus most likely to elicit the most neutral response; (3) the question was asked first, so participants did not yet have a clear idea as to what sorts of responses were being requested of them; or (4) the notion of “completing a task” does not imply any specific goal, and as a result participants may have a more difficult time making an evaluative judgement.
Also noteworthy is the discrepancy between how accurately participants answered legibility-based questions and their reported confidence in answering those questions. Specifically: as mentioned earlier, B/LV participants were asked to count the number of buildings on their map, and some were off by a factor of more than 15. The same was true when asked to measure the map using the scale bar (the east-west length of the map was 1,250 feet, but some participants answered that it was 280 feet). However, those same participants also reported high confidence ratings regarding learning map symbology.
This study is not designed to identify an explanation for the confidence/performance discrepancy, especially given that the relationship between the two variables is, at best, highly contingent (Moore and Healy 2008), but it is worth noting that while the maps on their own were, at least for this study’s participants, unable to provide enough information to be used without external assistance, this issue largely dissipated in the collaborative portion, once the B/LV participants were able to check their answers with a partner. Given that fewer than half of the B/LV participants answered legibility questions correctly, or were very close to doing so, the results seem to suggest that any partner—B/LV or sighted—is much better than none at all.
To reiterate some of the circumstances under which this study occurred: most B/LV participants had little or no experience with disaster planning, tactile maps, or even tactile graphics; most of the sighted participants had little or no experience with disaster planning and none had worked with B/LV people in any capacity previously. And yet, in spite of this almost complete lack of experience with the central elements of this study, participant pairs, with very few exceptions, were able to answer questions in this study without any major breakdowns in communication or fundamental disagreements over their interpretations of the maps.
This could simply be due to the fact that people are likely to cooperate when they stand to benefit from working together, even when the costs of cooperating outweigh the potential benefits (Jordan et al. 2016). In the case of this study, the participants were instructed (repeatedly) to work together, were given an incentive of $20 to finish the study by working together, and no significant penalty was levied for not doing so, except perhaps the consumption of time. This would, at the very least, suggest that tactile flood maps do not hinder cooperation. Indeed, B/LV participants reported that they were much more confident in answering questions knowing that their partner was using a visual version of their maps. One participant said on the phone, “when I was working with [my partner] it started making more sense . . . versus when I was looking at it alone.” Their partner agreed, “Definitely. Yes, definitely.”
In sum, this study’s B/LV participants were able to use even the limited information available to them, despite their lack of previous experience, to perform some basic spatial analysis with their partners and come to agreed-upon conclusions. Despite the maps being radically redesigned in comparison to the original FIRMs, B/LV participants were able to contribute to a simulation of a process that, while meant to engage the entirety of a community, currently does not provide them a viable means of participation for any portion involving spatial analysis. One B/LV participant said on the phone that the maps were “very well put-together, just really fancy. I think that we’re so used to seeing rudimentary maps.” Natural hazard mitigation planning does not currently include tactile maps a matter of course. However, this study demonstrates that the collaborative potential of even very simple maps warrants further investigation.
In terms of braille literacy, this study’s participants were less representative of the general B/LV population than they could have been. Recruitment advertisements for this study listed a requirement of at least some braille literacy, and thus every B/LV participant reported that they were at least “mostly comfortable” with reading braille. This is in stark contrast with national braille literacy statistics: as of 2009, as many as 90% of the legally blind people residing in the United States cannot read braille (National Federation of the Blind 2009), and although this exact figure is questioned, the consensus remains that braille is not taught or learned as widely as it could be with additional research and support (Graves 2018).
The number of participants in this study was limited due in part to the challenges associated with coordinating the schedules of three different people simultaneously for each session. Future research would likely benefit from a larger sample size in order to reach thematic saturation (Guest, Namey, and Chen 2020) and draw more substantive connections between the qualitative and quantitative results. Additionally, a real-world NHMP scenario would typically entail working with a large group of people, all of whom will have the capacity to influence a user’s reading of the tactile map. It is also possible that, in a group with more people who have worked with B/LV individuals in the past, or in a group with more B/LV individuals, or both, the outcomes may have changed.
Some B/LV and sighted participants expressed confusion or even frustration with the amount of information being provided by the maps, compared to the amount of information that they felt was necessary to answer certain questions. As discussed earlier, the choice to give participants maps of communities that they were unfamiliar with was a deliberate one in order to help ensure that study results focused on the maps themselves. However, the ultimate goal of this research is to develop means by which people with disabilities can contribute insights, perspectives, and expertise to the NHMP process, all of which would help to augment any limitations imposed by the tactile format and in the absence of additional modalities (Golledge 2005), so it would follow that research should examine that process of knowledge contribution. Future experiments could also examine the impact of additional modalities such as, for instance, the addition of alt text to be used alongside a tactile map.
Additionally, sighted participants in a real-world scenario would have access to full FIRMs and other maps. Simplified tactile maps and their Latin script equivalents were used here to reduce the number of overall variables under consideration, but future research will need to address the interactions between collaborators using visual FIRMs alongside those using tactile FIRMs, or other visual maps alongside their tactile equivalents.
The dynamics of flooding were noted to be confusing to certain participants. This is unsurprising given that flooding is indeed a complex phenomenon, and few participants had any previous experience with disaster planning. Indeed, FEMA supplies 116 information packets for those working to create NHMPs, such as guidance specifically regarding coastal structures, or for analyzing overland wave propagation. While this study gave participants a very broad overview of the flooding dynamics that they would be analyzing in their maps, future studies on the topics presented here could give participants additional information on flood dynamics and/or more time to learn this information, or even the chance to practice flood analysis.
Because the maps that were used by participants did not actually represent the communities that they live in, the participants did not necessarily have anything at stake personally that might have influenced their decision making. While outside experts may be called upon to work on real-world NHMPs, most people involved will be local officials, stakeholders and other residents (Federal Emergency Management Agency 2013), meaning that the people working with flood maps will have personal investments (financial, emotional, political, etc.) in whatever is being represented on the maps, almost certainly influencing their decision-making calculus. An experiment similar to what is described in this paper, wherein participants are given maps of their own communities—perhaps gamified so that the outcomes of their plans would be consequential in some regard—could prove to be highly generative.
The fact that participants had relatively low confidence in other B/LV people being able to use these maps suggests that further research should be conducted not in symbol legibility, but symbol learning. Theoretical frameworks for map use and cartographic communication, especially DiBiase’s “swoopy” diagram (1990), or MacEachren’s “cartography cubed” diagram (1995), have become extremely influential in cartographic scholarship (Çöltekin, Janetzko, and Fabrikant 2018), but these frameworks assume a visual mode of interaction. It may be possible to reconfigure them simply by removing any instances of the word “visual” given that the remaining elements (exploration, synthesis, interaction, etc.) certainly remain pertinent to tactile map use. Future research could investigate how models of visual map use apply (or do not apply) to tactile map use, potentially generating frameworks specific to tactile map use. As of now, we simply do not know.
So how do B/LV map users acquire, process, and deploy knowledge about tactile maps? For visual maps, Roth (2012) summarizes and synthesizes a set of cartographic interaction primitives, meant to taxonomize the ways in which people interact with maps—for example, “identify,” or “compare,” or “correlate.” Semi-equivalent work does exist for non-visual maps, especially by the team of Simon Ungar, Mark Blades, and Christopher Spencer (e.g., Ungar et al. 1997; Ungar, Blades, and Spencer 1997; Ungar, Blades, and Spencer 2002), which provides a substantive foundation for future research. What the study described in this paper contributes to this earlier work is the addition of thematic spatial data as well as a collaborative context.
In addition to better understanding spatial learning through tactile media, we may well benefit from a “multivalent” approach to accessible tactile cartography research, as was done in this study, given that the resources needed for accessing and producing tactile maps are still very much constrained by a number of social, political, and logistical factors (compared to equivalent visual maps which are more or less unburdened by any significant barriers to access).
Ensuring the autonomy of B/LV individuals remains a worthwhile goal, and better understanding how tactile maps are used can aid in pursuit of that goal. There are so many instances in which maps serve as the focal point of a communal event, whether it’s readying a community for the next natural hazard, designating a wildlife preserve, or building a house. This study demonstrates that tactile maps likewise have this capacity to bring people together, especially those that may otherwise be left out.
Aldrich, Frances K., and Linda Sheppard. 2001. “Tactile Graphics in School Education: Perspectives from Pupils.” British Journal of Visual Impairment 19 (2): 69–73. https://doi.org/10.1177/026461960101900204.
Arnör, Anders. 2014. “WBU Report on Disability-Inclusive Disaster Management.” World Blind Union. https://worldblindunion.org/wp-content/uploads/2020/10/WBU-Humanitarian-Crises-Report-Nov-2014.pdf.
Botha, Monique, Jacqueline Hanlon, and Gemma Louise Williams. 2023. “Does Language Matter? Identity-First Versus Person-First Language Use in Autism Research: A Response to Vivanti.” Journal of Autism and Developmental Disorders 53: 870–878. https://doi.org/10.1007/s10803-020-04858-w.
Braille Authority of North America. 2010. “Guidelines and Standards for Tactile Graphics.” Braille Authority of North America and the Canadian Braille Authority. http://www.brailleauthority.org/tg/.
Brittell, Megen E., Amy K. Lobben, and Megan M. Lawrence. 2018. “Usability Evaluation of Tactile Map Symbols across Three Production Technologies.” Journal of Visual Impairment & Blindness 112 (6): 745–758. https://doi.org/10.1177/0145482X1811200609.
Brown, Greg. 2012. “Public Participation GIS (PPGIS) for Regional and Environmental Planning: Reflections on a Decade of Empirical Research” URISA Journal 25 (2): 5–16.
Brown, Stephen. 2009. “Access Is Not a Text Alternative.” Journal of Museum Education 34 (3): 223–234. https://doi.org/10.1080/10598650.2009.11510639.
Buttenfield, Barbara. 1999. “Usability Evaluation of Digital Libraries.” Science & Technology Libraries 17 (3–4): 39–59. https://doi.org/10.1300/J122v17n03_04.
Cole, Harrison. 2021. “Tactile Cartography in the Digital Age: A Review and Research Agenda.” Progress in Human Geography 45 (4): 834–854. https://doi.org/10.1177/0309132521995877.
Çöltekin, Arzu, Halldór Janetzko, and Sara Fabrikant. 2018. “Geovisualization.” In Geographic Information Science & Technology Body of Knowledge (2nd Quarter 2018 Edition). https://doi.org/10.22224/gistbok/2018.2.6.
Coughlan, James M., and Joshua Miele. 2017. “AR4VI: AR as an Accessibility Tool for People with Visual Impairments.” In 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), edited by Wolfgang Broll, et al., 288–292. Piscataway, NJ: IEEE.
DiBiase, David. 1990. “Visualization in the Earth Sciences at Penn State.” Earth and Mineral Sciences 59 (2): 13–18.
Dunn, Dana S., and Erin E. Andrews. 2015. “Person-First and Identity-First Language: Developing Psychologists’ Cultural Competence Using Disability Language.” American Psychologist 70 (3): 255–264. https://doi.org/10.1037/a0038636.
Edman, Polly. 1992. Tactile Graphics. New York: Amer Foundation for the Blind.
Elwood, Sarah. 2006. “Critical Issues in Participatory GIS: Deconstructions, Reconstructions, and New Research Directions.” Transactions in GIS 10 (5): 693–708. https://doi.org/10.1111/j.1467-9671.2006.01023.x.
Engel, Christin, and Gerhard Weber. 2021. “Analyzing the Design of Tactile Indoor Maps.” In Human-Computer Interaction – INTERACT 2021, edited by Carmelo Ardito et al., 434–443. https://doi.org/10.1007/978-3-030-85623-6_26. Cham, Switzerland: Springer.
Federal Emergency Management Agency. 2013. “Local Mitigation Planning Handbook.” https://www.fema.gov/sites/default/files/2020-06/fema-local-mitigation-planning-handbook_03-2013.pdf.
Gaillard, Jean-Christophe, and Emmanuel A. Maceda. 2009. “Participatory Three-Dimensional Mapping for Disaster Risk Reduction.” Participatory Learning and Action 60: 109–118. http://pubs.iied.org/14573IIED/.
Gerber, Elaine. 2009. “Describing Tragedy: The Information Access Needs of Blind People in Emergency-Related Circumstances.” Human Organization 68 (1): 73–81. https://doi.org/10.17730/humo.68.1.tm17684j7u301668.
Ghodke, Uttara, Lena Yusim, Sowmya Somanath, and Peter Coppin. 2019. “The Cross-Sensory Globe: Participatory Design of a 3D Audio-Tactile Globe Prototype for Blind and Low-Vision Users to Learn Geography.” In Proceedings of the 2019 on Designing Interactive Systems Conference - DIS ’19, edited by Steve Harrison, Shaowen Bardzell, Carman Neustaedter, and Deborah Tatar, 399–412. New York: Association for Computing Machinery. https://doi.org/10.1145/3322276.3323686.
Giudice, Nicholas A., Benjamin A. Guenther, Nicholas A. Jensen, and Kaitlyn N. Haase. 2020. “Cognitive Mapping Without Vision: Comparing Wayfinding Performance After Learning From Digital Touchscreen-Based Multimodal Maps vs. Embossed Tactile Overlays.” Frontiers in Human Neuroscience 14. https://doi.org/10.3389/fnhum.2020.00087.
Golledge, Reginald G. 2005. “Reflections on Procedures for Learning Environments Without the Use of Sight.” Journal of Geography 104 (3): 95–103. https://doi.org/10.1080/00221340508978622.
Good, Gretchen A. 2015. “Emergency Plans in Schools: Individualised Disaster Planning for Students with Impaired Vision.” Journal of the South Pacific Educators in Vision Impairment 8 (1): 17.
Goodrick, B. E. 1984. “Tactual Mapping in Australia.” Cartography 13 (4): 270–275. https://doi.org/10.1080/00690805.1984.10439506.
Graves, Adam. 2018. “Braille Literacy Statistics Research Study: History and Politics of the ‘Braille Reader Statistic’: A Summary of AFB Leadership Conference Session on Education.” Journal of Visual Impairment & Blindness 112 (3): 328–331. https://doi.org/10.1177/0145482X1811200314.
Guest, Greg, Emily Namey, and Mario Chen. 2020. “A Simple Method to Assess and Report Thematic Saturation in Qualitative Research.” PLOS ONE 15 (5): e0232076. https://doi.org/10.1371/journal.pone.0232076.
Hamraie, Aimi. 2013. “Designing Collective Access: A Feminist Disability Theory of Universal Design.” Disability Studies Quarterly 33 (4). https://doi.org/10.18061/dsq.v33i4.3871.
Hasan, Sayed Kamrul, and Terje Gjøsæter. 2021. “Screen Reader Accessibility Study of Interactive Maps.” In Universal Access in Human-Computer Interaction. Design Methods and User Experience, edited by Margherita Antona and Constantine Stephanidis, 232–249. Cham, Switzerland: Springer International Publishing. https://doi.org/10.1007/978-3-030-78092-0_15.
Henly-Shepard, Sarah, Steven A. Gray, and Linda J. Cox. 2015. “The Use of Participatory Modeling to Promote Social Learning and Facilitate Community Disaster Planning.” Environmental Science & Policy 45: 109–122. https://doi.org/10.1016/j.envsci.2014.10.004.
Hennig, Sabine, Fritz Zobl, and Wolfgang W. Wasserburger. 2017. “Accessible Web Maps for Visually Impaired Users: Recommendations and Example Solutions.” Cartographic Perspectives 88: 6–27. https://doi.org/10.14714/CP88.1391.
Henstra, Daniel, Andrea Minano, and Jason Thistlethwaite. 2019. “Communicating Disaster Risk? An Evaluation of the Availability and Quality of Flood Maps.” Natural Hazards and Earth System Sciences 19 (1): 313–323. https://doi.org/10.5194/nhess-19-313-2019.
Jankowski, Piotr, and Timothy Nyerges. 2001. “GIS-Supported Collaborative Decision Making: Results of an Experiment.” Annals of the Association of American Geographers 91 (1): 48–70. https://doi.org/10.1111/0004-5608.00233.
Jehoel, Sandra, Don McCallum, Jonathan Rowell, and Simon Ungar. 2006. “An Empirical Approach on the Design of Tactile Maps and Diagrams: The Cognitive Tactualization Approach.” British Journal of Visual Impairment 24 (2): 67–75. https://doi.org/10.1177/0264619606063402.
Jordan, Jillian J., Moshe Hoffman, Martin A. Nowak, and David G. Rand. 2016. “Uncalculating Cooperation Is Used to Signal Trustworthiness.” Proceedings of the National Academy of Sciences 113 (31): 8658–8663. https://doi.org/10.1073/pnas.1601280113.
Katzschmann, Robert K., Brandon Araki, and Daniela Rus. 2018. “Safe Local Navigation for Visually Impaired Users With a Time-of-Flight and Haptic Feedback Device.” IEEE Transactions on Neural Systems and Rehabilitation Engineering 26 (3): 583–593. https://doi.org/10.1109/TNSRE.2018.2800665.
Kirkpatrick, Andrew, Joshue O’Connor, Alastair Campbell, and Michael Cooper. 2018. “Web Content Accessibility Guidelines (WCAG) 2.1.” World Wide Web Consortium. June 5, 2018. https://www.w3.org/TR/WCAG21/.
Koski, Christian, Mikko Rönneberg, Pyry Kettunen, Aurelija Armoškaitė, Solvita Strake, and Juha Oksanen. 2021. “User Experiences of Using a Spatial Analysis Tool in Collaborative GIS for Maritime Spatial Planning.” Transactions in GIS 25 (4): 1809–1824. https://doi.org/10.1111/tgis.12827.
Lobben, Amy K., Megen E. Brittell, and Nicholas A. Perdue. 2015. “Inclusive Cartographic Design: Overcoming Ocular-Centric Cartographies.” In Cartography - Maps Connecting the World: 27th International Cartographic Conference 2015 - ICC2015, edited by Claudia Robbi Sluter, Carla Bernadete Madureira Cruz, and Paulo Márcio Leal de Menezes, 89–98. Cham, Switzerland: Springer International Publishing. https://doi.org/10.1007/978-3-319-17738-0_7.
MacEachren, Alan M. 1995. How Maps Work: Representation, Visualization, and Design. New York: Guilford Publications.
———. 2001. “Cartography and GIS: Extending Collaborative Tools to Support Virtual Teams.” Progress in Human Geography 25 (3): 431–444. https://doi.org/10.1191/030913201680191763.
McCallum, Don, Kafeel Ahmed, Sandra Jehoel, Snir Dinar, and Derek Sheldon. 2005. “The Design and Manufacture of Tactile Maps Using an Inkjet Process.” Journal of Engineering Design 16 (6): 525–544. https://doi.org/10.1080/09544820500273946.
Miele, Joshua. 2004. “Tactile Map Automated Production (TMAP): Using GIS Data to Generate Braille Maps.” 2004 Conference Proceedings, CSUN Technology And Persons With Disabilities Conference. http://www.csun.edu/~hfdss006/conf/2004/proceedings/246.htm.
Moore, Don A., and Paul J. Healy. 2008. “The Trouble with Overconfidence.” Psychological Review 115 (2): 502–517. https://doi.org/10.1037/0033-295X.115.2.502.
Muredda, Angelo. 2012. “Fixing Language: ‘People-First’ Language, Taxonomical Prescriptivism, and the Linguistic Location of Disability.” The English Languages: History, Diaspora, Culture 3. https://jps.library.utoronto.ca/index.php/elhdc/article/view/17060.
National Federation of the Blind. 2009. “The Braille Literacy Crisis in America: Facing the Truth, Reversing the Trend, Empowering the Blind.” Jernigan Institute. https://nfb.org/images/nfb/documents/pdf/braille_literacy_report_web.pdf.
Papadopoulos, Konstantinos, Marialena Barouti, and Eleni Koustriava. 2018. “Differences in Spatial Knowledge of Individuals With Blindness When Using Audiotactile Maps, Using Tactile Maps, and Walking.” Exceptional Children 84 (3): 330–343. https://doi.org/10.1177/0014402918764300.
Pelzer, Peter, Stan Geertman, Rob van der Heijden, and Etiënne Rouwette. 2014. “The Added Value of Planning Support Systems: A Practitioner’s Perspective.” Computers, Environment and Urban Systems 48: 16–27. https://doi.org/10.1016/j.compenvurbsys.2014.05.002.
Ronoh, Steve, JC Gaillard, and Jay Marlowe. 2017. “Children with Disabilities in Disability-Inclusive Disaster Risk Reduction: Focussing on School Settings.” Policy Futures in Education 15 (3): 380–388. https://doi.org/10.1177/1478210317694500.
Roth, Robert E. 2012. “Cartographic Interaction Primitives: Framework and Synthesis.” The Cartographic Journal 49 (4): 376–395. https://doi.org/10.1179/1743277412Y.0000000019.
Rowell, Jonathan, and Simon Ungar. 2003. “The World of Touch: An International Survey of Tactile Maps. Part 1: Production.” British Journal of Visual Impairment 21 (3): 98–104. https://doi.org/10.1177/026461960302100303.
Rumsey, David. 2015. “Atlas for the Blind, 1837.” Cartographic Perspectives 80: 46–50. https://doi.org/10.14714/CP80.1312.
Shi, Lei, Yuhang Zhao, Ricardo Gonzalez Penuela, Elizabeth Kupferstein, and Shiri Azenkot. 2020. “Molder: An Accessible Design Tool for Tactile Maps.” In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, edited by Regina Bernhaupt et al., 1–14. New York: Association for Computing Machinery. https://doi.org/10.1145/3313831.3376431.
Sieber, Renee. 2006. “Public Participation Geographic Information Systems: A Literature Review and Framework.” Annals of the Association of American Geographers 96 (3): 491–507. https://doi.org/10.1111/j.1467-8306.2006.00702.x.
Snow, Kathie. 2007. “People First Language.” http://www.inclusioncollaborative.org/docs/Person-First-Language-Article_Kathie_Snow.pdf.
Stough, Laura M., and Donghyun Kang. 2015. “The Sendai Framework for Disaster Risk Reduction and Persons with Disabilities.” International Journal of Disaster Risk Science 6 (2): 140–49. https://doi.org/10.1007/s13753-015-0051-8.
Streeter, Larry. 2010. “The Continuing Saga of People-First Language.” Braille Monitor 53 (5). https://nfb.org//sites/default/files/images/nfb/publications/bm/bm10/bm1005/bm100509.htm.
Thevin, Lauren, Christophe Jouffrais, Nicolas Rodier, Nicolas Palard, Martin Hachet, and Anke Brock. 2019. “Creating Accessible Interactive Audio-Tactile Drawings Using Spatial Augmented Reality.” In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces, edited by Bongshin Lee, et al., 17–28. New York: Association for Computing Machinery. https://doi.org/10.1145/3343055.3359711.
Toyoda, Wataru, Eiji Tani, Susumu Oouchi, and Masaki Ogata. 2020. “Effects of Environmental Explanation Using Three-Dimensional Tactile Maps for Orientation and Mobility Training.” Applied Ergonomics 88: 103177. https://doi.org/10.1016/j.apergo.2020.103177.
Ungar, Simon, Angeles Espinosa Bayal, Mark Blades, Espernaza Ochaita, and Christopher Spencer. 1997. “Use of Tactile Maps by Blind and Visually Impaired People.” Cartographic Perspectives 28: 4–12. https://doi.org/10.14714/CP28.686.
Ungar, Simon, Mark Blades, and Christopher Spencer. 1997. “Strategies for Knowledge Acquisition from Cartographic Maps by Blind and Visually Impaired Adults.” The Cartographic Journal 34 (2): 93–110. https://doi.org/10.1179/caj.1997.34.2.93.
———. 2002. “Tactile Maps and a Test of the Conjoint Retention Hypothesis.” In Diagrammatic Representation and Reasoning, edited by Michael Anderson, Bernd Meyer, and Patrick Olivier, 141–154. London: Springer. https://doi.org/10.1007/978-1-4471-0109-3_8.
UNISDR. 2015. “Sendai Framework for Disaster Risk Reduction 2015 - 2030.” Sendai, Japan: United Nations International Strategy for Disaster Reduction. https://www.preventionweb.net/files/43291_sendaiframeworkfordrren.pdf.
Vaughn, Lisa M., and Farrah Jacquez. 2020. “Participatory Research Methods – Choice Points in the Research Process.” Journal of Participatory Research Methods 1 (1): 13244. https://doi.org/10.35844/001c.13244.