How Effective are Tabletop Role-Playing (Serious) Games in Understanding and Validating the Predictive Capabilities of Disaster Response Agent-based Models?

Corresponding author: Nimisingha Jacob Amakama, nimisingha.amakama@mines-ales.fr

Introduction

Tabletop role-playing games (TRPGs) are collaborative exercises in which participants take on the roles of characters in a fictional game world and are guided by a human supervisor called a “Game Master.” The Game Master (GM) acts as both storyteller and guide, working with players to help shape the game world. Creativity is central to TRPGs and influences both the players and the game master (Katifori et al., 2022). These serious games aim to train or prepare users in a specific area, such as disaster management, thereby improving the preparedness of emergency and crisis responders (Guiamalodin et al., 2024; Kwadwo & Morkla, 2024; Tanguid & Tanguid, 2023; Simões-Marques et al., 2020). Tabletop role-playing games (TRPGs) provide instructors with the opportunity to monitor group dynamics and decision-making activities while serving as immersive and effective tools for promoting soft skills (Linehan et al., 2009; Suguitan & Natividad, 2022). They provide a valuable environment for improving the understanding of abstract concepts such as drought and water-related challenges while improving communication and conflict resolution skills (Alejandria et al., 2023; Pasinetti et al., 2015). Serious games have also been integrated into educational contexts to improve learning outcomes and promote social behaviour. An example of this is the recycling of knowledge through immersive virtual reality environments (Poděbradská et al., 2020).

Because they collect data from player interaction data, they can accurately predict knowledge changes based on player interactions in the game. Serious role-playing games are also effective tools for understanding and validating the predictive capabilities of agent-based models in scenarios with limited computational resources (Alonso-Fernández et al., 2021; 2019; C. Liu et al., 2021). Games such as ANYCARE, a role-playing game (RPG), help players understand the challenges and intricacies of dealing with weather-related emergencies, thereby increasing their confidence in making timely decisions (Terti et al., 2019; Raimi et al., 2022). Cheng et al. (2020) found that role-playing games (RPGs) are valuable pedagogical tools for promoting students' understanding and responsibility, as well as their ability to reach consensus in multiple role scenarios. In the context of disaster response, RPGs can serve as useful tools for disaster prevention and promote communication between public and academic stakeholders (Sakai et al., 2019). Roleplay exercises such as RAMSETE provided participants with the opportunity to discuss and challenge concepts, policies, and practices in the areas of disaster risk reduction, management, and adaptation to climate change (Fleming et al., 2020; Deinkuro et al., 2021; Olalekan et al., 2018; 2022; Sanchez et al., 2021). A study by Hosseini et al. (2022) revealed that disaster and crisis games are more efficient than case-based approaches for improving nursing students' understanding and behavioral skills in crisis management. The use of role-playing in the development and validation of agent-based models offers several advantages. These benefits include adapting game parameters and player demographics to real-world scenarios, incorporating experimental design into data collection processes, and integrating operational validation phases into modeling procedures (Utomo et al., 2022).

This study examines the effectiveness of integrating role-playing to engage both healthcare professionals and nonexperts (students) in understanding and validating an agent-based model of disaster response strategies. This research is significant because it introduces a method to improve model validation and capture the complexity of agent-based models by leveraging the formal and informal knowledge and behavior of stakeholders, thereby bridging the gap between theoretical modeling and real-world situations. Hence, this study aims to evaluate the effectiveness of utilizing disaster response tabletop role-playing games (DRTRPGs) in facilitating the understanding of disaster management concepts and in validating the predictive capabilities of an agent-based model for a hypothetical disaster setting. To achieve the study’s goal, the following objectives were proposed: (1) Evaluate the knowledge enhancement achieved through the training and implementation of the DRTRPG via multiple-choice pre- and post-assessment tests; and (2) To visually and/or statistically compare the outcomes of the DRTRPG against the ABMS model outcomes (validation) by examining the impact of ambulance deployment policies on mortality outcomes. The authors assert that the lessons learned from this research can be integrated into disaster preparedness and response strategies, resulting in improved training, preparedness, and coordinated disaster responses. The subsequent aspects of this study consisted of a literature review and the methods and materials adopted for the study. The final two sections present the results and discussion and draw conclusions, including limitations and recommendations for future studies.

Literature Review

In a tabletop role-playing game (TRPG), players create characters and go on adventures under the guidance of a game master (Guzdial et al., 2020). Role-playing games (RPGs) are structured to instruct, guide, and equip players to overcome specific challenges by simulating real-world scenarios (“The Functions of Role-playing Games: How Participants Create Community, Solve Problems and Explore Identity,” 2011; Rye & Aktas, 2022). Disaster response tabletop role-playing games have a variety of uses in improving preparedness and response skills. With the help of these emergency simulation games, players can test their decision-making skills and practice response plans and better understand their role in disaster situations (McCourt & Watson, 2023). Suleman et al. (2022) assert that TTRPGs significantly improve the understanding of disaster management among healthcare workers, with an emphasis on strategies, communication, and collaboration. Additionally, tabletop exercises have been shown to improve nursing students' knowledge and disposition during disaster exercise, making them useful tools for improving disaster response skills (Ademiluyi et al., 2010; Husna et al., 2020; Raimi et al., 2020; Raimi et al., 2019).

Agent-based simulation, on the other hand, is a computational modeling approach that positions autonomous agents within an environment to investigate emergent outcomes resulting from interactions between agents and their surroundings (Y. Liu et al., 2023). McDonald and Osgood (2023) stated that these simulations are useful for deciphering macrolevel phenomena by studying individual-level interactions in complex systems. They offer advantages such as scalability, flexibility in representing diversity and the ability to include static and dynamic network contexts in addition to spatial aspects (Antelmi et al., 2022). Ogie et al. (2022) highlighted the potential benefits of using ABM for collaborative translation in crisis communication. Role-playing games and agent-based models together provide a vantage point for exploring a variety of phenomena. According to (Guyot et al., 2007), a combination of ABM and RPGs can serve three main purposes: (i) assisting stakeholders in decision-making in challenging scenarios; (ii) promoting group learning and negotiation between participants; and (iii) validating and refining a model by observing stakeholder behavior. In regard to modeling a specific case, RPG players are usually selected from among the key stakeholders at the case study site (D’Aquino & Bah, 2014). However, if the research goal includes hypothesis testing and group comparisons, participants may be selected from a broader population (Meijer et al., 2006).

As an additional method for validating agent-based models, role-playing games (RPGs) facilitate validation triangulation by observing agents' decision-making within the game environment (Villamor et al., 2023). Utomo et al. (2022) argued that role-playing games (RPGs) provide valuable insights for model validation by allowing observation of player behaviour that can be contrasted with agent behaviour in a model. This was supported by Pira et al. (2017), who asserted that RPGs could improve the robustness and relevance of agent-based models by enabling stakeholder-driven validation, especially in complicated situations such as urban freight transport decision-making. However, several studies suggest that validating ABMs using classical empirical methods is very challenging, as there is typically a paucity of data and ABMSs are based on future predictions (Marks, 2007; Takadama, Kawai and Koyama, 2008; Anand et al., 2016; Onggo & Karatas, 2016; Gore et al., 2016; Utomo et al., 2022). These findings reveal that the classical empirical validation of agent-based models is challenging because of features such as path dependence, emergence, and complex interactions, in addition to the lack of microlevel data. Other aspects cited by the researchers border the complexity of social processes and the stochasticity of the simulation. This may lead to disagreement in the results compared to an experiment that is well controlled.

However, Robinson et al. (2007) argue that models should be subjected to validation using various techniques and approaches and emphasize that the validity of ABMs depends on the accessibility and quality of the data for model calibration and validation. They also advocate the use of RPG-based validation to test and validate ABMs; however, they caution that the results should be verified with historical data and validated using alternative methods. According to Dubois et al. (2013), the settings of games can influence participants' attitudes and behaviour patterns and subsequently affect the outcomes of role-playing games. In addition, the interchangeability of the terms “player” and “agent” is debated as part of the discussion about combining role-playing games and agent-based models (“Introduction to Agent-Based Modeling,” 2019). Squazzoni et al. (2020) suggested that realism and validation in role-playing games (RPGs) depend on the assumptions and rules built into the game design. They suggested that if a role-playing game is well designed on the basis of the basic rules and assumptions of empirical data, it can realistically simulate human behavior and decision-making, thereby improving the validity of the validation process. These studies support the need for the current study by highlighting the context of disaster response strategies for disaster preparedness, training, and ABM validation. To achieve our research objectives, this study evaluates the effectiveness of using a disaster relief tabletop role-playing game for improving player knowledge and validating the results of an ABM by assessing the impact of ambulance policy on mortality outcomes—a topic not specifically addressed in previous studies.

Materials and Methods

Research Methodology: DRTRPG Design and Implementation

This research falls into the area of experimental research, particularly at the interface of simulation and educational research in the context of disaster protection strategies. This study aimed to improve the understanding of process and knowledge representation in agent-based models (ABMs) and role-playing games (RPGs) through the practical and interactive involvement of participants, including both experts and nonexperts. In addition, the present research addresses a specific gap in the literature by assessing and validating the impact of ambulance policy on mortality rates to improve the understanding of the ABMS model through the use of a disaster response tabletop role-playing game. As a validation strategy for this study, a scenario-based disaster response tabletop role-playing game (DRTRPG) was developed, as suggested by Ligtenberg et al. (2010) for the ABM model validation method using the RPG. The process of transforming ABMs into RPGs for the purpose of validation is summarized below:

• To develop a basic ABMS model,
• Develop a role-playing game based on the basic ABMS model in which players can engage in tasks that replicate the agent’s actions;
• Guide players in completing game tasks,
• The RPG was used to collect the data.
• The ABMS results are analyzed and contrasted with the RPG results.

DRTRPG exercise is divided into two phases. First, participants completed preparatory training and participated in exercises to familiarize themselves with the DRTRPG concept in a group setting. The goal of this phase is to refine both the game and the ABMS model components. In phase 2, response results are generated, analyzed and compared among doctors, master's students and the ABMS basic model in a group setting. The DRTRPG, developed between 2023 and 2024, includes two exercises with three training sessions each. Tabletop gaming sessions take place in a standard classroom at the IMT Mines Ales Clavières location and can comfortably accommodate 10 to 15 participants. With this setup, each group experiences the same crisis response scenario throughout the game. Prior to participating in the DRTRPG exercise, participants' understanding of disaster management and health-related decision-making was assessed using a multiple-choice pregame questionnaire. A simulation is then run that represents components of the ABMS model within the DRTRPG, creating a scenario in which players collaborate and make decisions. During the game, each group records the time spent on tasks while observers rate the players' actions. This approach aims to quickly collect large-scale data. Following the in-game activities, a debriefing will take place, along with a postgame questionnaire and an observer rating assessment completed by players and observers, respectively.

Participants

This study involved participants with diverse backgrounds and experiences playing the DRTRPG game in teams assigned specific roles (e.g., surgeons, triage nurses, emergency responders) to manage a hypothetical explosion event and provide optimal medical care to ensure supply. Provide care in a collaborative, interactive role-play environment. As part of the first practice session, the game was first tested in four groups with a total of 15 players (including observers). In phase 2, two groups with 11 participants were used to compare the actions of students and doctors. The researcher claims that the DRTRPG improves the validation and effectiveness of disaster response strategies by integrating agent-based models and tabletop role-playing techniques.

Concepts, principles, and experimental setup

Tabletop role-playing games designed for disaster response are helpful resources for confirming the accuracy of agent-based models used in disaster management. PREDIS and other similar games facilitate the decision-making processes of experts and nonexperts alike, allowing quicker and more precise responses (Rye & Aktas, 2022). According to Utomo et al. (2022), RPG data are used to validate ABMS models at both the macro- and microlevels to ensure that the model accurately represents real-world knowledge and processes. The graphical user interface of the developed ABMS simulation model underlying the design concept of the DRTRPG is shown in Figure 1.

The graphical user interface (GUI) of the ABMS model represents a simulated (720 runs) crisis scenario that is integral to the design of the disaster response role playing game (DRTRPG). Within the DRTRPG, participants assume various roles as agents responding to an explosion in the oil and gas sector, while regional and mobile field hospitals collaborate to determine optimal response strategies and provide care for the injured. These roles simplify the elements of the ABMS simulation. The development of the game required careful consideration of the game elements, roles, and simulations. Table 1 outlines the guiding principles of DRTRPG design and serves as a three-step guide for tabletop role-playing serious game exercises with a focus on disaster response and healthcare simulation training, education, and validation.

Experimental setup

Five essential progressive steps constitute the procedure, which are listed as follows. (i) A summary of the objectives, rules, and plot of the DRTRPG to the players.; (ii) role assignment; (iii) simulation

activities (in-game action); (iv) debriefing of the players based on the game; and (v) gathering of player and observer feedback, as presented in Table 2.

The GM assists players by introducing important plot points, providing hints, and initiating relevant conversations in accordance with the DRTRPG rules. Observers score the performance, both during in-game activities and provide an overall assessment of the game design and implementation. According to the researcher's findings, a playgroup with eight to sixteen participants would be more ideal; if the goal is to produce better datasets for validating the ABMS model, further activities are needed.

The DRTRPG guidelines

Based on traditional disaster-themed role-playing game guidelines, the DRTRPG runs according to the following rules:

• Prior to making decisions, only players on the same team are required to communicate and collaborate with their group.
• The MoD/GM provides instructions, directions, and clues for the game, and the observers observe the players and document their observations.
• Each group's discussion will be centered only on the emergency response scenario that will require prompt decision-making. There are time limits and consequences (mortality outcomes) for each in-game task to promote deliberative decision-making and boost game realism.
• Limited resources, such as ambulances or operating rooms, are incorporated into the game's design, which the players need to thoughtfully allocate to reduce mortality outcomes.

DRTRPG implementation materials

To succeed, materials that improve experience must be chosen for any tabletop role-playing game. The DRTRPG's resources are made up of various materials, each of which serves a distinct function within the ecosystem of the game. The following are the basic steps of the exercise.

• Maps of the disaster response scenario, depending on the in-game tasks
• PPT presentation slides
• Pencils/pens/stopwatches
• Pre/post assessment test/observer performance evaluation forms

Figure 2 shows a representation of the in-action phases in the implemented game setup for the disaster response tabletop role-playing game. In this configuration, each group is assigned an observer, while the GM continually provides direction to the process by emphasizing the game’s story and ensuring that the game mechanics are maintained. In this exercise, two teams (A and B) compete by completing a series of similar but challenging tasks involving 18 emergency challenges in a disaster scenario with varying patient numbers, response resources and severity indices.

Metrics, data generation and analysis

The mortality outcome of the disaster scenario team decisions was assessed using the DRTRPG activity assessment chart (Table 3). Although it promotes competition between groups for efficient time management, the points awarded do not affect the outcome of the game. The estimated six minutes required to complete a task (the in-game activity contained 18 tasks) corresponds to one tick in the ABMS model.

Using this scoring table and the recorded reaction times for each game task of each group, it was possible to estimate mortality outcomes based on the investigated ambulance policy strategies. An ambulance policy is a set of rules that control performance, patient care, ambulance dispatch, emergency response times, and other elements of an emergency response system. The two studied ambulance strategies are “always go to RH”, which is based on a traditional dispatching approach in which ambulances are dispatched based on patient severity and availability, with only the Regional Hospital (RH) available. The second strategy (“alternate”) involves a more dynamic approach involving the use of a mobile field hospital (MFH) to strategically support the RH in the scenario. The authors claim that shorter response times could be a sign of better decision making and team coordination. With NetLogo 6.3, large datasets could be generated by modeling and simulating complex systems. The BehaviorSpace interface, along with Python's Pandas library, gave the researchers access to a few useful libraries for working with and exploring the dataset. These tools were used for dataset cleaning, information filtration and aggregation. The analysis criteria for the ABMS and DRTRPG data are on our GitHub: https://github.com/Nimisingha/Ambulance_Policy_RPG/blob/main/codes_ambulance_policy_mortality_outc ome.ipynb and https://github.com/Nimisingha/Ambulance_Policy_RPG/blob/main/codes_DRTRPG.ipynb, respectively.

Results and Discussion

Results

A total of 12 participants (divided into four groups), with 2 observers, were involved in the preliminary 2023 game test. (tabletop simulation experiment). The participants were informed of their background and experience as follows: three PhD students, one medical doctor, one emergency responder, one postdoctoral student, and nine master's degree students. Ten participants (in two groups), with 2 observers, participated in the final DRTRPG activity held in March 2024.

Pre, post, and observer performance evaluation

The results of the assessment tests showed that medical doctors’ (Group A) knowledge of disaster management and health interactions increased significantly (11.5%). However, only a slight increase in knowledge (3.1%) was observed among master's degree students. These results were generated based on correctly answered questions in the multiple-choice assessment test. According to the observers' assessments, both teams also showed an elevated level of involvement and communication during the game. Nevertheless, both groups had some difficulty managing their time, suggesting that they could do better. According to Figures 4a and 4b, both teams were judged to be competitive.

While these results illustrate the potential and utility of TRPG design and implementation for disaster response class problems, further development is required to provide a more accurate representation of the complexity of complex systems.

ABMS model validation: Effects of the ambulance policy on mortality outcomes

An ambulance policy is a collection of policies that govern patient care, ambulance performance, deployment, and other elements of emergency response systems. The boxplot shown in Figure 5(a) represents the distribution of data from the ABMS model, which simulates the behavior and interactions of autonomous agents using complex system computational techniques. Figure 5(b) shows the results of the teams participating in the in-game activities based on the DRTRPG exercise, which use a more participatory and interactive simulation technique and a tabletop serious game to address aspects of the ABMS decision protocols. This assessment is based on two response strategies (“Always go RH” and “Alternative”).

The “always go to RH” ambulance policy is based on a traditional dispatching approach in which ambulances are dispatched based on patient severity and availability and only the Regional Hospital (RH) is operational. The “alternate” ambulance policy involves a more dynamic approach involving the deployment of a mobile field hospital (MFH) to strategically support the RH in the disaster response scenario. This study's approach to comparing the results of both methods aims to provide insights into the effectiveness of using role-playing methods in the validation of agent-based models.

Discussion

This study used a different approach than did the widely used traditional methods to validate and communicate the results of ABMS models. In this study, we compared the simulation results of an ABMS model (visual and statistical) with the results of a disaster response role-playing game rather than comparing the accuracy of the model results with those of an equivalent model in the real world. This participatory approach is intended to increase stakeholder confidence in the model's ability to predict an equivalent disaster situation.

For the simplified class case of the study, which is understandable to the role-play participants, the observers, and the modeler, the use of the group-based collaborative TRPG approach to data generation and validation of the ABM appears to provide promising results. According to the DRTRPG's findings, games can help emergency responders improve their disaster management skills, especially those with a medical background. Hence, the tests were of greater benefit to people with prior knowledge of medicine or disaster management, suggesting that the subjects were suitable for them. Players recognized that the game could be used as a teaching tool for disaster management and found it helpful and educational. The researcher emphasizes that when designing such games, it is important to take participants' backgrounds into account when optimizing their learning outcomes. Based on observer ratings (Figure 3) and insights into game activity, the results support a more nuanced assessment of team performance. Observers noted that both teams made poor use of time management, suggesting potential for development in future game design. While situational awareness was an area where Team A excelled, observers valued role connection slightly more than overall performance, which is why Team B received higher ratings. Overall, observers valued different competencies within each team, with no team emerging as leaders across all the metrics.

Based on a visual comparison of the results of the two strategies (Figure 5), the decision dynamics (the mortality results for the policies) follow a similar trend for both the ABMS model results and the DRTRPG results (alternative policies led to more lives saved). However, because the data distributions (shown at different plot scales) appear to be significantly different, basic statistics from the boxplots were used to examine the data in more detail. Regarding the ABMS results, the average mortality rate and standard deviation (SD) for the ABMS under the “always go to RH” policy was 23.22 and 8.98, respectively, which were greater than those under the “alternate” policy. The smaller mean of 15.85 and standard deviation (0.79) show that the policy has consistently saved lives. There was also some variation in mortality outcomes, as evidenced by the mean and standard deviation of the “alternate” policy, which were 19.75 and 8.27, respectively. The 25th, 75th and median percentiles, which are 20, 14 and 26, respectively, show a higher concentration of dataset points below the median.

According to the DRTRPG data, the “always go RH” policy had a slightly greater mean mortality (8.94) than did the “alternate” policy, with an S.D. of 2.6, indicating greater variability. On the other hand, the lives saved show good consistency, although with a mean of 29.72 and an S.D. of 2.86, above that of the “alternate” policy (33.0, 2.11). The results of the “alternate” policy showed an S.D. of 1.19, indicating lower variability and lower mean mortality (6.11) than those associated with the “always go RH” policy. In terms of the average number of lives saved, the “alternate” policy also resulted in a greater number of lives (33 lives saved), with an S.D. of 1.90, showing that this policy consistently saved more lives.

While the "alternate" policy has a lower average mortality than does the "always go to RH" policy (suggesting that the "alternate" policy may be a more effective strategy for reducing mortality/saving lives), the "always go to RH” policy produces more consistent results in saving lives, as evidenced by the lower S.D. for lives saved. For both policies, the DRTRPG results consistently showed lower mortality and more lives saved than did the ABMS results, with less variability in the results. This could mean that the ABMS overestimates variability or that the DRTRPG does not fully capture the range of variability encountered in the ABMS model. The maximum value for lives saved in the DRTRPG compared to the ABMS may indicate that the ABMS overestimates the lives saved.

The significant differences between the results of the ABMS model and the DRTRPG highlight the challenges in validating ABMS prediction capabilities through role-playing games. The limited number of runs (18) in the DRTRPG compared to the ABMS model (720) may contribute to this variation and reduce the statistical robustness of the DRTRPG result. This smaller sample size may also not accurately reflect the scope of the scenario, introducing uncertainty in the validation process. Additionally, players' knowledge, experience, and biases in the DRTRPG may have influenced the results, resulting in better decision-making than expected. To improve the validation, the researcher suggested adapting the ABM to better fit the interactive dynamics of the DRTRPG. Since the goal is to validate the ABMS protocol using the DRTRPG, it may be necessary to consider the DRTRPG only as a “best case” scenario. Further analysis could include comparing the models with real data (if available) to determine which strategy fits the actual results better and to examine the discrepancy between the two simulation results.

Disaster response tabletop role-playing games have been effectively utilized to validate agent-based models (De Rouck et al., 2023). These games function as simulations to assess healthcare workers' readiness for disasters and improve their understanding of disaster management (Rye & Aktas, 2022). Participatory simulation games such as the PREDIS have been used to validate decision support systems, demonstrating their viability in helping both experts and nonexperts make quicker and better-informed decisions after a disaster (Suleman et al., 2022). Another example is the study by Ramchurn et al. (2016), who validated an ABM to enhance comprehension and assess the system's efficacy in simulating scenarios using a tabletop disaster response role-playing game. In a study conducted by Utomo et al. (2021), the authors examined the behavior of agents and validated models through role-playing in an agent-based milk supply chain model. The study demonstrated the effectiveness of role-playing in validating agent-based models at both collective levels. Furthermore, Mariano and De Maria Albuquerque Alves (2020) used a simulation game focused on water management to study the actions of agents. An agent-based model was then built to simulate the results.

Conclusion

The purpose of the design and implementation of the disaster response tabletop role-playing game presented in this study is to validate the predictive capability of a disaster response agent-based model and to test the impact of the game on improving participants' knowledge in the domain of the study. By comparing the results between the pre- and posttest assessments for each group, the level of knowledge gained was calculated. A visual and statistical comparison between the ABM and the TRPG results provided insight into the effectiveness of the validation process. Although the combination of RPGs and ABMs is not one of the most popular ABM validation techniques, this study produced promising results.The results also suggest that this approach can lead to an increase in knowledge, particularly for participants with backgrounds in medicine and disaster risk management. Visual analysis of the results revealed a similar trend in the investigated ambulance policies. However, a statistical analysis comparing the results of the different approaches showed clear differences, highlighting the challenges in validating ABM using RPGs. The game scenario therefore not only offers the opportunity to practice basic skills but also illustrates the importance of timely and efficient coordination of various health facilities and emergency services. By providing a detailed analysis of the results of these strategies in the ABMS model and game environment, public health practitioners and policymakers can use this combined approach to develop more evidence-based and informed policies and protocols tailored to specific disaster response conditions.

Additionally, the combination of the DRTRPG and NetLogo (ABMS) approaches shows great promise for evaluating contingency plans. The methodological approach of this study resulted in a realistic simulation and a platform that can be customized for staff training and decision-making. The application of these strategies demonstrated the need for improved disaster response and reduced mortality rates. These collections of generative science and gamification techniques can help develop a valuable framework for consideration in subsequent research.

Limitations and Future Study

Although using TRPGs to simulate healthcare disaster situations offers stakeholders the opportunity to refine their skills and improve their decision-making ability in a safe and managed scenario, there are limitations to using RPGs to validate agent-based models. One limitation is that the game often does not have a complete representation of the scenario in the computational model. Additionally, since the study focused on predicting mortality outcomes in simplified systems, other factors that could lead to other secondary outcomes from interactions within the model and game (e.g., coordination, clinical staff, etc.) were not integrated into either strategy. Therefore, future ABMs should aim to involve stakeholders in the initial model development process. In addition, a dynamic online version of the game is proposed to reduce bias among participants and provide larger datasets for more robust statistical analysis.

Statements and Declarations

Acknowledgments

The authors thank everyone who participated in the exercises, especially the master’s and doctoral degree students of the Disaster Management and Environmental Impact, Laboratoire CERIS of IMT Mines Alés, who made themselves available for both the training sessions and the actual exercises. We also thank the Petroleum Technology Development Fund (PTDF) for funding this doctoral study.

Disclosure statement

With regard to this work, the authors declare that there are no financial or personal relationships that could be construed as conflicts of interest.

Funding

The authors received no financial support for this study or for the authorship or publication of this work.

Author contributions

Nimisingha Jacob Amakama: Conceptualization, Writing, Methodology, Game Design, Implementation, Data Collection, Analysis, and Visualization. Gilles Dusserre: Supervision, Review, Conceptualization, and Methodology, Review. Axelle Cadiere: Supervision, Review, Conceptualization. R. Wade Schuette: Conceptualization, Model Coding, Simulation and Analysis, Game Design, Review. Gregory Zacharewicz: Conceptualization, Methodology, Review.

References

• Ademiluyi, F., Amadi, S., & Amakama, N. (2010). Adsorption and Treatment of Organic Contaminants using Activated Carbon from Waste Nigerian Bamboo. Journal of Applied Science & Environmental Management, 13(3). https://doi.org/10.4314/jasem.v13i3.55351
• Alejandria, L. N., Bajenting, J. M. S., Pacatan, M. a. L. D., & Diquito, T. J. A. (2023). The use of educational board game as a supplemental tool in learning periodic table of elements among senior high school students. American Journal of Education and Technology, 2(1), 60–67. https://doi.org/10.54536/ajet.v2i1.1292
• Alonso-Fernández, C., Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2021). Improving evidence-based assessment of players using serious games. Telematics and Informatics, 60, 101583. https://doi.org/10.1016/j.tele.2021.101583
• Alonso‐Fernández, C., Martínez‐Ortiz, I., Caballero, R., Freire, M., & Fernández‐Manjón, B. (2019). Predicting students’ knowledge after playing a serious game based on learning analytics data: A case study. Journal of Computer Assisted Learning, 36(3), 350–358. https://doi.org/10.1111/jcal.12405
• Amakama, N. J., Dusserre, G., Cadiere, A., & Schuette, R. W. (2023a, September 5). Assessing the impact of wait times on patient mortality outcomes in a hypothetical oil and gas industry disaster scenario: An Agent-Based Modeling Approach using NetLogo. https://imt-mines-ales.hal.science/hal-04199308
• Amakama, N. J., Dusserre, G., Cadiere, A., Schuette, R. W., Zacharewicz, G., & Ahmed, N. (2023b, September 11). An Evaluation of the Effect of Ambulance Strategy on Patient Mortality Outcome from Hospital Systems: a Simulation Study in the Context of Industrial Disaster Response. https://imt-mines-ales.hal.science/hal-04208574
• Anand, N., Meijer, D., Van Duin, J., Tavasszy, L., & Meijer, S. (2016). Validation of an agent-based model using a participatory simulation gaming approach: The case of city logistics. Transportation Research. Part C, Emerging Technologies, 71, 489–499. https://doi.org/10.1016/j.trc.2016.08.002
• Antelmi, A., Cordasco, G., D’Ambrosio, G., De Vinco, D., & Spagnuolo, C. (2022). Experimenting with Agent-Based Model Simulation Tools. Applied Sciences, 13(1), 13. https://doi.org/10.3390/app13010013
• Cheng, P. H., Yeh, T. K., Chao, Y. K., Lin, J., & Chang, C. Y. (2020). Design ideas for an Issue-Situation-Based board game involving multirole scenarios. Sustainability, 12(5), 2139. https://doi.org/10.3390/su12052139
• D’Aquino, P., & Bah, A. (2014). Multilevel participatory design of land use policies in African drylands: A method to embed adaptability skills of drylands societies in a policy framework. Journal of Environmental Management, 132, 207–219. https://doi.org/10.1016/j.jenvman.2013.11.011
• De Rouck, R., Benhassine, M., Debacker, M., Dugauquier, C., Dhondt, E., Van Utterbeeck, F., & Hubloue, I. (2023). Creating realistic nerve agent victim profiles for computer simulation of medical CBRN disaster response. Frontiers in Public Health, 11. https://doi.org/10.3389/fpubh.2023.1167706
• Deinkuro, N. S., Knapp, C. W., Raimi, M. O., & Nimlang, N. H. (2021). Environmental Fate of Toxic Volatile Organics from Oil Spills in the Niger Delta Region, Nigeria. International Journal of Environment, Engineering and Education, 3(3), 89–101. https://doi.org/10.55151/ijeedu.v3i3.64
• Dubois, E., Barreteau, O., & Souchere, V. (2013). An Agent-Based model to explore game setting effects on attitude change during a role-playing game session. JASSS, 16(1). https://doi.org/10.18564/jasss.2065
• Fleming, K., Abad, J., Booth, L., Schueller, L., Baills, A., Scolobig, A., Petrovic, B., Zuccaro, G., & Leone, M. (2020). The use of serious games in engaging stakeholders for disaster risk reduction, management and climate change adaptation information elicitation. International Journal of Disaster Risk Reduction, 49, 101669. https://doi.org/10.1016/j.ijdrr.2020.101669
• Gore, R. J., Lynch, C. J., & Kavak, H. (2016). Applying statistical debugging for enhanced trace validation of agent-based models. Simulation, 93(4), 273–284. https://doi.org/10.1177/0037549716659707
• Guiamalodin, N. M., Culele, R. G. A., Silguerra, J., & Diquito, T. J. A. (2024). Unfolding the Experiences and Preparedness Mechanism of Senior Citizens in the Wake of Disaster: Toward a Disaster Resilient Community. American Journal of Society and Law, 3(1), 1–13. https://doi.org/10.54536/ajsl.v3i1.2299
• Guyot, P., Drogoul, A., & Honiden, S. (2007). Multi-Agent participatory simulations between experimental economics and Role-Playing games. In Agent-based social systems (pp. 11–19). https://doi.org/10.1007/978-4-431-71307-4_2
• Guzdial, M., Acharya, D., Kreminski, M., Cook, M., Eladhari, M., Liapis, A., & Sullivan, A. (2020). Tabletop Roleplaying Games as Procedural Content Generators. International Conference on the Foundations of Digital Games. https://doi.org/10.1145/3402942.3409605
• Hosseini, M. M., Hosseini, T. M., Qayumi, K., & Baeradeh, N. (2022). Game-based vs. Case-based Training for Increasing Knowledge and Behavioral Fluency of Nurse Students Regarding Crisis and Disaster Management; a Quasi-Experimental Study. DOAJ (DOAJ: Directory of Open Access Journals), 10(1), e77. https://doi.org/10.22037/aaem.v10i1.1739
• Husna, C., Kamil, H., Yahya, M., Tahlil, T., & Darmawati, D. (2020). Does tabletop exercise enhance knowledge and attitude in preparing disaster drills? Nurse Media : Journal of Nursing/Nurse Media Journal of Nursing, 10(2), 182–190. https://doi.org/10.14710/nmjn.v10i2.29117
• Introduction to Agent-Based modeling. (2019). In SAGE Publications Ltd. eBooks (pp. 14–34). https://doi.org/10.4135/9781529793543.n2
• Kwadwo, W., & Morkla, T. D. (2024). Occupational health and safety challenges faced national disaster management personnel. American Journal of Arts and Human Science, 3(1), 19–23. https://doi.org/10.54536/ajahs.v3i1.2406
• Ligtenberg, A., Van Lammeren, R. J., Bregt, A. K., & Beulens, A. J. (2010). Validation of an agent-based model for spatial planning: A role-playing approach. Computers, Environment and Urban Systems, 34(5), 424–434. https://doi.org/10.1016/j.compenvurbsys.2010.04.005
• Linehan, C., Lawson, S., & Doughty, M. (2009). Tabletop prototyping of serious games for “Soft skills” training. 2009 Conference in Games and Virtual Worlds for Serious Applications, 182–185. https://doi.org/10.1109/vs-games.2009.9
• Liu, C., Zhu, E., Zhang, Q., & Wei, X. (2021). Exploring the effects of computational costs in extensive games by modeling and simulation. International Journal of Intelligent Systems, 36(8), 4065–4087. https://doi.org/10.1002/int.22450
• Liu, Y., Fang, A., Moriarty, G., Kraut, R., & Zhu, H. (2023). Agent-based simulation for online mental health matching. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2303.11272
• Macal, C. M. (2016). Everything you need to know about agent-based modeling and simulation. Journal of Simulation, 10(2), 144–156. https://doi.org/10.1057/jos.2016.7
• Mariano, D. J. K., & De Maria Albuquerque Alves, C. (2020). The application of role-playing games and agent-based modeling to the collaborative water management in peri-urban communities. RBRH, 25. https://doi.org/10.1590/2318-0331.252020190100
• Marks, R. E. (2007). Validating Simulation Models: A general framework and four applied examples. Computational Economics, 30(3), 265–290. https://doi.org/10.1007/s10614-007-9101-7
• McCourt, E., & Watson, K. (2023). Creating a disaster ready pharmacy workforce: Evaluation of a Disaster tabletop exercise. Prehospital and Disaster Medicine, 38(S1), s106–s107. https://doi.org/10.1017/s1049023x2300290x
• McDonald, G. W., & Osgood, N. D. (2023). Agent-Based Modeling and its Tradeoffs: An Introduction & Examples. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2304.08497
• Meijer, S., Hofstede, G. J., Beers, G., & Omta, S. W. F. (2006). Trust and Tracing game: learning about transactions and embeddedness in a trade network. Production Planning & Control, 17(6), 569–583. https://doi.org/10.1080/09537280600866629
• Ogie, R., O’Brien, S., & Federici, F. (2022). Toward using agent-based modeling for collaborative translation of crisis information: A systematic literature review to identify the underlying attributes, behaviours, interactions, and environment of agents. International Journal of Disaster Risk Reduction, 68, 102717. https://doi.org/10.1016/j.ijdrr.2021.102717
• Olalekan MR, Albert O, Iyingiala AA, Sanchez DN, Telu M (2022) An environmental/scientific report into the crude oil spillage incidence in Tein community, Biseni, Bayelsa state Nigeria. J Environ Chem Toxicol. 2022;6(4):01-06., doi:10.37532/pulject.2022.6(4);01-06.
• Olalekan, R. M., Omidiji, A. O., Nimisngha, D., Odipe, O. E., & Olalekan, A. S. (2018). Health Risk Assessment on Heavy Metals Ingestion through Groundwater Drinking Pathway for Residents in an Oil and Gas Producing Area of Rivers State, Nigeria. Open Journal of Yangtze Oil and Gas, 03(03), 191–206. https://doi.org/10.4236/ojogas.2018.33017
• Onggo, B. S., & Karatas, M. (2016). Test-driven simulation modeling: A case study using agent-based maritime search-operation simulation. European Journal of Operational Research, 254(2), 517–531. https://doi.org/10.1016/j.ejor.2016.03.050
• Pasinetti, N. J., Alvarado, Y., Fernández, J., Guerrero, R. A., & Rodríguez, G. (2015). A serious game about recycling rules. XXI Congreso Argentino De Ciencias De La Computación (Junín, 2015). http://sedici.unlp.edu.ar/bitstream/handle/10915/50227/Documento_completo.pdf?sequence=1
• Pira, M. L., Marcucci, E., & Gatta, V. (2017). Role-playing games as a mean to validate agent-based models: an application to stakeholder-driven urban freight transport policy-making. Transportation Research Procedia, 27, 404–411. https://doi.org/10.1016/j.trpro.2017.12.060
• Poděbradská, M., Noel, M., Bathke, D. J., Haigh, T. R., & Hayes, M. J. (2020). Ready for drought? A Community Resilience Role-Playing game. Water, 12(9), 2490. https://doi.org/10.3390/w12092490
• Raimi, M. O., Iyingiala, A., Sawyerr, O. H., Saliu, A. O., Ebuete, A. W., Emberru, R. E., Sanchez, N. D., & Osungbemiro, W. B. (2022). Leaving no one behind: Impact of soil pollution on biodiversity in the Global South: A Global Call for Action. Sustainable Development and Biodiversity, 205–237. https://doi.org/10.1007/978-981-19-3326-4_8
• Raimi, M. O., Adindu, I. B., Udensi, E. O., Funmilayo, A. A., Opufou, T., Deinkuro, N. S., Adekunle, A. P., & Adeniji, A. O. (2020). Health Impact Assessment: Expanding public policy tools for promoting Sustainable Development Goals (SDGs) in Nigeria. EC EMERGENCY MEDICINE AND CRITICAL CARE, 9, 95–107. https://www.researchgate.net/profile/Morufu-Raimi/publication/344124781_Health_Impact_Assessment_Expanding_Public_Policy_Tools_for_Promoting_Sustainable_Development_Goals_SDGs_in_Nigeria/links/5f536b7392851c250b929a98/Health-Impact-Assessment-Expanding-Public-Policy-Tools-for-Promoting-Sustainable-Development-Goals-SDGs-in-Nigeria.pdf
• Raimi, M. O., O, O. A., Ayibatonbira, A. A., Anu, B., Odipe, O. E., & Deinkuro, N. S. (2019). “Digging Deeper” Evidence on water crisis and its solution in Nigeria for Bayelsa State: A study of current scenario. Social Science Research Network. https://doi.org/10.2139/ssrn.3422483
• Ramchurn, S. D., Huynh, T. D., Wu, F., Ikuno, Y., Flann, J., Moreau, L., Fischer, J. E., Jiang, W., Rodden, T., Simpson, E., Reece, S., Roberts, S., & Jennings, N. R. (2016). A Disaster Response System based on Human-Agent Collectives. Journal of Artificial Intelligence Research/˜the œJournal of Artificial Intelligence Research, 57, 661–708. https://doi.org/10.1613/jair.5098
• Robinson, D. T., Brown, D. G., Parker, D. C., Schreinemachers, P., Janssen, M. A., Huigen, M., Wittmer, H., Gotts, N., Promburom, P., Irwin, E., Berger, T., Gatzweiler, F., & Barnaud, C. (2007). Comparison of empirical methods for building agent-based models in land use science. Journal of Land Use Science, 2(1), 31–55. https://doi.org/10.1080/17474230701201349
• Rye, S., & Aktas, E. (2022). Serious Games as a validation tool for PREDIS: a decision support system for disaster management. International Journal of Environmental Research and Public Health/International Journal of Environmental Research and Public Health, 19(24), 16584. https://doi.org/10.3390/ijerph192416584
• Sakai, K., Shimizu, H., Toyoda, Y., & Kanegae, H. (2019). A study on gaming of participatory Evacuation planning in Tourist areas using agent simulation. In Translational systems sciences (pp. 247–256). https://doi.org/10.1007/978-981-13-8039-6_23
• Sanchez, D. N., Knapp, C. W., Olalekan, R. M., & Nanalok, N. H. (2021). Oil spills in the Niger Delta Region, Nigeria: Environmental fate of toxic volatile organics. Research Square (Research Square). https://doi.org/10.21203/rs.3.rs-654453/v1
• Simões-Marques, M., Moreno, D., & Correia, A. (2020). Developing a serious game to support disaster management preparedness - a design thinking approach. In Advances in intelligent systems and computing (pp. 108–115). https://doi.org/10.1007/978-3-030-51369-6_15
• Squazzoni, F., Polhill, J. G., Edmonds, B., Ahrweiler, P., Antosz, P., Scholz, G., Chappin, É. J. L., Borit, M., Verhagen, H., Giardini, F., & Gilbert, N. (2020). Computational models that matter during a global pandemic outbreak: a call to action. JASSS, 23(2). https://doi.org/10.18564/jasss.4298
• Suguitan, L., & Natividad, E. (2022). Localized Game-Based Learning Activity Sheets in Mathematics 7. American Journal of Multidisciplinary Research and Innovation, 1(4), 210–227. https://doi.org/10.54536/ajmri.v1i4.723
• Suleman, I., Pomalango, Z. B., & Slamet, H. (2022). DISASTER EXERCISE TABLETOP MEDIA IMPROVES KNOWLEDGE OF HEALTH PERSONNEL ABOUT DISASTER MANAGEMENT. Jambura Journal of Health Sciences and Research, 5(1), 90–99. https://doi.org/10.35971/jjhsr.v5i1.16633
• Tanguid, S. B. J., & Tanguid, R. V. (2023). Disaster resilience and capability in the maintenance of public order during disaster of Calamity-Prone municipalities in Oriental Mindoro, Philippines. American Journal of Multidisciplinary Research and Innovation, 2(5), 83–88. https://doi.org/10.54536/ajmri.v2i5.2058
• Takadama, K., Kawai, T., & Koyama, Y. (2008). Microand macrolevel validation in agent-based simulation: Reproduction of human-like behaviors and thinking in a sequential bargaining game. Journal of Artificial Societies and Social Simulation, 11(2), 9.
• Terti, G., Ruin, I., Kalas, M., Láng, I., Alonso, A. C. I., Sabbatini, T., & Lorini, V. (2019). ANYCaRE: a role-playing game to investigate crisis decision-making and communication challenges in weather-related hazards. Natural Hazards and Earth System Sciences, 19(3), 507–533. https://doi.org/10.5194/nhess-19-507-2019
• The functions of role-playing games: how participants create community, solve problems and explore identity. (2011). Choice/Choice Reviews, 48(06), 48–3103. https://doi.org/10.5860/choice.48-3103
• Utomo, D. S., Onggo, B. S. S., Eldridge, S., Daud, A. R., & Tejaningsih, S. (2021). Eliciting agents’ behavior and model validation using role playing game in agent-based dairy supply chain model. Journal of the Operational Research Society, 73(12), 2670–2693. https://doi.org/10.1080/01605682.2021.2013137
• Villamor, G. B., Van Noordwijk, M., & Troitzsch, K. G. (2023). Triangulating agent-based models, role-playing games, and a stakeholder-centric approach to change scenarios. Current Opinion in Environmental Sustainability, 64, 101323. https://doi.org/10.1016/j.cosust.2023.101323