Jane Butler, Ross Coffey, Kathryn Pegues, and John Salvador

Earlier this year, The Strategy Bridge asked university and professional military education students to participate in our fifth annual writing contest by sending us their thoughts on strategy.

Now, we are pleased to present one of the essays selected for Honorable Mention, from Jane Butler, Ross Coffey, Kathryn Pegues, and John Salvador, recent graduates from the Joint and Combined Warfighting School in Norfolk, Virginia.

"Data analytics is so important for the [Department of Defense] because the margin for error is shrinking, the decision cycle is shortening, and the attack surface is growing."

—Juliana Vida[1]

Increased computational power piqued the appetite for data analytics to support decision-making within government, industry, and academia. The Department of Defense Data Strategy, the guiding document for transforming the Defense Department into a data-centric organization, highlights the importance of data to the warfighter by underscoring that future battlefield survival will hinge on data utilization.[2] In the opening of the document, Deputy Secretary of Defense David Norquist called on all defense leaders to "treat data as a weapon system and manage, secure, and use data for operational effect."[3] Joint forces must collect and fuse data from diverse sources, employ data analytics that creates superior information to improve situational awareness, and leverage available information for disaggregated, precision effects.[4] Commanders leverage data to support decision-making, the activity defined as "the cognitive process leading to the selection of a course of action among alternatives."[5] This process incorporates intelligence analysis, information gathering, strategic guidance synthesis, and cross-functional integration to define the decision space. Peter Drucker recognized in his article, The Effective Decision, that each decision is a risk-taking judgment.[6] Commanders use information underscored by data to decrease the cumulative risk accepted when making a decision. Data analytics transforms data into information and thereby reduces inherent risk throughout the decision space. The incorporation of data analytics stands to improve decision-making by reducing uncertainty. 

Incorporating data analytics within decision-making is particularly important to the 11 Combatant Commands that provide command and control of military forces globally. Combatant Commands shoulder significant responsibilities with the primary being detection and deterrence of adversaries. The Combatant Commands plan, conduct, and assess all missions, operations, and activities in either their respective areas of responsibility or in support of their transregional functions.[7] To complete their missions, combatant commanders will, out of necessity, leverage data as a weapon system as it constitutes the basis of information development within the commander's decision space. With new sensors, the amount of collected data continues to climb, making more data available for transformation into actionable information supporting decision-making. Given the enormous volume of data that presently exists and the supply of trained analysts within a command, the commander and staff are assumed to have the capability to effectively employ data analytics to support the planning and execution of operations within the area of responsibility decisively. The perception is partially true. While senior leaders recognize the value of data analytics and attempt to employ the tools effectively, limited resources continue to challenge the Defense Department's ability to use data analytics to support operations.

The resource problem is twofold. First, organizations collect more data than they have analytic capacity.[8] In 2011, the Defense Science Board concluded collected data was not operationally of value since organizations lacked the "ability to store, process, exploit and disseminate it."[9] Secondly, while a vast amount of data is collected, the data gathered does not provide information on all critical interest areas. According to a Congressional Research Report published in 2011, the existing intelligence, surveillance, and reconnaissance capabilities are not capable of "support[ing] operations in the grey zone or support[ing] combat operations in a highly contested environment."[10] In this case, the term "capable" means having sensors in all five domains, air, land, sea, cyberspace, and space, to collect information, thereby allowing the commander to out-think the enemy. Compounding these challenges is the lack of an enterprise-level data management system to ensure that necessary data is accessible by decision-makers in a "real-time, usable, secure, and linked manner."[11] Therefore, while some resources exist to support data analytics at the Combatant Command level, the Defense Department has not sufficiently committed to the use of data analytics to enhance decision-making and reduce overall risk. A different approach to the decision-making process must provide the combatant commander efficacy using limited resources leveraging data analytics. 

Limited access to relevant, timely data hinders swift and appropriate action by the command to mitigate existing threats. Commanders collect and analyze data to gain better situational awareness and to reduce risk in decisions driving action. Before a commander decides to act, the staff must identify the commander's information requirements and then explore ways to collect relevant data from the operational environment to fulfill the requirements. Examining a commander's critical areas of interest and its decision-making process provides insight into how data analytics adds value to the quality of a commander's decisions. This insight enables prioritizing the limited data analytic resources within each command, focusing on critical areas of interest.

Understanding the Commander's Decision Space

The Combatant Commander is charged with understanding a global strategic landscape to define an operational environment that provides insight into assigned missions and tasks. The commander is responsible for aligning tasks with strategy and linking tasks with desired national and military strategic end states. The successful accomplishment of these tasks hinges on the commander being able to determine existing challenges and then assess the best approach to tackle complex issues. An accepted mental model for decision-making within military operations is the Observe-Orient-Decide-Act (OODA) Loop shown below. The OODA model was developed in the 1950s by military strategist Colonel John Boyd to explain how fighter pilots make quick, accurate decisions.[12] The process begins when the decision-maker observes a change in the environment. With new information, the decision-maker orients by developing a perception of the world based on current knowledge and the ability to analyze and synthesize data within the context of previous experience. The decision-maker then develops courses of action to move from the current state to a better future state. At this point, the individual decides which course of action to pursue. Once the choice is made, the decision-maker acts and then observes the environmental reaction. The OODA Loop includes the essential tasks of collecting information, putting the information in context, determining available courses of action, and decision-making with the intent of reaching a better future. The four-step, iterative process is effectively used in a dynamic environment.[13]

Decades after it was initially presented, the OODA loop is still used to make decisions, and the framework serves as a basis for follow-on decision-making methodologies.[14] The OODA loop is used as a staff works through a planning cycle and four primary decision areas for the commander: evaluating the operational environment, aligning military strategies with political goals, determining end states, and assessing risk. Today the operational environment is so complex that it is physically impossible for the Combatant Commander to observe everything directly or analyze and synthesize the data collected. A staff and commander will never have perfect information. The key is knowing when enough information is presented and accepting risk to act. Carl von Clausewitz describes this as the commander's coup d'oeil—intuition paired with information to make quick and knowledgeable decisions.[15] Part of the commander's evaluation of the operational environment is mitigating the scale and scope by positioning sensors throughout an area of interest to collect data. The data is then transformed into information with the help of data analytics.

Data Analytics: A Valuable Tool within the Decision Space

At its most basic level, data can be understood as a "representation of facts, concepts, or instructions in a formalized manner suitable for communication, interpretation, or processing by humans or by automated means."[16] For example, a piece of data historically tracked by the Defense Department is the number of improvised explosive devices that occur along a defined stretch of road in a set period. Many times, individual pieces of data may not provide enough insight for commanders to make decisions. The data must be synthesized or fused with other elements of data to create actionable information. In the example, the number of explosive devices could indicate insurgent activities at the location but by itself would not be sufficient to decide the number of explosive ordnance disposal teams to deploy in the region or explicitly where those teams should be positioned. Synthesis of data from multiple sources could build a common understanding of the prevalence of explosive device emplacement within the region over time. The common understanding could be sufficient to decide the number of explosive ordnance disposal teams to deploy. Data must be processed into actionable information by including context and relevance to provide usefulness. Actionable information encompasses information, knowledge, and understanding, constituted by context, instructions, and explanation that enables the decision-maker to increase the probability of making the best decision. One way to turn data into actionable information is through data analytics, a set of processes to gather information from data. Data analytics can be broken down into four distinct methods, shown below.

Descriptive data analytics, the most applied type, uses statistical methods to summarize historical data. In the case of the Combatant Command, descriptive analytics answers what is happening in the area of responsibility. The number of effective improvised explosive device attacks in a defined area can be determined using data analytics.[27] Diagnostic data analytics is one method to understand contributing factors; why an event did or did not occur. Researchers analyzed insurgent activity trends to better understand the impact of friendly force explosive device countermeasures on combat operations.[28] The team found that the employment of explosive device countermeasures correlated with a decrease in the number of insurgent non-explosive device attacks on friendly forces.[29] The decrease was attributed to the insurgents' re-allocation of resources to sustain device emplacement levels with the additional burden to overcome countermeasures.[30] Predictive analytics is forecasting using trends identified in historical datasets. This type of data analytics develops models to help answer what is likely to happen in a scenario. Predictive data analytics was used to develop models for explosive blasts, which improved equipment survivability research and development efforts.[31] While less prevalent, prescriptive analytics plays an essential role in providing commanders with feedback on potential outcomes of decisions. Using findings from descriptive, diagnostic, and predictive analytics, prescriptive analytics develops models to quantify the outcomes of proposed courses of action. The field of data analytics exists to help make sense of data and transform it into actionable information. The foundation for all types of data analytics is a clear articulation of what the decision-maker needs to know and the quality of the data available for analysis. The figure below illustrates data transformation into actionable information, providing insight in the commander's decision space and how data analytics enables quality decision making.

To provide value to the decision-making process at a Combatant Command, a network of carefully placed sensors collect data to provide better situational awareness about the physical domain. The physical domain provides data to help describe the current state of the operational environment, the status of political agendas, progress toward end states, and the level of risk. Once collected, the data enters the information domain, the collected data is stored, fused, and processed into information using data analytics. The staff then uses the information to improve situational awareness through shared understanding. At this point, the information is part of the cognitive domain. As information triggers a decision point, the commander uses existing strategic guidance and personal experience to decide what course of action to take. Once the decision is made, the command starts the process over to sense the impact that the decision made on the physical environment. The collection of data allows the commander to observe the environment at specific locations with the flexibility to move sensors as required.

By collecting germane pieces of data, the commander removes blind spots from the decision process and creates maneuver space in a decision window to think and decide on the best course of action. Data collection and synthesis provide the commander with better situational awareness and the staff a shared understanding of the four primary areas of the commander's decision space mentioned earlier in the context of the OODA loop: (1) the operational environment, (2) political goals, (3) transitions, terminations, and end states, and (4) risk. The figure below illustrates how data analytics supports the orient and the act steps of the OODA Loop. Within the orient phase, data analytics allows the decision-maker to develop a perception of the world faster. An article published last year in the Strategy Bridge critiques Secretary Mattis and General Dunford's use of the term "speed of relevance." Posed in the report is the crux of decision-making and an important factor—time; "technological changes in warfare, compounded by heightened competition, are increasing the time pressure on decision-makers across the gamut of defense."[32] Data is turned into information using advanced computational processing where patterns are quickly identified. Aware of the time pressure, the decision-maker can assess the impact of action using appropriate measures of effectiveness. Data analytics serves to enhance a commander's decision-making by decreasing the time to analyze existing data, quantifying the differences between existing courses of action, and measuring the decision's impact.

Four Primary Areas of the Combatant Commander's Decision Space

The Combatant Commander's decision space is the range of practical choices to conduct the mission constrained by feasibility and acceptability, with decisions derived across four primary areas. The table below provides a brief description of each area, describes why a commander needs the information to support decision-making and explains how decisions in the identified area are improved with data analytics. The following paragraphs provide additional details on the four areas.

Operational Environment

The operational environment is a complex, constantly evolving system of systems described by Joint Publication 5-0, Joint Planning, as the "composite of the conditions, circumstances, and influences that affect the employment of capabilities and bear on the decisions of the commander [emphasis added]."[34] The operational environment is the compilation of the political, military, economic, social, information, and infrastructure systems. Since the operational environment is dynamic, an iterative assessment is necessary to keep the Combatant Command staff aware of changes that impact the mission. Timely, relevant information about changes in the operational environment triggers the staff to review current decision points and develop courses of action that will result in the desired end state. Given the complexity, size, and scope of a Combatant Command's operational environment, data analytics plays an essential role in transitioning collected data into actionable information. Emerging analytic technologies can fuse and correlate data automatically that would otherwise take human-in-the-loop hours or days to complete.  

One example of where data analytics provides valuable information to the commander is access to, use, and control of water and electric infrastructure. Without stable access to both resources, civil unrest grows. The current state of infrastructure in Iraq and Syria continues to contribute to the instability of the states as it impacts human health and economic potential.[35] A contributing factor is knowledge of natural resources and the importance of assessing critical infrastructure recovery and the living conditions of local populations.[36] Limitations in the breadth of collected data and the constrained capability to analyze the available data impact the commander's ability to understand the operational environment. With limited resources, the staff possesses incomplete situational awareness of the political, military, economic, and social factors that affect the area of responsibility.

Political Goals

In developing their best operational approach, commanders keep the achievement of desired political outcomes in mind. In Assessing War: The Challenge of Measuring Success and Failure, Cordesman and Rothstein suggest that integral to any strategy is how planners think military operations will lead to desired political outcomes.[37] Planners must determine how to measure the success of military activities in achieving political objectives. Carefully selected indicators for the achievement of political objectives need to be identified as soon as possible. Continued analysis of those indicators provides the commander valuable feedback on whether applied instruments of national power positively affect the achievement of the desired outcome. One example of how data analytics could provide insight into an identified political outcome in a fragile state is corruption within the government. Corruption is defined by the United Nations (U.N.) as "the improper use of public or official position for private gain."[38] The U.N. Office of Drugs and Crime used survey data to assess the Afghanistan population's perception of corruption between 2009 and 2012. During that period, the perceptions have not improved significantly. In fact, at the end of 2012, over half of the survey respondents said they paid a bribe for access to a public service.[39] This result would indicate that if decreasing corruption in Afghanistan was a political objective for the U.S. during that period, the resources applied to the issue had no effect. Thus, data analytics offers the opportunity to solidify the collection of indicators and use emergent computational power to detect patterns and suggest alternative paths towards progress or regression avoidance before deciding upon the next steps.

Transitions, Terminations, & End State

Another component of combatant command strategic-level decision-making is recognizing the moment that a transition, termination, or end state is reached. As the decision to transition is political, an accurate assessment of the need to transition will most likely necessitate a holistic review with input from all U.S. government entities involved in the operation. The Combatant Commander needs a repeatable method to determine when the original campaign is no longer valid or if it is time to transition to a new objective. Transition or termination of a plan is determined by deliberate planning efforts to identify specific, measurable conditions. A possible scenario involving a Combatant Command is the transition of security functions to a host nation in a coalition environment. Appendix E of the Marine Corps' Transition Handbook provides an exhaustive list of transition assessments and metrics for this scenario. To assess health development, the staff could evaluate the available types of health services (primary versus specialty care), the use of health services by the general population, and the quality of available services.[40] Data analytics support developing a holistic assessment of health in a region using survey data on perceptions of service, import reports, medical infrastructure, and disease surveillance data. Incorporating data analytics will arrive at a solution faster than the current analysis construct.[41] Recommendations routed to the commander expeditiously can speed up the decision-making process, potentially improve the predicted outcome, and substantiates operating at the speed of relevance.

Risk

Risk is "the probability and consequence of an event causing harm to something valued."[42] The Combatant Commander's decision-making process encompasses risk because perfect knowledge of the operational environment is impossible and creates uncertainty. Dealing with uncertainty in planning necessitates the identification of assumptions to fill information gaps. For all assumptions used in planning, the staff is responsible for articulating the impact of any invalid assumption and providing the commander a holistic assessment of the expected risk and potential consequences. 

The figure below illustrates that while information available to a commander increases over time, the inverse is true for a commander's flexibility.[43] Risk, shown as the red line, is most significant at the start. As time progresses and more information becomes available, the risk lowers. However, as time progresses, a commander's flexibility decreases, and risk increases. The incorporation of data analytics can accelerate the availability of information in leading to the decision point and widening the optimal decision window. A larger optimal decision window provides the commander greater flexibility resulting in a reduction of overall risk. The opposite occurs when there is insufficient data for identified critical information requirements and a shortage of resources to analyze data. A Combatant Command with scarce data analysis resources slows information flow and increases risk. With less information, the optimal decision window decreases. Ideally, data analysis is used to move the decision point to the left, which means broadening the decision window and increasing flexibility by creating more opportunities for cycling through OODA. Insufficient data or resources to process data inhibits achieving the ideal and creates risk through the diminished flexibility that comes from decisions made closer to inextricable deadlines. 

A Combatant Command with adequate data analysis resources can apply data analytics to transform decision-making, increase flexibility, and reduce risk. Intelligent data analytics can change the commander's decision space by enabling a temporal balance between information and flexibility. The commander's identification of trouble earlier in the decision window provides an opportunity to flatten the risk curve ahead of the optimal decision window and allow the commander to employ more adaptable operations. In 2015, the Air Force Research Laboratory demonstrated success with data analytics in the cyber domain for insider threat detection.[45] The laboratory developed a Behavior-Based Access Control model of data analytics to analyze network traffic automatically using machine learning.[46] The research laboratory concluded that using their model of data analytics is critical "to defend Department of Defense networks, namely the ability to assess trustworthiness of actors, services, and documents in real-time, at enterprise scale, and with actionable accuracy."[47] Harnessing emerging data analytics technologies can transform data into actionable information near real-time and enables a decision-centric approach to manage risk and increase flexibility.

Decision-making with Data Analytics in a Resource-Constrained Environment

Incorporating data analytics into the decision-making process improves a commander's situational awareness of the area of responsibility. Commanders will have access to leading indicators of change on the battlefield, allowing for increased time to decide and act. As stated earlier, the Defense Department currently struggles to implement data analytics that improves the commander's decision-making quality due to two resource limitations. First, the Defense Department collects more data than it has the analytic capability to process. A shortage of data analytical capability results in a commander having less information with which to make decisions. Secondly, while the Defense Department generally collects an enormous amount of data, blind spots for the Combatant Command still exist for critical interest areas that do not have the appropriate type and quantity of sensors to gather pertinent data. Keeping the goal of an enhanced understanding of the decision space in mind, commanders need to think critically about where to position limited sensors to collect data on critical areas. Decision-making in a resource-constrained environment necessitates incorporating two additional steps into the OODA loop. The proposed enhanced model, shown in the figure below, forces decision-makers to acknowledge that they cannot observe everything and must prioritize what data to collect. At the start of each cycle, the staff must Scan the entire environment to determine the critical information required to make decisions. This step focuses the commander's observation efforts on areas of high importance. After completing the Scan step, the decision-maker will Observe the environment by aggregating data from multiple sources. Following the Observe step, the second new step, Select, is inserted. During the Select step, the Combatant Commander will prioritize analytic efforts given known priorities. Just as the strategic environment is complex and large today, the observable data can also be complex and vast. In a resource-constrained environment, the command will not be able to analyze and synthesize all data collected. Using appropriate prioritization, the staff will focus resources on critical areas first. 

The following steps of Orient, Decide, and Act remain unchanged. Implementation of the Scan-Observe-Select-Orient-Decide-Act (SOSODA) Loop is imperative at Combatant Commands given the uneven collection of data across the areas of interest and the overwhelming amount of available data for the limited analytical resources afforded to the command.

Concluding Perspectives: Combatant Command Decision-making with Data Analytics

To accomplish their missions, Combatant Commanders need support in understanding the operational environment, assessing progress towards political goals, identifying arrival at transition, termination, or end states, and evaluating risk. Decision-making, defined as "the cognitive process leading to the selection of a course of action among alternatives," is challenging given the complex, dynamic operating environment in which Combatant Commands operate.[48] The use of data analytics to enhance the decision-making process is a necessary, critical component of retaining a competitive advantage. Incorporating descriptive, diagnostic, predictive, and prescriptive analytics helps the staff identify trends in operations, quantify risk, and assess the outcomes of proposed courses of action within the commander's decision space. Descriptive analytics uses statistical methods to summarize historical data. While dependent on access to quality data, the results help in the visualization of the operational environment. Diagnostic analytics illuminate contributing factors and improve understanding as to why events occur. Predictive analytics forecasts the likelihood of future events. Lastly, prescriptive analytics assist the decision-maker in determining what action to take.

Data analytics enhances the orient and act steps of the OODA Loop, a widely used decision-making process within the Defense Department. In the Orient step, data analytics increases the speed at which the decision-maker can develop a perception of the world. Computational methods allow for the extraction, fusion, and synthesis of data from disparate sources to create new information leading to identifying trends. In addition, prescriptive analytics assists the decision-maker in evaluating possible courses of action. Data analytics is used to assess if the decision made had the intended effect during the act step.

As stated earlier, the Defense Department is challenged to implement data analytics to improve the Combatant Commander's decision-making quality due to resource limitations. First, the Defense Department collects more data than it has the analytic capability to process. A shortage of data analytical capability results in a Combatant Commander having less information to make decisions. Secondly, blind spots exist for the command where critical interest areas do not have appropriate sensors to collect pertinent data. Decision-making in a resource-constrained environment necessitates incorporating two additional steps into the OODA loop. The new updated decision-making process, Scan-Observe-Select-Orient-Decide-Act (SOSODA) Loop, is designed to be used when the capability for data analytics exists but is resource-constrained. The addition of the Scan step at the start of the decision-making process allows the commander to select areas of interest within the operating environment from which to collect data. The Scan step enables the command to use limited sensors to collect data on critical areas of the decision space. During the Observe step, data is aggregated from multiple sources. Entering the Select step, the commander will prioritize what areas of the decision space information are most needed. Data analytic resources will then be allocated based on the prioritized list. The Orient, Decide, and Act steps remain unchanged from the OODA Loop. Implementation of the SOSODA Loop is imperative at Combatant Commands given the unfocused collection of data across the areas of interest and the overwhelming amount of available data for the limited analytical resources afforded to the command. The future of Combatant Commander decision-making rests in the Defense Department's commitment to prioritizing data analytics for a decision-centric approach to warfare.

John Salvador, Kathryn Pegues, Jane Butler, and Ross Coffey currently serve as part of Combatant Command staff and Military Academic Institutions. This essay reflects their own views and not necessarily those of the U.S. government or the Department of Defense.

John Salvador, is an officer in the United States Air Force. He was commissioned through ROTC at the University of Michigan in 2007. John earned a BSE in Electrical Engineering from the University of Michigan 2007 and an M.S. in Engineering and Technology Management from Oklahoma State University in 2015.

Kathryn Pegues is an officer in the United States Army. Kathryn was commissioned through the United States Military Academy in 2002. She earned a B.S. in Mathematics from USMA in 2002, an M.S. in Applied Mathematics and an M.S. in Operations Research from the Naval Postgraduate School in 2008, and a PhD from Clemson University in 2019.

Jane Butler is an officer in the United States Navy. She was commissioned through NROTC at University of California, Los Angeles in 2004. Jane earned a B.A. in Business Economics from UCLA in 2004 and a MBA from University of Phoenix in 2010.

Ross Coffey is an officer in the United States Army. Ross is a Military Professor of National Security Affairs at the U.S. Naval War College. He earned his commission through the United States Military Academy in 1994. Ross earned a B.S. in Mathematics from USMA in 1994, a M.S. in Administration from the Central Michigan University in 2004, and a M.A. from the School of Advanced Military Studies in 2006.

The views expressed are the authors’ alone and do not reflect those of the Department of Defense, or the U.S. Government.

Have an idea for your own article? Follow the logo below, and you too can contribute to The Bridge:

Enjoy what you just read? Please help spread the word to new readers by sharing it on social media.

Header Image: Sometimes you just have to look up, February, 2017 (Joshua Sortino).

Notes:

[1] David Vergun, “DOD Moves to Use Data More Effectively in Decision-Making,” U.S. Department of Defense, February 14, 2020, https://www.defense.gov/Explore/News/Article/Article/2085331/dod-moves-to-use-data-more-effectively-in-decision-making/.

[2] US Department of Defense, DoD Data Strategy (Washington, D.C.: US Department of Defense, 2020).

[3] Ibid.

[4] Ibid.

[5] Wilhelm Kirch, ed., “Decision Making Process,” in Encyclopedia of Public Health (Dordrecht: Springer Netherlands, 2008).

[6] Peter F. Drucker, “The Effective Decision,” Harvard Business Review, January 1967.

[7] White House, Unified Command Plan (Washington, DC, 2021). The AOR is a complex environment and commanders must consider boundaries associated with domains and functions. For the purposes of this research, the commander should have awareness of the domains and functions relevant to their authority and responsibilities.

[8] David Zelaya and Nicholas Keeley, “The Input-Output Problem: Managing the Military’s Big Data in the Age of AI,” War on the Rocks, 13 2020, https://warontherocks.com/2020/02/the-input-output-problem-managing-the-militarys-big-data-in-the-age-of-ai/.

[9] US Department of Defense, “Counterinsurgency (COIN) Intelligence, Surveillance, and Reconnaissance (ISR) Operations” (Washington, DC, 2011), 49.

[10] John R. Hoehn and Nishawn S. Smagh, Intelligence, Surveillance, and Reconnaissance Design for Great Power Competition, CRS Report No. R46389 (Congressional Research Service, 2020), https://crsreports.congress.gov/product/pdf/R/R46389.

[11] Yasmin Tadjdeh, “New DoD Strategy Charts Path to ‘Data-Centric’ Future,” National Defense, October 2020, https://www.nationaldefensemagazine.org/articles/2020/10/8/new-dod-data-strategy-charts-path-to-data-centric-future.

[12] Robert Coram, Boyd: The Fighter Pilot Who Changed The Art of War (Boston, NY: Back Bay Books, Little Brown and Company, 2004).

[13] Ibid.

[14] Berndt Brehmer, “The Dynamic OODA Loop: Amalgamating Boyd’s OODA Loop and the Cybernetic Approach to Command and Control” (10th International Command and Control Research and Technology Symposium: The Future of C2, National Defense University, Washington, D.C., 2005); Richard Breton and Robert Rousseau, “The C-OODA: A Cognitive Version of the OODA Loop to Represent C2 Activities” (10th International Command and Control Research and Technology Symposium: The Future of C2, National Defense University, Washington, D.C., June 2005); Richard Breton and Robert Rousseau, “The M-OODA: A Model Incorporating Control Functions And Teamwork In The OODA Loop” (Defence Research and Development Canada, Valcartier, 2005).

[15] Carl von Clausewitz, On War, ed. Michael Eliot Howard and Peter Paret (Princeton, N.J: Princeton University Press, 1989).

[16] US Department of the Navy, Department of the Navy: Strategy for Data and Analytics Optimization (Washington, DC, 2017), 10.

[17] Clay Wilson, Improvised Explosive Devices (IEDs) in Iraq and Afghanistan: Effects and Countermeasures, CRS Report No. RS22330 (Congressional Research Service, 2007), https://fas.org/sgp/crs/weapons/RS22330.pdf.

[18] Jairus Grove, “An Insurgency of Things: Foray into the World of Improvised Explosive Devices,” International Political Sociology 10, no. 4 (December 2016): 332–51, https://doi.org/10.1093/ips/olw018.

[19] Alec D. Barker, “Improvised Explosive Devices in Southern Afghanistan and Western Pakistan, 2002–2009,” Studies in Conflict & Terrorism 34, no. 8 (August 2011): 600–620, https://doi.org/10.1080/1057610X.2011.582630.

[20] Nathan J. McNeese et al., “Identification of the Emplacement of Improvised Explosive Devices by Experienced Mission Payload Operators,” Applied Ergonomics 60 (April 2017): 43–51, https://doi.org/10.1016/j.apergo.2016.10.012.

[21] Matthew A. Hanson, “The Economics of Roadside Bombs,” SSRN Electronic Journal, 2008, https://doi.org/10.2139/ssrn.1069541.

[22] Matthew A. Price et al., “An Approach to Modeling Blast and Fragment Risks from Improvised Explosive Devices,” Applied Mathematical Modelling 50 (October 2017): 715–31, https://doi.org/10.1016/j.apm.2017.06.015.

[23] Jelena Paripovic, “Characterization and Modeling of Materials Used in Improvised Explosive Devices” (master’s thesis, Purdue University, 2013), ProQuest (UMI 1549442).

[24] Price et al., “An Approach to Modeling Blast and Fragment Risks from Improvised Explosive Devices.”

[25] Kyle Lin and Jeffrey Dayton, “Game-Theoretic Models for Jamming Radio-Controlled Improvised Explosive Devices,” Military Operations Research 16, no. 3 (September 2011): 5–13, https://doi.org/10.5711/1082598316305.

[26] Jaff Guo, Joe Armstrong, and David Unrau, “Predicting Emplacements of Improvised Explosive Devices,” The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology 10, no. 1 (January 2013): 75–86, https://doi.org/10.1177/1548512912439951.

[27] Wilson, Improvised Explosive Devices (IEDs) in Iraq and Afghanistan: Effects and Countermeasures.

[28] Hanson, “The Economics of Roadside Bombs.”

[29] Ibid.

[30] Ibid.

[31] Price et al., “An Approach to Modeling Blast and Fragment Risks from Improvised Explosive Devices.”

[32] Joe Dransfield, “How Relevant Is the Speed of Relevance?: Unity of Effort Towards Decision Superiority Is Critical to Future U.S. Military Dominance,” The Strategy Bridge, accessed May 6, 2021, https://thestrategybridge.org/the-bridge/2020/1/13/how-relevant-is-the-speed-of-relevance-unity-of-effort-towards-decision-superiority-is-critical-to-future-us-military-dominance.

[33] Joint Chiefs of Staff, Joint Planning, JP 5-0 (Washington, D.C.: Joint Chiefs of Staff, 2020), IV–6.

[34] Ibid.

[35] Kevin Rosner, Water and Electric Power in Iraq and Syria: Conflict and Fragility Implications for the Future, State Fragility Initiative (Austin, TX: Robert Strauss Center for International Security and Law, 2016), https://www.strausscenter.org/wp-content/uploads/Water-and-Electric-Power-in-Iraq-and-Syria-2016.pdf.

[36] Ibid.

[37] Leo J. Blanken, Hy Rothstein, and Jason J. Lepore, Assessing War: The Challenge of Measuring Success and Failure (Washington, DC: Georgetown University Press, 2015), 319.

[38] United Nations Office on Drugs and Crime, Corruption in Afghanistan: Recent Patterns and Trends (Vienna, Austria: United Nations, 2012).

[39] Ibid.

[40] US Marine Corps, Transition Planning Handbook, MCRP 5-10.2 (Washington DC, 2016).

[41] US Department of Defense, DoD Data Strategy.

[42] Chairman of the Joint Chiefs of Staff, Joint Risk Analysis, CJCSM 3105.01 (Washington, DC: Department of Defense, 2016).

[43] Joint Staff J7, Deployable Training Division, Insights and Best Practices Focus Paper: Assessment and Risk (Suffolk, VA: The Joint Staff, 2020).

[44] Ibid.

[45] Michael Mayhew et al., “Use of Machine Learning in Big Data Analytics for Insider Threat Detection,” in MILCOM 2015 - 2015 IEEE Military Communications Conference (Tampa, FL, USA: IEEE, 2015), 915–22, https://doi.org/10.1109/MILCOM.2015.7357562.

[46] Ibid.

[47] Ibid.

[48] Kirch, “Decision Making Process.”