Threat Modelling Evaluation

After all threats have been fixed and the risks around them assessed, a determination of whether these efforts are working will be made. It is essential to test the applied fixes to gain confidence in the measures taken against previously identified threats. This will be the objective of the threat modeling validation process. Validation will require testing. Testing is a vital element of the quality assurance process. 1 It will require the ability to understand issues found when determining denial of service threats, manage denials, look for structured query language (SQL) injection, use fuzzing, and create or manage tests that are not based on threat modeling. 2 The threat modeling validation process will be carried out in three phases: validation of threat models, validation of the quality of threats and mitigations, and validation of the information captured.  

Objective of Validation 

The validation process aims to establish the effectiveness of mitigations. It is a crucial step since it ensures that threat models perform their intended purpose, and that that the project’s deliverables are attained. 

Validation of the Threat Model 

The first process in evaluating the cyber threat model’s effectiveness will involve validating the whole threat model. This process will involve a determination of whether the diagram matches the final code. By doing so, we will establish whether the threat model’s processes are well-placed to mitigate the threats. Secondly, an establishment of whether the threats are enumerated will be made. The minimum requirement for this process will be the determination of STRIDE for each element that touches a stride boundary. 3   

It will also be important to determine whether the testers or quality assurance (QA) have reviewed the threat model. At some point, it may be necessary to deliver or deploy the software; thus, quality assurance will be crucial. “Threat models can be subjected to a quality assurance process”. 4 It’s important to “close the loop” and ensure proper handling of threats. The quality assurance approach often reveals problems with threat models. 5 As the project gets close to completion, a confirmation of whether the threat model works will be done. A checklist will be used to ensure that all items in the threat list are addressed, the model is close to reality, and that all bugs are closed. The checklist will comprise a list of all threats and bugs. The quality assurance team will use the list to determine threats that have been addressed and those that still need attention. Checking the model’s conformance to reality will indicate whether the list of threats found is relevant to what we build. 6 The team will also check whether the threat model mitigates all threats and whether all mitigations are done properly. All these items will be checked before the final security review.  

Validating the Quality of Threats and Mitigations 

The second stage of the threat model validation process is the validation of the quality of threats and mitigations. As illustrated in Figure 1, threats and mitigations. Hence a validation of both should be conducted. This stage will involve determination of whether the identified threats describe the attack, the context of the attack, and its impact. In this way, the validation process will ensure that the threat model addresses all possible vulnerabilities.  

Figure 1. Relationship between threats, mitigations and requirements. 

Validation of mitigations will also be conducted. Particularly, an investigation of whether the mitigations are associated with specific threats, whether they are described, and whether they file bugs will be conducted. For instance, fuzzing is not mitigation. 7 Instead, it is a test tactic. Mitigations will also be tested by further causing bugs to trigger and creating tests to see whether the mitigations are working. These tests will either be automatic or manual. Since coding automated tests requires high expertise, we will mainly opt for manual tests. However, since the preferred bugs do not regress, automation is the best option. We will include variations of the attack to make the testing more effective. In developing tests for threats, we will consider how mitigations can be attacked. Testing will call for expertise in shell coding or cryptography. Shell coding is the creation of user interfaces for accessing an operating system’s services. 8 Therefore, testers might require skills in creating command-line or graphical user interfaces. Cryptographers’ skills and expertise will be needed to protect information adequately during testing.  

As the project get closer to completion, a review of mitigation test bugs will be done. This review will ensure that all test bugs have been closed. This is the stage where tags such as the tmtest will be helpful. 9 Where some bugs will not have been closed, they will be triaged like other bugs. The bugs will be fixed, or if necessary, moved to a later revision stage.  

Validation of Information Captured 

Software dependencies are “relationships among code that exist as a result of external influences on the software development process”. 10 Since they affect the development of systems, we will also validate them. Basically, we will determine other code being used, the security functions embedded in the code, and assumptions made during the development of the threat model. For instance, it may be assumed that GenRandom will give crypto-strong randomness. This is a valid assumption since GenRandom generates random data with quality level. 11 On the other hand, it may be assumed that protection from malformed messages will be obtained from linear predictive coding, which is not true. These examples show the importance of validating our assumptions to ensure the threat model’s effectiveness.  

Outputs of the Validation Process 

The main output of the validation process is the effectiveness of the threat model. After evaluation, an establishment of what threats the threat model works against, the model’s areas of weakness that need improvement, and the impact of threats on the model will be done. 

Figure 2 shows the model validation process. 

Figure 2. Diagram of the threat modelling validation process. 

References 

Awojana, T. B. (2018).  Threat Modelling and Analysis of Web Application Attacks  [Master's thesis].  http://hdl.handle.net/10342/7049 

Aydin, M. (2016).  Engineering Threat Modelling Tools for Cloud Computing  (Doctoral dissertation, University of York). 

Eng, D. (2017).  Integrated Threat Modelling  [Master's thesis].  https://www.duo.uio.no/handle/10852/55699 

Fagade, T. (2018).  A multi-domain approach for security compliance, insider threat modelling and risk management  (Doctoral dissertation, University of Bristol). 

Fagade, T., Spyridopoulos, T., Albishry, N., & Tryfonas, T. (2017, July). System dynamics approach to malicious insider cyber-threat modelling and analysis. In  International Conference on Human Aspects of Information Security, Privacy, and Trust  (pp. 309-321). Springer, Cham. 

Iureva, R. A., Kremlev, A. S., Drannik, A. L., & Cech, M. (2019, October). Threat Modelling for Critical Infrastructure Object. In  2019 11th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT)  (pp. 1-4). IEEE. 

Majhi, S. K., & Dhal, S. K. (2016). Threat modelling of virtual machine migration auction.  Procedia Computer Science ,  78 (C), 107-113. 

Nweke, L. O., & Wolthusen, S. (2020). A Review of Asset-Centric Threat Modelling Approaches. 

Saigushev, N. Y., Vedeneeva, O. A., Melekhova, Y. B., Ryazanova, L. S., & Konovalov, M. V. (2019, October). Threat modelling for creating technical information protection system at automated machine building plants. In  Journal of Physics: Conference Series  (Vol. 1333, No. 3, p. 032073). IOP Publishing. 

Schaad, A., & Reski, T. (2019). " Open Weakness and Vulnerability Modeler"(OVVL)–An Updated Approach to Threat Modeling. 

1 Nweke, L. O., & Wolthusen, S. (2020). A Review of Asset-Centric Threat Modelling Approaches. 

2 Fagade, T., Spyridopoulos, T., Albishry, N., & Tryfonas, T. (2017, July). System dynamics approach to malicious insider cyber-threat modelling and analysis. In  International Conference on Human Aspects of Information Security, Privacy, and Trust  (pp. 309-321). Springer, Cham. 

3 Eng, D. (2017).  Integrated Threat Modelling  [Master's thesis].  https://www.duo.uio.no/handle/10852/55699 

4 Iureva, R. A., Kremlev, A. S., Drannik, A. L., & Cech, M. (2019, October). Threat Modelling for Critical Infrastructure Object. In  2019 11th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT)  (pp. 1-4). IEEE. 

5 Schaad, A., & Reski, T. (2019). " Open Weakness and Vulnerability Modeler"(OVVL)–An Updated Approach to Threat Modeling. 

6 Saigushev, N. Y., Vedeneeva, O. A., Melekhova, Y. B., Ryazanova, L. S., & Konovalov, M. V. (2019, October). Threat modelling for creating technical information protection system at automated machine building plants. In  Journal of Physics: Conference Series  (Vol. 1333, No. 3, p. 032073). IOP Publishing. 

7 Fagade, T. (2018).  A multi-domain approach for security compliance, insider threat modelling and risk management  (Doctoral dissertation, University of Bristol). 

8 Aydin, M. (2016).  Engineering Threat Modelling Tools for Cloud Computing  (Doctoral dissertation, University of York). 

9 Awojana, T. B. (2018).  Threat Modelling and Analysis of Web Application Attacks  [Master's thesis].  http://hdl.handle.net/10342/7049 

10 Saigushev, N. Y., Vedeneeva, O. A., Melekhova, Y. B., Ryazanova, L. S., & Konovalov, M. V. (2019, October). Threat modelling for creating technical information protection system at automated machine building plants. In  Journal of Physics: Conference Series  (Vol. 1333, No. 3, p. 032073). IOP Publishing. 

11 Majhi, S. K., & Dhal, S. K. (2016). Threat modelling of virtual machine migration auction.  Procedia Computer Science ,  78 (C), 107-113.