DOI QR코드

DOI QR Code

Generating Test Cases and Scripts from Requirements in Controlled Language

구조화된 자연어 요구사항으로부터 테스트 케이스 및 스크립트 생성

  • Received : 2018.12.14
  • Accepted : 2019.06.19
  • Published : 2019.08.31

Abstract

This paper proposes a method to generate test cases and test scripts from software requirements written in a controlled natural language, which helps develop reliable embedded software. In the proposed method, natural language requirements are written in a controlled language, the requirements are parsed and then inputs, outputs and operator data are extracted from the requirements. Test cases are generated from the extracted data following test case generation strategies such as decision coverage, condition coverage or modified condition/decision coverage. And then the test scripts, physical inputs of the test cases are generated with help of the test command dictionary. With the proposed method, it becomes possible to directly check whether software properly satisfies the requirements. Effectiveness of the proposed method is verified empirically with an requirement set.

본 논문은 신뢰성 있는 임베디드 시스템 소프트웨어 개발을 위해 제한된 자연어 형식으로 작성된 소프트웨어 요구사항으로부터 테스트 케이스 및 테스트 스크립트를 생성하는 방안을 제안한다. 제안하는 방법에서는 자연어로 기술된 요구사항을 제한된 자연어 형식으로 작성하고, 이를 파싱하여 테스트에 사용되는 입력, 출력 및 연산자를 추출한다. 추출된 정보를 이용하여 Decision Coverage, Condition Coverage, Modified Condition/Decision Coverage와 같은 테스트 케이스 생성 전략을 적용하여 테스트 케이스를 생성한다. 또한 테스트 명령어 사전을 이용하여 임베디드 시스템의 물리적인 입력 값인 테스트 스크립트를 생성한다. 제안한 방법을 이용하면, 개발된 소프트웨어가 요구사항에 적합하게 개발되었는지를 직접적으로 테스트하는 것이 가능하다. 제안한 방법의 효과는 요구사항 세트에 적용하여 실험적으로 보인다.

Keywords

JBCRJM_2019_v8n8_331_f0001.png 이미지

Fig. 1. Flow Diagram for Generating Test Cases and Test Scripts

JBCRJM_2019_v8n8_331_f0002.png 이미지

Fig. 2. A STK Template Example[10]

JBCRJM_2019_v8n8_331_f0003.png 이미지

Fig. 3. A Screen Shot for Correct Requirement

JBCRJM_2019_v8n8_331_f0004.png 이미지

Fig. 4. A Screen Shot for Incorrect Requirement

JBCRJM_2019_v8n8_331_f0005.png 이미지

Fig. 5. An Example of Input/Output Information

JBCRJM_2019_v8n8_331_f0006.png 이미지

Fig. 6. The Input Command Dictionary Format

JBCRJM_2019_v8n8_331_f0007.png 이미지

Fig. 7. Test Case & Test Script Generator Screen Shot

JBCRJM_2019_v8n8_331_f0008.png 이미지

Fig. 8. Input/Output Information for the Requirements

JBCRJM_2019_v8n8_331_f0009.png 이미지

Fig. 9. Requirement Examples

JBCRJM_2019_v8n8_331_f0010.png 이미지

Fig. 10. The Generated Test Cases Adequate to MCDC

JBCRJM_2019_v8n8_331_f0011.png 이미지

Fig. 11. The Command Dictionary for the Experiment

JBCRJM_2019_v8n8_331_f0012.png 이미지

Fig. 12. Generated Test Scripts

Table 1. STK System Rule Examples[10]

JBCRJM_2019_v8n8_331_t0001.png 이미지

Table 2. STK Writing Rules[10]

JBCRJM_2019_v8n8_331_t0002.png 이미지

Table 3. An Example of Data Extracted from Requirement

JBCRJM_2019_v8n8_331_t0003.png 이미지

Table 4. The Comparison Operators

JBCRJM_2019_v8n8_331_t0004.png 이미지

Table 5. The Command Operators

JBCRJM_2019_v8n8_331_t0005.png 이미지

Table 6. The Link Operators

JBCRJM_2019_v8n8_331_t0006.png 이미지

Table 7. The Converted Requirement

JBCRJM_2019_v8n8_331_t0007.png 이미지

Table 8. The Structured Requirement Data for Table 7

JBCRJM_2019_v8n8_331_t0008.png 이미지

Table 9. Test Case Examples for Different Test Strategies

JBCRJM_2019_v8n8_331_t0009.png 이미지

Table 10. The Structured Data to Generate Test Cases

JBCRJM_2019_v8n8_331_t0010.png 이미지

References

  1. Rolf Schwitter, "Controlled Natural Languages for Knowledge Representation," Coling 2010: Poster Volume, Beijing, pp. 1113-1121, 2010.
  2. L. H. Tahat, B. Vaysburg, B. Korel, and A. J. Bader, "Requirement-based automated black-box test generation," in Proceeding of the 25th Annual International Computer Software and Applications Conference, pp.489-495, 2001.
  3. P. S. Loo, and W. K. Tsai, "Random Testing Revisited," Information and Software Technology, Vol.30, Iss.7, pp.402-417, Sep. 1988 https://doi.org/10.1016/0950-5849(88)90037-7
  4. Rick Kuhn, Raghu Kacker, Yi Lei, and Justin Hunter, "Combinatorial Software Testing," IEEE Computer Society, Vol.42, Iss.8, pp.94-96, Aug. 2009.
  5. M. Conrad, H., Dorr, I. Fey, and A. Yap, "Model-based Generation and Structured Representation of Test Scenarios," Workshop on Software-Embedded Systems Testing (WSEST), Gaithersburg, USA, Nov. 1999.
  6. H. S. Park, "Generating Structural Test Cases for MATLAB Stateflow Model Using Rapidly-exploring Random Tree," Ajou Univ, Engineering doctoral dissertation, 2014.
  7. M. Utting, and B. Legeard, "Practical Model-Based Testing: A Tools Approach," Morgan kaufmann, 2007
  8. C. Denger and M. Mora, "Test case derived from Requirement Specifications," IESE-Report No.033.03/E version 1.0, Apr. 24 2003.
  9. Kang Tae Hoon, Kim Dae Joon, Chung Ki Hyun, and Choi Kyung Hee, "A Method to Automatically Generate Test Scripts from Checklist for Testing Embedded System," KIPS, Vol.5, No.12, pp.641-652, May 2016.
  10. K. W. Kwon, K. H. Chung, H. S. Yang, J. J. Jang, D. S. Lee, U. H. Jo, J. H. Shin, and H. J. Cho, "Rules of SRS for the SRS Adequacy Test," KIMST, 2018.
  11. A. P. Mathur, "Foundations of Software Testing," Pearson Education, 2008.
  12. Farrell, James A. (August 1995), "Compiler Basics: Extended Backus Naur Form," Archived from the original on 5 June 2011. Retrieved May 11, 2011.
  13. KOMORAN [Internet], https://github.com/shin285/KOMORAN
  14. Dae Joon Kim, Ki Hyun Chung, and Kyung Hee Choi, "A Hierarchical Checklist to Automatically Generate Test Scripts," KIPS, Vol.6, No.5, pp.245-256, Jun. 2017.
  15. H. D. Kim, H. J. Cho, J. H. Shin, K. H. Chung, and K. H. Choi, "Automatically Generating Simulink/Stateflow Model and Test Case from Requirements," KIMST, 2017.
  16. The Yices SMT Solver [Internet], http://www.csl.sri.com.
  17. Functional-Safety [Internet], https://www.iec.ch/functionalsafety/
  18. ASD Simplified Technical ENGLISH ASD-STE100 [Internet], http://www.asd-ste100.org/
  19. Alun Preece, Diego Pizzocaro, Dave Braines, David Mott, Geeth de Mel, and Tiem Pham, "Integrating Hard and Soft Information Sources for D2D Using Controlled Natural Language," In 2012 15th International Conference on Information Fusion, pp.1330-1337, Jul. 2012.
  20. FlaLavia A. Barros, Lai.s Neves, L Erica Hori and Dante Torres, "The ucsCNL: A Controlled Natural Language for Use Case Specifications," in SEKE, pp.250-253, 2011.