Quality Control Methods Using Quality Characteristics in Development and Operations †
Abstract
:1. Introduction
- Plan: Defines the task management and development requirements for the entire project;
- Code: Programmers create code according to development requirements;
- Build: The application that will actually run is built from the source code;
- Test: Test the built application for bugs and other defects;
- Deploy: Deploy the application into production;
- Operation: Perform maintenance and management tasks to ensure continuous service;
- Monitor: Review information obtained from operations, user assessments, requests, etc.
2. Considerations for Using Quality Model in Agile Development Processes
- Increased development productivity by being able to find features and performance and load degrades in the build pipeline;
- Increased testing efficiency through automation;
- Increased coverage of automated tests to capture more quality data.
3. Enhance of Management of Test Results for Agile Projects
4. Using Quality Characteristics for KPI under Agile Development Projects
- Designing quality requirements created at the time of the business requirements review and quality requirements for the design and development of the software. Ensure traceability of each issue so that the quality requirements created in the business requirements review can be checked for validity in design and evaluation;
- Add quality attribute items to Jira and create a Kanban that quality attributes can reference;
- Before starting a sprint, we created a sprint backlog by selecting tickets to be worked on by the sprint from the product backlog containing the business requirements and mapped the quality characteristics required by the business requirements.
- Define the quality requirements for each functional requirement and set the criteria by considering the metrics to be used;
- Evaluate the design and implementation checkpoints of the functional elements for each quality attribute;
- Establish evaluation criteria for each quality characteristic and map them to the quality characteristics to be ensured for each test activity;
- Visualize the progress of the established criteria by assessing the conformance of the quality characteristics at each sprint.
- Projected development scale: 2.6 KL;
- Review density: 20 man-hours/KL;
- Density of test items: 150 items/KL;
- Number of bugs: 10/KL.
5. Results of Two Practices
- When using quality characteristics in the agile process, it is assumed that if they are applied to a project for which a project plan has already been drawn up, the overhead of using quality characteristics is large and is likely to significantly affect the sprint’s activities;
- If the use of quality features is considered from the project planning stage, the following actions can be taken to provide evidence of quality assurance to project management;
- Declaring the use of quality features in the project charter;
- Assigning quality characteristics to different activities;
- The overhead of using quality characteristics should be included in the rough estimate before the project starts.
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- DevOps, 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr. 2009. Available online: https://www.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-cooperation-at-flickr (accessed on 15 January 2022).
- Jenkins. Available online: https://www.jenkins.io/ (accessed on 1 February 2020).
- ISO/IEC 25010; Systems and Software Engineering—Systems and Software Quality Requirements and Evaluation (SQuaRE)—System and software quality models. ISO: Geneva, Switzerland, 2011.
- ISO/IEC 25000; Systems and Software Engineering—Systems and Software Quality Requirements and Evaluation (SQuaRE)—Guide to SQuaRE. ISO: Geneva, Switzerland, 2014.
- Ito, J.; Yamaguchi, S.; Okazaki, K.; Yokosuka, S.; Kimoto, K.; Yamanaka, M.; Nagata, A.; Yamaguchi, T.; Hosoya, S. Quality Assurance by Quality Stepwise Refinement in Agile Development. Report on the Results of the Subcommittee Meeting of the Software Quality Control Research Group. 2018, pp. 137–144. Available online: https://www.juse.or.jp/sqip/workshop/report/at-tachs/2018/4_aqa_ronbun.pdf (accessed on 15 January 2022). (In Japanese).
- JIS X 25051; Software Engineering-Systems and Software Quality Requirements and Evaluation (SQuaRE)-Requirements for Quality of Ready to Use Software Product (RUSP) and Instructions for Testing. JISC: Tokyo, Japan, 2016.
- Cohn, M. The Forgotten Layer of the Test Automation Pyramid. Available online: https://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid/ (accessed on 1 February 2022).
- ISO/IEC 25030; Systems and Software Engineering—Systems and Software Quality Requirements and Evaluation (SQuaRE)—Quality Requirements Framework. ISO: Geneva, Switzerland, 2019.
- GitHub. Available online: https://github.com/ (accessed on 1 February 2022).
- Atlassian. Available online: https://www.atlassian.com/ (accessed on 1 February 2020).
- Kato, D.; Shimizu, A.; Ishikawa, H. Quality classification for testing work in DevOps. In Proceedings of the 14th International Conference on Management of Digital EcoSystems (ACM MEDES 2022), Venice, Italy, 19–21 October 2022. [Google Scholar]
- MSTest. Available online: https://github.com/Microsoft/testfx-docs/ (accessed on 1 February 2021).
- NUnuit. Available online: https://nunit.org/ (accessed on 1 February 2021).
- Ranorex. Available online: https://ranorex.com/ (accessed on 1 February 2020).
- Kato, D.; Ishikawa, H. Develop quality characteristics based quality evaluation process for ready to use software projects. Com-put. Sci. Inf. Technol. 2016, 6, 9–21. [Google Scholar] [CrossRef]
- ISO/IEC 25051; Software Engineering—Systems and Software Quality Requirements and Evaluation (SQuaRE)—Requirements for Quality of Ready to Use Software Product (RUSP) and Instructions for Testing. ISO: Geneva, Switzerland, 2014.
- ISO/IEC/IEEE 29119-3; Software and Systems Engineering—Software Testing—Part 3: Test Documentation. ISO: Geneva, Switzerland, 2021.
- Kato, D.; Okuyama, A.; Ishikawa, H. Introduction of test management based on quality characteristics. In Proceedings of the 1st Inter-national Workshop on Experience with SQuaRE Series & Their Future Direction IWESQ 2019), Putrajaya, Malaysia, 2 December 2019. [Google Scholar]
- Shang, W. Bridging the divide between software developers and operators using logs. In Proceedings of the 34th International Conference on Software Engineering (ICSE‘12), Zurich, Switzerland, 2–9 June 2012. [Google Scholar]
Metrics Types | Metrics |
---|---|
Defect density | Ratio per page of design documents |
Ratio per line or step of codes | |
Ratio per test cases | |
Defect removal rates | Removal rate per phases |
Removal rate per components | |
Removal rate per test cases | |
Pass rates | Pass rates per test sets |
Pass rates per test cases | |
Pass rates per function points | |
Coverages | Documents review coverage |
Test coverage per line of codes | |
Test coverage per test cases |
Activities | Quality Characteristics | Quality Sub-Characteristics |
---|---|---|
Activities in the Whole Development Process | ||
Coding Rules | Security | Confidentiality |
Integrity | ||
Maintainability | Modularity | |
Reusability | ||
Modifiability | ||
Testability | ||
Design Review | Functional Suitability | Function completeness |
Function correctness | ||
Reliability | Maturity | |
Availability | ||
Fault tolerance | ||
Recoverability | ||
Maintainability | Testability | |
Inspection Review | Reliability | Maturity |
Activities in CI process | ||
Static Analysis | Reliability | Maturity |
Security | Integrity | |
Maintainability | Analysability | |
Unit test Integration test Regression test | Functional Suitability | Function completeness |
Function correctness | ||
Function appropriateness | ||
Reliability | Maturity | |
E2E test | Functional Suitability | Function completeness |
Function correctness | ||
Function appropriateness | ||
Usability | Operability | |
User error protection | ||
Reliability | Maturity | |
Trough out the CI process | ||
Measurement of processing time | Performance Efficiency | Time behavior |
Resource utilization | ||
Maintainability | Analysability | |
Activity in CD operations | ||
install test | Portability | Installability |
Quality Characteristics | Quality Sub Characteristics | Evaluation by |
---|---|---|
Function Suitability | Functional Completeness |
|
Functional Correctness | ||
Functional Appropriateness | ||
Performance Efficiency | Time Behaviour | |
Resource Utilization | ||
Capacity | ||
Compatibility | Co-existence | |
Interoperability | - | |
Usability | Appropriateness Recognizability | - |
Learnability | - | |
Operability |
| |
User Error Protection | ||
User Interface Aesthetics | ||
Accessibility | - | |
Reliability | Maturity |
|
Availability | ||
Fault Tolerance | ||
Recoverability | - | |
Security | Confidentiality |
|
Integrity | ||
Non-repudiation | - | |
Authenticity | - | |
Accountability | - | |
Maintainability | Modularity |
|
Reusability | ||
Analysability | ||
Modifiability | ||
Testability | ||
Portability | Adaptability | - |
Installability |
| |
Replaceability | - |
Quality Characteristics | Quality Analysis |
---|---|
Function suitability | Functional suitability is ensured because the functional requirements that satisfy the functional requirements are considered in the design document, implemented, and confirmed to work properly through various tests. |
Performance efficiency | Compared to the operation before the enhancement, the performance of the functions implemented in the previous version has not deteriorated, and the import of new reports, a new function, is comparable to the import speed of the report functions implemented in the previous version, thus meeting the performance requirements. As load and capacity requirements have not been verified for this project, it can be concluded that ensuring only performance efficiency is not a problem. |
Compatibility | There are no problems with interfacing with other systems, such as EXCEL output and ZIP compression, and compatibility is ensured because there are no problems with co-existence with other systems in the same environment and no code-level implementation that would affect others. |
Usability | The enhancement does not create new screens but enhances existing screens and identifies no new usability issues, ensuring usability. |
Reliability | Reliability is ensured because functional requirements are met throughout the process: requirements->specification->design->implementation->testing The quality of the requirements developed during the requirements review is ensured by the associated activities, and any bugs found during the various reviews and tests are fixed. |
Security | The same level of security as the previous version has been confirmed by testing. The same level of security as the previous version is guaranteed. |
Maintainability | Maintainability is ensured because it is based on coding conventions, various documents are written in the same format as the previous version, and the level of description is written at the same or higher granularity than the previous version. |
Portability | The installation procedure is the same as the previous version. Portability is, therefore, guaranteed. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kato, D.; Ishikawa, H. Quality Control Methods Using Quality Characteristics in Development and Operations. Digital 2024, 4, 232-243. https://doi.org/10.3390/digital4010012
Kato D, Ishikawa H. Quality Control Methods Using Quality Characteristics in Development and Operations. Digital. 2024; 4(1):232-243. https://doi.org/10.3390/digital4010012
Chicago/Turabian StyleKato, Daiju, and Hiroshi Ishikawa. 2024. "Quality Control Methods Using Quality Characteristics in Development and Operations" Digital 4, no. 1: 232-243. https://doi.org/10.3390/digital4010012