posted by organizer: gulyx || 4502 views || tracked by 4 users: [display]

JSS SI on Test Automation 2021 : Special Issue on “Test Automation: Trends, Benefits, and Costs” - Journal of Systems and Software (Elsevier)


When N/A
Where N/A
Submission Deadline Mar 31, 2021
Notification Due Jun 30, 2021
Final Version Due Oct 1, 2021
Categories    special issue   software testing   software engineering   testing

Call For Papers

Today software has a significant impact on all aspects of our society, and its functioning is crucial for a multitude of economic, social, and educational activities. As a consequence, assuring the correctness and quality of software systems and components becomes paramount.

Automation across the Software-Testing Process is a powerful asset: while originally conceived for test execution, nowadays it is increasingly used for test generation, test prioritization and selection, test repair and evolution, among others, as well as for automatically comparing the actual with the expected outcome. Investments on automation of both tests synthesis and their execution are pursued to help ensure the adequate quality of software systems/applications while reducing the high effort and costs incurred in the testing of complex systems. De-facto, test automation is central in many modern approaches to software development such as model-driven engineering, agile frameworks, or TDD (Test Driven Development). It also represents a cornerstone for evolutionary development life-cycle focusing on “Continuous Practices” (i.e., continuous integration, testing, delivery, and deployment), or more recently on DevOps.

Nevertheless, as with any piece of software, development and maintenance of test code require considerable effort and skills. Besides, tests themselves need to be kept aligned with the ever-evolving system/application under test. Therefore, the potential benefits aimed by test automation have to be weighted against its costs and drawbacks.

In today's competitive world, companies demand a good Return On Investment (ROI) for every task concerning software development, and software testing is no exception. And in complex technical endeavours such as software testing, calculation of investments and returns cannot be reduced to just a balancing of money flow, but should also consider other non-financial, yet very concrete and critical aspects, such as social acceptance, developer's reputation, security and privacy, etc.

On the one side, research in test automation is very active, and in the last years, several solutions and tools have emerged that claim to improve the cost/effectiveness of testing. For instance, design patterns used for structuring test code, refactoring strategies and tools helping in development and maintenance. However, we are not yet able to foresee the investment costs versus money saved when a proposed test automation solution is introduced in a Company.

On the other side, managers demand evidence that proposed solutions actually improve the ROI and need applicable metrics to estimate the concrete costs/benefits of test automation. They would also need support in deciding which solution is best for their context, and often this is impeded by the large gap between an academic solution and the many concrete issues encountered in transferring it to practice, e.g., lack of scalability, flakiness, tight time-to-market. Moreover, inadequate testing and wrong decisions in test automation may be clearly the cause of a technical debt, and being able to reason on costs and benefits of test automation solutions can help prevent and reduce it.

Based on the above reasoning, we believe that research in test automation needs: i) to better investigate how to integrate means to assess the usefulness and applicability of new techniques and approaches; ii) to understand what are the issues that prevent broader adoption of existing tools or solutions, and iii) to collect evidence of (financial, technical and social) costs and benefits of the current trend and practices in test automation.

This special issue welcomes contributions regarding approaches, techniques, tools and experience reports about adopting and improving test automation cost/effectiveness, as well as empirical studies and case studies investigating the cost and benefits of test automation.


Topics of interest include, but are not limited to, the following:
* Approaches and tools for improving test automation effectiveness and ROI, for example by
** Reducing development and maintenance efforts/costs (e.g., automated generation of test code and self-healing test scripts)
** Increasing test coverage and bug detection

* Costs and benefits of Test Automation in modern systems and applications, for example:
** Web applications
** Mobile applications (e.g., Android, iOS)
** Internet of things systems
** Machine learning and artificial intelligence based systems

* Costs and benefits of Test Automation at any testing level, for example:
** End-to-End, acceptance, integration, functional, unit

* Measurement frameworks relevant for Test Automation
** Software and process metrics
** Technical debt in software testing

* Empirical studies concerning costs and benefits analysis and evaluating the cost-benefit trade-off of Test Automation (NO Surveys, SLR or Mapping Studies)
** Experiments
** Case studies
** Comparative experiences and evaluations


* Maurizio Leotta, University of Genova, Italy
* Guglielmo De Angelis, CNR-IASI, Italy
* Filippo Ricca, University of Genova, Italy
* Antonia Bertolino, CNR-ISTI, Italy


* Wing-Kwong Chan
* Raffaela Mirandola


* Paris Avgeriou
* David Shepherd

Related Resources

SI-DL4BDA 2023   Emerging Trends and Applications of Deep Learning for Biomedical Data Analysis
EDIT 2023   International Conference on Education & Integrating Technology
SI ETRACNS 2023   Special Issue on Emerging Trends and Recent Advances in Communication Networks Security
EDUIT 2023   International Conference on Education & Information Technology
IARCE 2023   2023 7th International Conference on Industrial Automation, Robotics and Control Engineering (IARCE 2023)
NLPI 2023   4th International Conference on NLP & Information Retrieval
ICBDB 2023   2023 5th International Conference on Big Data and Blockchain(ICBDB 2023)
NLPTT 2023   4th International Conference on NLP Trends & Technologies
SEAS 2023   12th International Conference on Software Engineering and Applications
ICACAR 2024   2024 7th International Conference on Advanced Control, Automation and Robotics (ICACAR 2024)