Experimentation in Software Engineering
Software engineering technologies are not being properly assessed, i.e., practitioners do not really know whether or not a technology is effective, and, if they do, they cannot be sure how effective and applicable it is. This lack of proper assessment is seriously undermining industry’s capability to produce competitive and quality software.
Experimental software engineering (ESE) is a branch of software engineering that aims to produce reliable information for practitioners about which technologies should be used in software development projects. ESE uses empirical studies (experiments, quasi-experiments, case studies, etc.) to assess the effectiveness of software technologies.
The objective of this subject is to train students in the basic skills required to apply empirical methods. We focus on experiments as they are the most mature and well-understood type of empirical study in SE. Students will learn how to perform, analyse, aggregate and replicate experiments (in industry and academic environments).
- Understand the current limitation of software engineering technologies because they are not researched scientifically
- Understand the meaning of experimental laboratory and experiment in software engineering
- Know the different elements of a software engineering experiments: response variable, factors, levels, etc.
- Design experiments for software engineering technologies: randomization, other control strategies, types of designs, etc.
- Understand software engineering experiment reports
- Apply basic statistical data analysis techniques (i.e. t-test, ANOVA, sample size calculation)
- Introduction to experimental software engineering
- Basics of experimentalism
- The scientific method
- Scientific Rules: Cause-effect relationships
- Scientific immaturity of software engineering
- Laboratory and experiment
- The concept of laboratory
- The concept of experiment
- A lab for software engineering
- An experiment for software engineering
- Elements of an experiment
- Response variables
- Factors and levels
- Types of empirical studies
- Designing experiments
- Types of variables
- Types of control
- Data analysis
- Basics of inferential statistics
- Parametric tests for independent samples
- Parametric tests for related samples
- Non-parametric tests
- Natalia Juristo; Ana Moreno: "Basics of software engineering experimentation". Kluwer 2001
- Claes Wohlin et al.: "Experimentation in software engineering: An introduction". Kluwer 2000
- Basic knowledge of statistics.
- Students will be assessed on the basis of assignments only. Students shall not have to sit an exam.
- The assessment of assignments will depend on: (1) a presentation made by students and (2) the correctness of the results.
- The final grade will be calculated using a weighted average.
|CEM1||Examination of the state of the art to identify research problems related to the design, construction, use and evaluation of complex software-intensive sociotechnical systems||A|
|CEM3||Application of relevant research methods to open problems in the field of software engineering related to both the particular features of the software and software development management||A|
|Code||Learning Outcome||Associated competences||Proficiency level|
|RA-IS-23||Understand the application of the experimental paradigm in software engineering||CEM1, CEM3||A|
|RA-IS-24||Design software engineering experiments, including experiment replications||CEM1, CEM3||A|