Experimentation in Software Engineering
Software engineering technologies are not being properly evaluated. That is, professionals do not know for sure whether a technology is effective or not and, if so, cannot be sure how effective and applicable it is. This lack of proper evaluation undermines the ability of the industry to produce competitive quality software.
Experimental software engineering (ESE) is a discipline of Software Engineering that aims to produce reliable information for professionals about what technologies should be used in software development projects. ESE uses empirical studies (experiments, quasi-experiments, case studies, etc.) to evaluate the effectiveness of technologies for software development.
The objective of this subject is to train students in the basic skills required to apply empirical methods. We focus on experiments as they are the most mature and well-understood type of empirical study in SE. Students will learn how to perform, analyse, aggregate and replicate experiments (in industry and academic environments).
- Understand the current limitation of software engineering technologies because they are not researched scientifically
- Understand the meaning of experimental laboratory and experiment in software engineering
- Know the different elements of a software engineering experiments: response variable, factors, levels, etc.
- Design experiments for software engineering technologies: randomization, other control strategies, types of designs, etc.
- Understand software engineering experiment reports
- Apply basic statistical data analysis techniques (i.e. t-test, ANOVA, sample size calculation)
- Introduction to experimental software engineering
- Basics of experimentalism
- The scientific method
- Scientific Rules: Cause-effect relationships
- Scientific immaturity of software engineering
- Laboratory and experiment
- The concept of laboratory
- The concept of experiment
- A lab for software engineering
- An experiment for software engineering
- Elements of an experiment
- Response variables
- Factors and levels
- Types of empirical studies
- Designing experiments
- Types of variables
- Types of control
- Data analysis
- Basics of inferential statistics
- Parametric tests for independent samples
- Parametric tests for related samples
- Non-parametric tests
- Natalia Juristo; Ana Moreno: "Basics of software engineering experimentation". Kluwer 2001
- Claes Wohlin, Per Runeson: "Experimentation in software engineering: An introduction". Kluwer 2000
- Basic knowledge of statistics.
|Examination of the state of the art to identify research problems related to the design, construction, use and evaluation of complex software-intensive sociotechnical systems
|Application of relevant research methods to open problems in the field of software engineering related to both the particular features of the software and software development management
|Understand the application of the experimental paradigm in software engineering
|Design software engineering experiments, including experiment replications
|Aggregate Software Engineering experiments