Master in Software and Systems

Experimental Software Engineering

Lecturer (Coordinator):
Sira Vegas
Natalia Juristo


Second semester




Software engineering technologies are not being properly assessed, i.e., practitioners do not really know whether or not a technology is effective, and, if they do, they cannot be sure how effective and applicable it is. This lack of proper assessment is seriously undermining industry’s capability to produce competitive and quality software.

Experimental software engineering (ESE) is a branch of software engineering that aims to produce reliable information for practitioners about which technologies should be used in software development projects. ESE uses empirical studies (experiments, quasi-experiments, case studies, etc.) to assess the effectiveness of software technologies.

The objective of this subject is to train students in the basic skills required to apply empirical methods. We focus on experiments as they are the most mature and well-understood type of empirical study in SE. Students will learn how to perform, analyse, aggregate and replicate experiments (in industry and academic environments).

Learning Goals


  1. Introduction to experimental software engineering
    1. Basics of experimentalism
    2. The scientific method
    3. Scientific Rules: Cause-effect relationships
    4. Scientific immaturity of software engineering
  2. Laboratory and experiment
    1. The concept of laboratory
    2. The concept of experiment
    3. A software engineering lab
    4. A software engineering experiment
  3. Elements of an experiment
    1. Response variables
    2. Factors and levels
    3. Types of empirical studies
  4. Designing experiments
    1. Types of variables
    2. Control variables
    3. Validity
  5. Data analysis
    1. Basics of inferential statistics
    2. Parametric tests for independent samples
    3. Parametric tests for related samples
    4. Non-parametric tests

Recommended reading


Assessment method

Tuition language


Subject-Specific Competences

More information:

This table shows the code, description and proficiency level for each subject-specific competence

Code Competence Proficiency Level
SSC1 Examination of the state of the art to identify research problems related to the design, construction, use and evaluation of complex software-intensive sociotechnical systems A
SSC3 Application of relevant research methods to open problems in the field of software engineering related to both the particular features of the software and software development management A

Learning Outcomes

More information:

This table shows the code, description and proficiency level for each subject learning outcome

Code Learning Outcome Associated competences Proficiency level
RA-IS-9 Understand the application of the experimental paradigm in software engineering SSC1, SSC3 A
RA-IS-10 Design software engineering experiments, including experiment replications SSC1, SSC3 A

Learning Guide

Subject learning guide for Experimental Software Engineering