Speaker: Andrea Arcuri , Scienta, Norway, and the University of Luxembourg
Area: Empirical Studies in Testing and Analysis
When and where: Monday, July 18, 16:05 – 17:35 at DFKI
You developed technique X and compared to Y to show it is better. Just in case, you repeat your experiments 10 times. X solves the problem 7 times, whereas Y only 5. Great! Your technique is better by 20%, and you proudly publish it at ISSTA. Congratulations, you just added your name to a potentially rubbish study! Yes, you read right, I used the word “rubbish”… although the main key word was actually “potentially”. It might well be that your technique is indeed great, and likely it is, but you just have shown no compelling evidence to support it. The +20% is quite meaningless, and it might well be that Y was actually better.
If this statement puzzles you, then great, this seminar is for you! With some (I promise) basic probability analysis, hopefully I will manage to convince you that the usage of statistics is an essential requirement in most empirical studies. Furthermore, if you have ever felt the pain of adding your experiment data to a Microsoft Word document by hand again and again, then you might find beneficial to see how to fully automate it (plus all needed statistics) in R and Latex. To most benefit from this seminar, it is best if you bring a laptop with installed Git, Latex, R and a bash shell.
More on Andrea:
Dr. Andrea Arcuri received his PhD in software testing from the University of Birmingham, UK, in 2009. He then worked as research scientist in software testing at Simula Research Laboratory, Norway. Afterwards, for few years he had been a senior software engineer at Schlumberger, an oil & gas service company. He is currently working as a Test Leader consultant, with clients like Telenor (the largest Scandinavian telecom).
Dr. Arcuri also works as research fellow at the University of Luxembourg, doing research on software testing topics. His main research interests are on software testing topics with practical relevance on industry, and on how empirical studies should be carried out and analyzed with the appropriate statistical methods.