Best Paper Award
Have you ever wondered how the best paper award is selected? So have we. This year round we spent a significant amount of time thinking about who should receive this years best paper award, and what we believe a fair selection process should look like for this conference and in the future.
An easy way might be to simply chose the paper with the highest review scores. However, this would penalize authors who received reviews by some of the more critical members of the program committee (you know who you are…:-D …). Another method which was suggested to us was the program chairs make this decision. This, however, would bias the selected paper towards the research we personally find interesting, which again, does not feel right.
With the help of our awards chair – Yuta Itoh – we came up with the following process: We put together an awards committee which combines all levels of seniority: from researchers who have recently received their PhD to senior members of our Steering committee. Committee members are from a range of geographic locations, and their research represents a range of approaches towards human augmentation.
Based on the review scores, the program chairs selected 10 papers which might potentially receive the award. The committee are currently ranking these. The final ranking will be calculated based on the harmonic mean of the rankings (Why the harmonic mean? Who are the committee members? Scroll to the bottom of this post for more info).
Lasting Impact and Special Recognition Award
We are formalizing the award processes, looking forward to future conferences. However, we also have 10 successful conferences to look back on. Reflecting on the 1st Augmented Human Conference, ten years ago, we now have a better idea which papers were impactful. A simple way of detecting impact is by looking at citations. We will award the paper from 10 years ago with the most citations with the Lasting Impact Award.
However, as we recognize that citation-counts are a poor measure of true impact, we will also present a second award, which we call Special Recognition Award. This award will be given to work published at Augmented Human conferences, which has had a substantial impact in ways not reflected in the citation count. This award will not necessarily be presented annually, and may be presented to more than one paper.
How will these awards be selected? The Lasting Impact Award will be chosen based on citation count, as per Google Scholar. Should the result be ambiguous – for example, if two papers have only marginally different citation counts – the steering committee will vote on the paper they believe has had the larger impact.
The Special Recognition Award will be selected by the Steering Committee. Should the steering committee be unable to decide on a single paper, the Steering Committee can opt to award no papers, or multiple papers. Anyone can nominate a paper for the Special Recognition Award. To do so, please send an e-mail to ahs20 at easychair dot org. Please indicate “Special Recognition Award” in the subject. In the e-mail, please provide title of the paper, and please explain why it should receive the award.
Other Awards
We originally had plans for a Best Talk, Best Poster and Best Demo award. Due to the nature of this years conference, We have decided not to present these awards.
Best Paper Committee
Yuta Itoh (Chair)
Masahiko Inami
Pedro Lopes
Cindy Hsin-Liu Kao
Aske Mottelson
Yamen Saraiji
Marc Teyssier
Why Harmonic Mean?
We wish to recognize excellence and bold ideas. These are prone to be controversial. We would prefer to give the award to a paper which 3 people loved, and 2 people hated, than to a paper which everyone felt look-warm about. Also, while we assume that the difference between the paper ranked #1 and ranked #2 is very relevant, we probably care less about the difference between #9 and #10.
Let’s compare some hypothetical scores:
Paper A: 2, 3, 3, 3, 4
Paper B: 1, 1, 1, 2, 8
Paper C: 1, 1, 4, 5, 5
Paper D: 2, 4, 5, 4, 5
Intuitively papers B & C appear to be strong contenders for the award, because these were the papers that committee members felt were #1. Paper A seems like a strong contender, but nobody actually thought that it clearly deserved first place. Paper D should probably not get the award.
These papers would have the following arithmetic means (aM) and harmonic means (hM):
Paper A: aM 3.0, hM 2.8
Paper B: aM 3.2, hM 1.4
Paper C: aM 3.2, hM 1.9
Paper D: aM 4.0, hM 3.6
We see that, using the arithmetic mean, paper A, which was nobodies first choice, has the best score. The papers which ranked as #1 by committee members, are in second and third place. Using the harmonic mean, this is no longer the case, the papers which received the most #1 placements, are now ranked highest. Paper A, which is solid but nobodies first choice, no longer wins the award. Paper D comes in last – as it should – using both methods. We believe the harmonic mean is best suited for selecting an excellent paper, which presents a bold idea.
by Paul Strohmeier
