Learning from Authoritative Security Experiment Results

Workshop Goals

While everyone has a notion of what science is – sound science is neither widely nor well-practiced. The goal of this forum is to help the community change that.

To effect such change, we need to be specific about what constitutes sound science, and about what it means to practice it.  "Science" is the process of linking facts and fact-based theory, across disciplines, to create a common framework of understanding. Such frameworks can be informed by both positive and negative results. While the scientific process clearly includes both theory and practice (where practice is usually in the form of experimentation), LASER's primary focus is on experimentation. There are three essential issues, each of which will play a critical role in determining the suitability of papers submitted:

(1) Is it science?  How persuasive is the author's (possibly implicit) claim that the work is scientific, i.e., that it links theory and practice (via experimentation)?  What is the theory and how does the experiment inform it, or vice versa, what is the experiment, and how is it informed by theory?

(2) Is it well executed?  A paper or project can have a decidedly scientific bent, as previously defined, and yet be poorly executed. Good (experimental) scientific execution includes: a clear statement of a research question with an explicit claim or hypothesis or problem being solved; an experimental study type or design suited to addressing that research question (e.g., observational, random controlled trial, etc.); adequate sample size; proper statistical analysis; freedom from confounds; sound methodology; rigorous collection of data (where appropriate); a reproducible experiment; and sensible, justifiable conclusions that are well supported by experimental evidence.

(3) Is it well reported?  Any scientifically sound experiment is of little use if it is not properly and thoroughly reported.  A good report/paper includes at least a structured abstract (details below); introduction and background; related work; problem being solved and/or hypothesis being tested; experimental methods (where appropriate, methods include apparatus and instrumentation, materials, subjects/objects of study, instructions to subjects, design, procedure); data; analysis; results; discussion; and conclusion.

Referees will be asked to judge the merits of papers regarding the following:

• Is the title appropriate for the work described?
• Is there a structured abstract?
• Does the paper contain all of the sections you would expect (Introduction, Related work, Theory, Methods, Analysis, etc.)?
• Is there a clear statement of the problem being solved or the hypothesis being tested or the research question being asked?
• Does the author do a good job of synthesizing the literature?
• Is the experimental work reproducible?
• Is the methodology clearly explained (including details noted above)?  A complete methodology bears on reproducibility.
• Is the paper well-organized?
• Are the sections well-developed?
• Does the author answer the questions s/he sets out to answer?
• Does this science represent a meaningful contribution to the literature?
• Does the theory connect to the data?
• Is the paper well-written and easy to understand?
• Are you convinced by the author's results? Why or why not?

Referees are instructed to base their judgments on the scientific merits of the papers, consistent with the goals of the workshop.

The best papers meeting the above criteria will be accepted for presentation and publication. Papers falling modestly short of these criteria will be accepted for a works-in-progress embedded workshop and tutorial on executing and reporting good science. Please note that all guidelines in this Call for Papers apply equally to papers in plenary and works-in-progress sessions.

Quod gratis asseritur, gratis negatur.
“What is asserted without reason may be denied without reason.”

Embedded Works-in-Progress Workshop

To help authors improve the scientific soundness of their research in the interest of their submitting papers in the future, to LASER or to other venues, there will be experienced senior researchers that will interact with the presenter as well as the audience to contribute constructive feedback on how to make the work more rigorous and more scientifically sound.

Travel Scholarships

Travel support is available in modest amounts for students in need.

Important Dates

Submission and Review Process

Full papers are solicited and must follow the posted workshop paper guidelines. Papers follow a typical pattern of submission, review, notification, pre-conference version, conference presentation, and final post-conference version. 

All papers must be submitted via OpenConf.

At least one author from every accepted paper must attend the workshop and present the paper. 

The LASER workshop is funded in part by NSF Grant #1143766 and by the Applied Computer Security Associates (ACSA).

Please direct all questions to info@laser-workshop.org.