If you were to start an independent, lightweight, open-access journal, the main obstacle will be bootstrapping credibility. How do you go from an unknown, probably-spam, scare-quotes “journal” with zero issues to a venue that people might actually consider submitting to?
Part of the answer is clearly brute force: coerce good people into reviewing or submitting on blind faith, or be so famous that your name alone engenders reverence and awe. But the design of the venue’s purpose and operation also matter. There are better ways to begin than blithely going toe-to-toe against established venues.
Here are two ideas for making a new venue easier to trust—and, incidentally, more exciting.
Incentivize Good Reviews
Reviewing in computer science is not as effective as it could feasibly be. I can’t provide hard evidence for this perspective, since reviews are generally secret and sensitive, but I am not the only one who has it.
Reviewing is problematic not because reviewers are lazy or malicious; they are almost universally not. It’s primarily because our conferences are unintentionally architected to incentivize quick decisions and conservatism. High PC load, short PC meetings, secret reviews, limited author–reviewer dialogue, and the conference-to-conference resubmission cycle all contribute to an environment where the expedient reviewer looks for a low-hanging reason to reject (or delegates to a student to do so).
On the assumption that authors are as tired of reading rushed evaluations as reviewers are of writing them, a new venue could make itself attractive by carefully constructing its process to invert the incentives. I envision a journal where:
- Reviewers produce a public, signed review to accompany every accepted paper. Reviewing should be outward-facing, to add peer context to the authors’ own marketing, in addition to inward-facing, to convince an editorial committee to accept or reject. Attaching names to reviews will create a way for reviewers to cultivate a reputation as a careful observer of others’ work, a skill that we do not celebrate enough. (Reviewers would remain anonymous before publication to avoid retaliation.)
- A rolling deadline keeps reviewer load low. Instead of organizing papers into batches—conference programs or journal issues—the venue should review and publish papers without a fixed schedule. Each reviewer will be responsible for one or two papers at a time rather than dozens as current PC members are.
- Iterated revision is a dialogue between authors and reviewers. As in current journals, reviewers should be able to recommend substantial changes to the paper—and then review those changes after the author makes them. A good final product needs input from invested outsiders, of which the conference system’s reject-and-resubmit cycle is a remarkably inefficient implementation.
The prospect of a humane review process attracts me as a reviewer, and the promise of careful reviewing attracts me as an author. I hope the same is true of the community as a whole.
Address an Underserved Market
To avoid competing with existing conferences, a new venue should focus on kinds of papers that they do not serve well. Some subgenres that are a poor match for conferences in my community include:
- Interdisciplinary work: Research that is always “too X for a Y conference, and too Y for an X conference.” My community has ASPLOS, but this means that work that fits well neither at PLDI nor at ISCA has only one conference per year.
- Implementation-heavy work: Papers that bridge the gap between research and practice. This kind of work is notorious for garnering derision for “lack of novelty” even as it succeeds without publication.
- Pre-publication ideas: Short (say, three-page) gists of an upcoming project as a way to elicit early-stage feedback.
- Post-publication recognition: Authors nominate papers that have already appeared in an established conference. The venue recognizes and adds peer perspective to the “best” of the community, not unlike SIGPLAN Research Highlights, IEEE MICRO Top Picks, or PubMed Commons. (This idea is due to my advisor, Luis, who’s running Top Picks this year along with Karin Strauss.)
- A student-focused forum: For example, single-author papers in which students envision long-term research agendas (which their advisors may not approve of).
A new venue can build initial credibility by attracting papers that can’t go anywhere else. But even if it starts modestly—by highlighting existing publications, say—the venue can capitalize on an initial reputation to build something bigger. I am optimistic enough to believe that a niche publication that does it right could eventually expand to displace an established venue—or at least encourage it to evolve.