The goal of this project is to advance our knowledge of neural nets, through collective research.
Success will raise a secondary question, especially in academia: how should we share credit? This page lays out a commonsense approach based on the project life cycle. Our experience is that good credit assignment naturally flows from clear expectations during the course of a project. Please take a look at CONTRIBUTING.md first, to understand how contributions can be structured.
What happens when it's time to write up some exciting results?
For an academic paper, we recommend:
- Project owner(s) will be responsible for determining the author list according to contributions
- If there are other core contributors, consider adding as co-first authors.
- Anyone who made a significant contribution that is part of the write-up should be an author
- We suggest being generous with what counts as "useful"
- At the same time, it should be a real contribution, and you shouldn't assume you'll be an author if you just put your name on a page, or corrected a typo.
- Authorship should be discussed explicitly by the time writing starts.
- These conversations can feel awkward, but are important to have
- There are several senior academics on this project who can give advice if there's a conflict
Parallel experiments: Grad Student X has a hypothesis for how reasoning models do arithmetic, and realizes that the best way to test this is to do a series of complicated but separate experiments. They write a project page describing the research question and concrete methods, listing themselves as owner. Two college students realize they could help, and all three work on the experiments. When the paper is published, Grad Student X is lead author, and the two college students are also authors.
Growing team: Postdoc A documents a weird new neural net behavior, and has a guess for what it means. They write up a page about this, putting themselves as owner. The page attracts a lot of attention, and pretty soon a set of six people are working on experiments. One person in particular, Amateur Z, is putting in a lot of time, and has an idea that makes everyone rethink the project. Postdoc A sees this, and reaches out to Amateur Z to suggest they become an owner too. (This turns out to be helpful when Postdoc A takes a well-earned vacation, and Amateur Z keeps up with a deluge of results.) When they write up the paper, Postdoc A and Amateur Z decide to be joint first authors. As they write, Artist Alpha drops in and creates some beautiful diagrams of the results, which everyone wants in the paper. The final publication has many authors in addition to Postdoc A and Amateur Z, including Artist Alpha.
Resurrection: Professor Plum suggests a project, filling in "Plum" as the owner. After writing a page of notes and recording a few experimental results, Professor Plum forgets about the project. Three months later, Undergrad U sees the page, and has an inspired idea about what to do next based on the notes. After a conversation with Professor Plum, Undergrad U takes on ownership of the project, and after many more experiments writes up a paper. During this time, Professor Plum is on sabbatical at the University of the Virgin Islands and is unable to do much more than copyedit the paper. They discuss authorship, and agree that Undergrad U sould be first author; Professor Plum does remain an author, since the original results and ideas were important to the research.
Duplication: Project X and Project Y both have active, productive, and completely separate teams. They're studying different questions, doing related but not identical experiments, and they both discover something fascinating: Phenomenon 1. Professor Plum, browsing the wiki, realizes both projects are getting parallel results, and introduces the two teams. When the teams meet, they realize that together they now have a set of convincing evidence that Phenomenon 1 is deeply important. They agree to start writing their NeurIPS paper together, merging results, with multiple first authors and a fairly large author list overall. Professor Plum receives a hearty acknowledgment in the paper, but is not an author.
Just curious: Martin writes a research question asking for clarity on when "thinking models" switch between languages. He has no desire to do this research himself. However, three other people see this, start a project, and get some amazing results. Over the course of the project, Martin drops into the wiki and makes some encouraging comments and suggests a name for a certain phenomenon. When the paper is written up, the three project contributors are coauthors. Martin is not a coauthor (he didn't really contribute much, after all language-switching was a known phenomenon already) but he gets a nice mention in the acknowledgments.
Ultimately, credit assignment is a judgment call. And this kind of open research is still relatively unusual in the machine learning world. We strongly suggest approaching this with a generous attitude: please contribute without an expectation that any comment will automatically earn you authorship; on the flip side, when making decisions about authors, please take a broad view of what counts as a significant contribution.