Frequently Asked Questions
General
Q: I thing TMLR is great, and I'd like to be involved! How can I sign up to be a reviewer (or Action Editor)?
A: Great! We have filled our reviewer and AE pools for now, but you can solicit to review any submission via OpenReview (a solicit review button should be visible if you are logged in). You can also ask a current Action Editor that can vouch for your qualifications to add you as a reviewer. If you would like to be an Action Editor and believe that you are qualified through your academic credentials as a professor or through your experience (e.g. as an Area Chair or Editor), please contact the Editors-in-Chief.
Q: Will there be multiple back-and-forth rounds between reviewers and authors, as in some journals, or just one round as with most conferences?
A: There is one discussion phase, but during that phase there can be as much (and as frequent) back and forth, including revisions to the manuscript, as the authors and reviewers need. The discussion phase will be at least 2 weeks and up to 4 weeks long.
Q: How are reviewers assigned? Will it be entirely up to the action editors, or will there be a bidding process?
A: Assignments will largely be up to action editors. However, because submissions are shown publicly on OpenReview, a user who is not in conflict with the submission may request to be a reviewer (and it will be up to the action editor to accept the request).
Q: How does the double blind policy work? Would papers already on arXiv be allowed? How about papers that are visible on authors' websites, etc.?
A: You can still submit, even if a preprint of your work already is available online somewhere (as long as it's not in official proceedings for some other publishing venue). We follow the same approach as double blind conferences, asking that reviewers don't actively search online to figure out who the authors are.
Q: Will other artifacts of a paper be available, such as software, data, talk recordings or videos, slides, public discussions?
A: Code and videos may be uploaded and associated with camera ready papers. This may expand over time.
Q: Are you using Toronto paper matching in the review system?
A: OpenReview has its own paper matching system to rank similarity of reviewers and papers, and we will be leveraging it.
Q: Will accepted TMLR papers have the opportunity to be presented in ML conferences like NeurIPS, ICML, and ICLR?
A: We hope that TMLR will eventually partner with workshops and conferences (see blog post). We don't have such agreements with these conferences, but we'd like to down the road.
Q: Will non-anonymous OpenReview users be able to comment on the paper during the review process?
A: Yes.
Q: Can I request to anonymously review a submission?
A: Yes. For this, log in to OpenReview, visit the page of the submission and click on button "Solicit Review". The AE in charge of the submission will receive a notification asking them to accept or deny your request.
Q: Does TMLR have a preference for methodological/theoretical work or will application papers be welcome?
A: TMLR does not have a preference. As long as the paper is within scope and we have an action editor who is qualified to handle a submission, we should be open to receiving it.
Q: Does TMLR accept survey papers?
A: Yes. Authors should make sure to emphasize the contributions made by the survey. Ideally, we want survey papers that draw new, previously unreported connections between several pieces of work in an area, and/or that clearly highlight trends in the area and/or suggest currently open problems. It should also be noted that if a submission has more than 12 pages of main content, then TMLR's normal short review timeline will not be enforced.
Authors
Q: How is TMLR different from other machine learning journals such as JMLR?
A: TMLR provides a similar experience to submitting to a conference – double-blind, open access reviews, fast turnaround, and novel, unpublished content – but with a rolling, year-round review process and evaluation criteria that focuses on technical correctness of claims rather than more subjective, editorial measures. Consider submitting shorter manuscripts to TMLR, and submitting extended works which incorporate previously published material to JMLR.
Q: How long will it take to review my manuscript?
A: TMLR does not make any guarantees, and manuscripts whose main body exceeds 12 pages are subject to longer timescales, but in general the rolling review process aims to deliver a final decision approximately 9 weeks after submission.
Q: Are replications of previous research welcome?
A: Yes.
Q: Can I include the appendix in the main pdf so that hyperlinks are preserved, or should it go in the supplementary zip file?
A: Appendices can be included in the main pdf. The supplementary zip file should contain additional material such as videos or code.
Q: Can I use the TMLR style and template if I upload my paper to arXiv?
A: We don't have a policy proscribing the use of our template on arXiv. However, make sure to use the "preprint" option of the stylefile if you do.
Reviewers
Q: How long will reviewers have to complete their review?
A: We ask for reviews to be submitted within 2 weeks, followed by 2 weeks of discussion with the authors. Exceptions will be made for long papers (>12 pages of main content, excluding appendices). While the timeframe will be tight, we're trying to compensate by ensuring a low workload at any given time (no more than 1 paper per reviewer).
Q: When submitting my review, the form isn't asking for a decision recommendation or a score. Is that normal?
A: Yes. Initially, we don't ask for a formal recommendation from you, so you can keep your mind open during the discussion period with the authors. However, two weeks after the 3rd review for a submission has been submitted, you will be asked to separately submit to the Action Editor an official recommendation for a decision on the submission.
Q: What if I have seen the paper at a workshop, on arXiv, on Twitter, etc, and I know who the authors are?
A: You should not try to de-anonymize any submission by actively searching for it. However, you may come across it by chance. If this happens, and you believe that it will influence your judgment, please contact the action editor, who will help decide whether you should be replaced with a new reviewer.
Q: The submission has an appendix and supplementary material. Am I obligated to consult it?
A: Much like for reviewing at conferences, consulting appendices and supplementary material is left to the discretion of the reviewer.
Q: Can I discuss the paper privately with the other reviewers and/or the Action Editor?
A: Yes, but only once all 3 reviewers have posted their review. Simply post a comment that is only visible to them to start the discussion.
Q: How can I influence what type of papers I'll be asked to review?
A: Go in your openreview account profile and click on Edit (https://openreview.net/profile/edit) and then if you scroll down you'll see your imported publications. You can remove those that you don't want to influence which papers are assigned to you.
Action Editors
Q: Can I invite someone to join TMLR's pool of reviewers?
A: Yes, this can be done through the AE Console on OpenReview, by clicking on link "Recruit Reviewer" and then on button "Reviewer Recruitment by AE".
Q: Can I invite someone to review a specific submission I'm in charge of, without them joining TMLR's general pool of reviewers?
A: Yes, simply ask them to visit the submission's OpenReview page, where they can request to review the submission by clicking on button "Solicit Review". You'll then receive a notification asking you to accept their request. Once you do, they will become an official reviewer for that submission.
Q: I am concerned that a submission, which is otherwise acceptable, violates the TMLR ethical guidelines; how should I handle this?
A: Consider whether the submission would be acceptable if the authors added or improved a broader impact statement, and if so, this can be required for the camera-ready version. If the submission carries significant risk of harm, contact the Editors-in-Chief.
Q: Three reviewers have now submitted their official recommendations. I want to submit my decision, but I don't seem able to do it on OpenReview. What should I do?
A: Before you can submit your decision, you must rate the quality of the reviews submitted by the reviewers, on OpenReview. Once you have submitted these ratings, the submission of your decision will be enabled on OpenReview.
| © TMLR 2022. |