Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Artifact Evaluation Process: Communication #5

Open
jon-bell opened this issue Nov 24, 2019 · 3 comments
Open

Artifact Evaluation Process: Communication #5

jon-bell opened this issue Nov 24, 2019 · 3 comments
Labels
policy An issue about refining a policy

Comments

@jon-bell
Copy link
Collaborator

How much communication should be allowed between evaluators and authors during the review period?

Examples range from:

  • No communication other than the authors submit an artifact and eventually get reviews
  • A "kick-the-tires" phase, in which evaluators can do some sanity checking, ask questions, and get a response (one time).
  • Open discussion, where evaluators can freely discuss with authors (blind or not)
@jon-bell jon-bell added the policy An issue about refining a policy label Nov 24, 2019
@sbaltes
Copy link

sbaltes commented Nov 30, 2019

I very much prefer the latter, which I experienced during the FSE 2018 artifact evaluation. In that case, they used GitHub issues to discuss the artifacts using anonymized user accounts.

@piskachev
Copy link

At least "kick-the-tires" phase should be there.

@wagnerst
Copy link

Fully open, in my opinion. There are always clarifications necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
policy An issue about refining a policy
Projects
None yet
Development

No branches or pull requests

4 participants