How do conferences / journals select reviewers?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












When you submit a paper / manuscript to a peer-reviewed venue, the organizers usually forward your work to 2-3 other "experts in your field" to assess your work.



I'm curious how this process of selection works, i.e. how is this pool of reviewers put together? and how are the best people from this pool chosen once there is a paper to review?



Is there an official list that each venue maintains and to which people can sign on if they want to be reviewers? (If so, I've never seen any "calls for reviewers" so how do they get people?)



Do organizers cold-call people/professors and ask them if they were willing to review a paper? If so, how are these people selected? [I realize that this might differ considerably, so I'm happy with anecdotal evidence, too]










share|improve this question

























    up vote
    1
    down vote

    favorite












    When you submit a paper / manuscript to a peer-reviewed venue, the organizers usually forward your work to 2-3 other "experts in your field" to assess your work.



    I'm curious how this process of selection works, i.e. how is this pool of reviewers put together? and how are the best people from this pool chosen once there is a paper to review?



    Is there an official list that each venue maintains and to which people can sign on if they want to be reviewers? (If so, I've never seen any "calls for reviewers" so how do they get people?)



    Do organizers cold-call people/professors and ask them if they were willing to review a paper? If so, how are these people selected? [I realize that this might differ considerably, so I'm happy with anecdotal evidence, too]










    share|improve this question























      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      When you submit a paper / manuscript to a peer-reviewed venue, the organizers usually forward your work to 2-3 other "experts in your field" to assess your work.



      I'm curious how this process of selection works, i.e. how is this pool of reviewers put together? and how are the best people from this pool chosen once there is a paper to review?



      Is there an official list that each venue maintains and to which people can sign on if they want to be reviewers? (If so, I've never seen any "calls for reviewers" so how do they get people?)



      Do organizers cold-call people/professors and ask them if they were willing to review a paper? If so, how are these people selected? [I realize that this might differ considerably, so I'm happy with anecdotal evidence, too]










      share|improve this question













      When you submit a paper / manuscript to a peer-reviewed venue, the organizers usually forward your work to 2-3 other "experts in your field" to assess your work.



      I'm curious how this process of selection works, i.e. how is this pool of reviewers put together? and how are the best people from this pool chosen once there is a paper to review?



      Is there an official list that each venue maintains and to which people can sign on if they want to be reviewers? (If so, I've never seen any "calls for reviewers" so how do they get people?)



      Do organizers cold-call people/professors and ask them if they were willing to review a paper? If so, how are these people selected? [I realize that this might differ considerably, so I'm happy with anecdotal evidence, too]







      journals peer-review conference paper-submission






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 1 hour ago









      FirefoxMetzger

      1634




      1634




















          3 Answers
          3






          active

          oldest

          votes

















          up vote
          4
          down vote













          In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:



          1. They have published with us before

          2. They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board

          3. Our associate editors approached them through a scan of authors of papers on the same topic

          4. They have volunteered to be placed on our list of reviewers

          5. They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)

          Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.



          As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.



          In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.






          share|improve this answer
















          • 2




            For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
            – David Ketcheson
            43 mins ago


















          up vote
          1
          down vote













          Answering from the perspective of computer science conferences:



          There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.



          Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.






          share|improve this answer



























            up vote
            1
            down vote













            Examples from my experience:



            • Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).

            • Author-suggested reviewers (which we try to use sparingly since they often know the author personally)

            • Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)

            • From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.

            • Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).

            • Personal contacts.

            To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).



            Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.






            share|improve this answer




















              Your Answer







              StackExchange.ready(function()
              var channelOptions =
              tags: "".split(" "),
              id: "415"
              ;
              initTagRenderer("".split(" "), "".split(" "), channelOptions);

              StackExchange.using("externalEditor", function()
              // Have to fire editor after snippets, if snippets enabled
              if (StackExchange.settings.snippets.snippetsEnabled)
              StackExchange.using("snippets", function()
              createEditor();
              );

              else
              createEditor();

              );

              function createEditor()
              StackExchange.prepareEditor(
              heartbeatType: 'answer',
              convertImagesToLinks: true,
              noModals: false,
              showLowRepImageUploadWarning: true,
              reputationToPostImages: 10,
              bindNavPrevention: true,
              postfix: "",
              noCode: true, onDemand: true,
              discardSelector: ".discard-answer"
              ,immediatelyShowMarkdownHelp:true
              );



              );













               

              draft saved


              draft discarded


















              StackExchange.ready(
              function ()
              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2facademia.stackexchange.com%2fquestions%2f118620%2fhow-do-conferences-journals-select-reviewers%23new-answer', 'question_page');

              );

              Post as a guest






























              3 Answers
              3






              active

              oldest

              votes








              3 Answers
              3






              active

              oldest

              votes









              active

              oldest

              votes






              active

              oldest

              votes








              up vote
              4
              down vote













              In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:



              1. They have published with us before

              2. They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board

              3. Our associate editors approached them through a scan of authors of papers on the same topic

              4. They have volunteered to be placed on our list of reviewers

              5. They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)

              Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.



              As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.



              In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.






              share|improve this answer
















              • 2




                For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
                – David Ketcheson
                43 mins ago















              up vote
              4
              down vote













              In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:



              1. They have published with us before

              2. They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board

              3. Our associate editors approached them through a scan of authors of papers on the same topic

              4. They have volunteered to be placed on our list of reviewers

              5. They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)

              Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.



              As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.



              In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.






              share|improve this answer
















              • 2




                For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
                – David Ketcheson
                43 mins ago













              up vote
              4
              down vote










              up vote
              4
              down vote









              In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:



              1. They have published with us before

              2. They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board

              3. Our associate editors approached them through a scan of authors of papers on the same topic

              4. They have volunteered to be placed on our list of reviewers

              5. They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)

              Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.



              As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.



              In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.






              share|improve this answer












              In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:



              1. They have published with us before

              2. They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board

              3. Our associate editors approached them through a scan of authors of papers on the same topic

              4. They have volunteered to be placed on our list of reviewers

              5. They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)

              Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.



              As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.



              In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.







              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered 48 mins ago









              St. Inkbug

              2,830726




              2,830726







              • 2




                For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
                – David Ketcheson
                43 mins ago













              • 2




                For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
                – David Ketcheson
                43 mins ago








              2




              2




              For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
              – David Ketcheson
              43 mins ago





              For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
              – David Ketcheson
              43 mins ago











              up vote
              1
              down vote













              Answering from the perspective of computer science conferences:



              There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.



              Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.






              share|improve this answer
























                up vote
                1
                down vote













                Answering from the perspective of computer science conferences:



                There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.



                Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.






                share|improve this answer






















                  up vote
                  1
                  down vote










                  up vote
                  1
                  down vote









                  Answering from the perspective of computer science conferences:



                  There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.



                  Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.






                  share|improve this answer












                  Answering from the perspective of computer science conferences:



                  There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.



                  Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered 36 mins ago









                  Thomas

                  7,68731733




                  7,68731733




















                      up vote
                      1
                      down vote













                      Examples from my experience:



                      • Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).

                      • Author-suggested reviewers (which we try to use sparingly since they often know the author personally)

                      • Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)

                      • From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.

                      • Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).

                      • Personal contacts.

                      To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).



                      Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.






                      share|improve this answer
























                        up vote
                        1
                        down vote













                        Examples from my experience:



                        • Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).

                        • Author-suggested reviewers (which we try to use sparingly since they often know the author personally)

                        • Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)

                        • From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.

                        • Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).

                        • Personal contacts.

                        To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).



                        Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.






                        share|improve this answer






















                          up vote
                          1
                          down vote










                          up vote
                          1
                          down vote









                          Examples from my experience:



                          • Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).

                          • Author-suggested reviewers (which we try to use sparingly since they often know the author personally)

                          • Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)

                          • From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.

                          • Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).

                          • Personal contacts.

                          To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).



                          Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.






                          share|improve this answer












                          Examples from my experience:



                          • Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).

                          • Author-suggested reviewers (which we try to use sparingly since they often know the author personally)

                          • Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)

                          • From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.

                          • Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).

                          • Personal contacts.

                          To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).



                          Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.







                          share|improve this answer












                          share|improve this answer



                          share|improve this answer










                          answered 19 mins ago









                          Allure

                          19.9k1265110




                          19.9k1265110



























                               

                              draft saved


                              draft discarded















































                               


                              draft saved


                              draft discarded














                              StackExchange.ready(
                              function ()
                              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2facademia.stackexchange.com%2fquestions%2f118620%2fhow-do-conferences-journals-select-reviewers%23new-answer', 'question_page');

                              );

                              Post as a guest













































































                              Popular posts from this blog

                              How to check contact read email or not when send email to Individual?

                              Displaying single band from multi-band raster using QGIS

                              How many registers does an x86_64 CPU actually have?