Difference between revisions of "Peer Review"

From P2P Foundation
Jump to navigation Jump to search
m (Peer-review moved to Peer Review: title consistency)
Line 4: Line 4:
 
It should be noted that the process of vetting in peer production, i.e. [[Communal Validation]], based on [[Anti-Credentialism]], is different.
 
It should be noted that the process of vetting in peer production, i.e. [[Communal Validation]], based on [[Anti-Credentialism]], is different.
  
=Difference between [[Communal Validation]] and peer review=
+
 
 +
=Discussion=
 +
 
 +
==Difference between [[Communal Validation]] and peer review==
  
  
Line 14: Line 17:
 
“One of the early precedents of open source intelligence is the process of academic peer review. As academia established a long time ago, in the absence of fixed and absolute authorities, knowledge has to be established through the tentative process of consensus building. At the core of this process is peer review, the practice of peers evaluating each other's work, rather than relying on external judges. The specifics of the reviewing process are variable, depending on the discipline, but the basic principle is universal. Consensus cannot be imposed, it has to be reached. Dissenting voices cannot be silenced, except through the arduous process of social stigmatization. Of course, not all peers are really equal, not all voices carry the same weight. The opinions of those people to whom high reputation has been assigned by their peers carry more weight. Since reputation must be accumulated over time, these authoritative voices tend to come from established members of the group. This gives the practice of peer review an inherently conservative tendency, particularly when access to the peer group is strictly policed, as it is the case in academia, where diplomas and appointments are necessary to enter the elite circle.  The point is that the authority held by some members of the group- which can, at times, distort the consensus-building process - is attributed to them by the group, therefore it cannot be maintained against the will of the other group members."
 
“One of the early precedents of open source intelligence is the process of academic peer review. As academia established a long time ago, in the absence of fixed and absolute authorities, knowledge has to be established through the tentative process of consensus building. At the core of this process is peer review, the practice of peers evaluating each other's work, rather than relying on external judges. The specifics of the reviewing process are variable, depending on the discipline, but the basic principle is universal. Consensus cannot be imposed, it has to be reached. Dissenting voices cannot be silenced, except through the arduous process of social stigmatization. Of course, not all peers are really equal, not all voices carry the same weight. The opinions of those people to whom high reputation has been assigned by their peers carry more weight. Since reputation must be accumulated over time, these authoritative voices tend to come from established members of the group. This gives the practice of peer review an inherently conservative tendency, particularly when access to the peer group is strictly policed, as it is the case in academia, where diplomas and appointments are necessary to enter the elite circle.  The point is that the authority held by some members of the group- which can, at times, distort the consensus-building process - is attributed to them by the group, therefore it cannot be maintained against the will of the other group members."
 
(Felix Stalder in: http://news.openflows.org/article.pl?sid=02/04/23/1518208 )
 
(Felix Stalder in: http://news.openflows.org/article.pl?sid=02/04/23/1518208 )
 +
 +
 +
==Peer Review is not Obsolete==
 +
 +
Ward Cunningham at http://www.re-public.gr/en/?p=141
 +
 +
"Does the proliferation of wikis mark the eventual end of peer review? How is this development changing the nature of scientific communities?
 +
 +
 +
W.C.: '''Wiki does not threaten peer review. Science needs peer review and it will get it. I do not see knowledge produced through wikis as being on the same ground with scientific knowledge. Wiki is best seen as a way of reporting, sharing, coordinating, problem framing and agenda setting.''' A wiki works best where you’re trying to answer a question that you can’t easily pose, where there’s not a natural structure that’s known in advance to what you need to know.
 +
 +
 +
'''Science is based on repeatable experiment. The peer review is a means of assessing the quality of the experiments''', not voting on the preference for a particular result. But we should not forget that what you get as a wiki reader is access to people who had no voice before. The people to whom we are giving voice are aware of what it’s like to write, and ship, a computer program.
 +
 +
 +
If you want to contribute to a scientific journal you should be peer reviewed. Part of peer review is that you’re familiar with all the other literature. And the other literature somehow that has spiralled off into irrelevance. What was being written about programming didn’t match what practicing programmers felt. With wiki, practicing programmers who don’t have time to master the literature and get a column in a journal that’s going to be read have a place where they could say things that are important to them. The wiki provides a different view. In fact you can tell when someone is writing on wiki from their personal experience versus when they are quoting what they last read. "
 +
(http://www.re-public.gr/en/?p=141)
  
  
Line 27: Line 47:
  
 
[[Category:Education]]
 
[[Category:Education]]
 +
 +
[[Category:Science]]
 +
 +
[[Category:Governance]]

Revision as of 08:27, 16 May 2007

In Peer Review, scientific articles are vetted by scientific colleagues.


It should be noted that the process of vetting in peer production, i.e. Communal Validation, based on Anti-Credentialism, is different.


Discussion

Difference between Communal Validation and peer review

Peer production is based on equipotential participation (see Equipotentiality, i.e. the a priori self-selection of participants, and the communal vetting of the quality of their work in the process of production itself. Peer review is based on credentialism, peer production vetting is based on Anti-Credentialism. Peer review is part of an elaborate process of institutional and prior validation of what constitutes valid knowledge; peer production vetting is a posteriory vetting by the community of participants.


A quote on the difference between peer to peer processes and academic peer review:

“One of the early precedents of open source intelligence is the process of academic peer review. As academia established a long time ago, in the absence of fixed and absolute authorities, knowledge has to be established through the tentative process of consensus building. At the core of this process is peer review, the practice of peers evaluating each other's work, rather than relying on external judges. The specifics of the reviewing process are variable, depending on the discipline, but the basic principle is universal. Consensus cannot be imposed, it has to be reached. Dissenting voices cannot be silenced, except through the arduous process of social stigmatization. Of course, not all peers are really equal, not all voices carry the same weight. The opinions of those people to whom high reputation has been assigned by their peers carry more weight. Since reputation must be accumulated over time, these authoritative voices tend to come from established members of the group. This gives the practice of peer review an inherently conservative tendency, particularly when access to the peer group is strictly policed, as it is the case in academia, where diplomas and appointments are necessary to enter the elite circle. The point is that the authority held by some members of the group- which can, at times, distort the consensus-building process - is attributed to them by the group, therefore it cannot be maintained against the will of the other group members." (Felix Stalder in: http://news.openflows.org/article.pl?sid=02/04/23/1518208 )


Peer Review is not Obsolete

Ward Cunningham at http://www.re-public.gr/en/?p=141

"Does the proliferation of wikis mark the eventual end of peer review? How is this development changing the nature of scientific communities?


W.C.: Wiki does not threaten peer review. Science needs peer review and it will get it. I do not see knowledge produced through wikis as being on the same ground with scientific knowledge. Wiki is best seen as a way of reporting, sharing, coordinating, problem framing and agenda setting. A wiki works best where you’re trying to answer a question that you can’t easily pose, where there’s not a natural structure that’s known in advance to what you need to know.


Science is based on repeatable experiment. The peer review is a means of assessing the quality of the experiments, not voting on the preference for a particular result. But we should not forget that what you get as a wiki reader is access to people who had no voice before. The people to whom we are giving voice are aware of what it’s like to write, and ship, a computer program.


If you want to contribute to a scientific journal you should be peer reviewed. Part of peer review is that you’re familiar with all the other literature. And the other literature somehow that has spiralled off into irrelevance. What was being written about programming didn’t match what practicing programmers felt. With wiki, practicing programmers who don’t have time to master the literature and get a column in a journal that’s going to be read have a place where they could say things that are important to them. The wiki provides a different view. In fact you can tell when someone is writing on wiki from their personal experience versus when they are quoting what they last read. " (http://www.re-public.gr/en/?p=141)


More Information

Lessons from the History and Philosophy of Science regarding the Research Assessment Exercise. URL = http://www.ucl.ac.uk/sts/gillies/

This article by Prof. Donald Gillies shows examples of why an excessive reliance on peer review can impede scientific progress, as major advances were in their time rejected by their peers.