科学家对同行科研成果的评价不可靠
文章来源:未知 文章作者:enread 发布时间:2013-10-10 07:49 字体: [ ]  进入论坛
(单词翻译:双击或拖选)
Are scientists any good at judging the importance of the scientific work of others? According to a study published 8 October in the open access journal PLOS Biology (with an accompanying editorial), scientists are unreliable judges of the importance of fellow researchers' published papers. The article's lead author, Professor Adam Eyre-Walker of the University of Sussex, says: "Scientists are probably the best judges of science, but they are pretty bad at it."
 
Prof. Eyre-Walker and Dr Nina Stoletzki studied three methods of assessing published scientific papers, using two sets of peer-reviewed articles. The three assessment1 methods the researchers looked at were: Peer review: subjective2 post-publication peer review where other scientists give their opinion of a published work; Number of citations3: the number of times a paper is referenced as a recognised source of information in another publication; Impact factor: a measure of a journal's importance, determined5 by the average number of times papers in a journal are cited by other scientific papers. 
 
The findings, say the authors, show that scientists are unreliable judges of the importance of a scientific publication: they rarely agree on the importance of a particular paper and are strongly influenced by where the paper is published, over-rating science published in high-profile scientific journals. Furthermore, the authors show that the number of times a paper is subsequently referred to by other scientists bears little relation to the underlying6 merit of the science.
 
As Eyre-Walker puts it: "The three measures of scientific merit considered here are poor; in particular subjective assessments7 are an error-prone, biased8 and expensive method by which to assess merit. While the impact factor may be the most satisfactory of the methods considered, since it is a form of prepublication review, it is likely to be a poor measure of merit, since it depends on subjective assessment."
 
The authors argue that the study's findings could have major implications for any future assessment of scientific output, such as currently being carried out for the UK Government's forthcoming Research Excellence9 Framework (REF). Eyre-Walker adds: "The quality of the assessments generated during the REF is likely to be very poor, and calls into question whether the REF in its current format4 is a suitable method to assess scientific output."
 
PLOS Biology is also publishing an accompanying Editorial by Dr Jonathan Eisen of the University of California, Davis, and Drs Catriona MacCallum and Cameron Neylon from the Advocacy department of the open access organization the Public Library of Science (PLOS).
 
These authors welcome Eyre-Walker and Stoletski's study as being "among the first to provide a quantitative10 assessment of the reliability11 of evaluating research," and encourage scientists and other to read it. They also support their call for openness in research assessment processes. However, they caution that assessment of merit is intrinsically a complex and subjective process, with "merit" itself meaning different things to different people, and point out that Eyre-Walker and Stoletski's study "purposely avoids defining what merit is."
 
Dr Eisen and co-authors also tackle the suggestion that the impact factor is the "least bad" form of assessment, recommending the use of multiple metrics that appraise12 the article rather than the journal ("a suite13 of article level metrics"), an approach that PLOS has been pioneering. Such metrics might include "number of views, researcher bookmarking, social media discussions, mentions in the popular press, or the actual outcomes of the work (e.g. for practice and policy)."


点击收听单词发音收听单词发音  

1 assessment vO7yu     
n.评价;评估;对财产的估价,被估定的金额
参考例句:
  • This is a very perceptive assessment of the situation.这是一个对该情况的极富洞察力的评价。
  • What is your assessment of the situation?你对时局的看法如何?
2 subjective mtOwP     
a.主观(上)的,个人的
参考例句:
  • The way they interpreted their past was highly subjective. 他们解释其过去的方式太主观。
  • A literary critic should not be too subjective in his approach. 文学评论家的看法不应太主观。
3 citations f545579a8900192a0b83b831bee7f711     
n.引用( citation的名词复数 );引证;引文;表扬
参考例句:
  • The apt citations and poetic gems have adorned his speeches. 贴切的引语和珠玑般的诗句为他的演说词增添文采。 来自《现代汉英综合大词典》
  • Some dictionary writers use citations to show what words mean. 有些辞典的编纂者用引文作例证以解释词义。 来自辞典例句
4 format giJxb     
n.设计,版式;[计算机]格式,DOS命令:格式化(磁盘),用于空盘或使用过的磁盘建立新空盘来存储数据;v.使格式化,设计,安排
参考例句:
  • Please format this floppy disc.请将这张软盘格式化。
  • The format of the figure is very tasteful.该图表的格式很雅致。
5 determined duszmP     
adj.坚定的;有决心的
参考例句:
  • I have determined on going to Tibet after graduation.我已决定毕业后去西藏。
  • He determined to view the rooms behind the office.他决定查看一下办公室后面的房间。
6 underlying 5fyz8c     
adj.在下面的,含蓄的,潜在的
参考例句:
  • The underlying theme of the novel is very serious.小说隐含的主题是十分严肃的。
  • This word has its underlying meaning.这个单词有它潜在的含义。
7 assessments 7d0657785d6e5832f8576c61c78262ef     
n.评估( assessment的名词复数 );评价;(应偿付金额的)估定;(为征税对财产所作的)估价
参考例句:
  • He was shrewd in his personal assessments. 他总能对人作出精明的评价。 来自《简明英汉词典》
  • Surveys show about two-thirds use such assessments, while half employ personality tests. 调查表明,约有三分之二的公司采用了这种测评;而一半的公司则采用工作人员个人品质测试。 来自百科语句
8 biased vyGzSn     
a.有偏见的
参考例句:
  • a school biased towards music and art 一所偏重音乐和艺术的学校
  • The Methods: They employed were heavily biased in the gentry's favour. 他们采用的方法严重偏袒中上阶级。
9 excellence ZnhxM     
n.优秀,杰出,(pl.)优点,美德
参考例句:
  • His art has reached a high degree of excellence.他的艺术已达到炉火纯青的地步。
  • My performance is far below excellence.我的表演离优秀还差得远呢。
10 quantitative TCpyg     
adj.数量的,定量的
参考例句:
  • He said it was only a quantitative difference.他说这仅仅是数量上的差别。
  • We need to do some quantitative analysis of the drugs.我们对药物要进行定量分析。
11 reliability QVexf     
n.可靠性,确实性
参考例句:
  • We mustn't presume too much upon the reliability of such sources.我们不应过分指望这类消息来源的可靠性。
  • I can assure you of the reliability of the information.我向你保证这消息可靠。
12 appraise JvLzt     
v.估价,评价,鉴定
参考例句:
  • An expert came to appraise the value of my antiques.一位专家来对我的古玩作了估价。
  • It is very high that people appraise to his thesis.人们对他的论文评价很高。
13 suite MsMwB     
n.一套(家具);套房;随从人员
参考例句:
  • She has a suite of rooms in the hotel.她在那家旅馆有一套房间。
  • That is a nice suite of furniture.那套家具很不错。
TAG标签: work paper scientists
发表评论
请自觉遵守互联网相关的政策法规,严禁发布色情、暴力、反动的言论。
评价:
表情:
验证码:点击我更换图片