HomeLatest ThreadsGreatest ThreadsForums & GroupsMy SubscriptionsMy Posts
DU Home » Latest Threads » Forums & Groups » Topics » Science » Science (Group) » Empirical analysis tells ...

Wed Jul 1, 2020, 07:31 AM

Empirical analysis tells Reviewer 2: "Go F' Yourself"

Disclaimer: I have no problems with reviewers 2 or 3 . I just thought this was funny..

Got it via RetractionWatch:

https://arstechnica.com/science/2020/06/empirical-analysis-tells-reviewer-2-go-f-yourself/

Peer review is often the key hurdle between obtaining some data and getting it published in the scientific literature. As such, it's often essential to keeping questionable results out of the scientific literature. But for vast numbers of scientists with solid-but-unexciting results, it can be a hurdle that raises frustrations to thermonuclear levels. So it's no surprise that many scientists privately wish that certain reviewers would end up engaged in activities that aren't mentionable in a largely family-friendly publication like Ars.

What was a surprise was to see a peer-reviewed publication make this wish public. Very public. As in entitling the paper "Dear Reviewer 2: Go F Yourself" levels of public.

Naturally, we read the paper and got in touch with its author, Iowa State's David Peterson, to find out the details of the study. The key detail is that the title is somewhat misleading: it's actually the person who is somewhat randomly assigned to the Reviewer 3 slot who's the heartless bastard who keeps trying to torpedo the careers of other academics. For the rest, well, read on.

9 replies, 963 views

Reply to this thread

Back to top Alert abuse

Always highlight: 10 newest replies | Replies posted after I mark a forum
Replies to this discussion thread
Arrow 9 replies Author Time Post
Reply Empirical analysis tells Reviewer 2: "Go F' Yourself" (Original post)
CatLady78 Jul 1 OP
Sancho Jul 1 #1
intrepidity Jul 1 #2
NNadir Jul 1 #3
CatLady78 Jul 2 #4
NNadir Jul 2 #5
CatLady78 Jul 2 #6
CatLady78 Jul 4 #9
Igel Jul 3 #7
CatLady78 Jul 4 #8

Response to CatLady78 (Original post)

Wed Jul 1, 2020, 11:00 AM

1. loved it...

Reply to this post

Back to top Alert abuse Link here Permalink


Response to CatLady78 (Original post)

Wed Jul 1, 2020, 06:01 PM

2. I love that this is the Conclusion:

Conclusions
Reviewer #2 is not the problem. Reviewer #3 is. In fact, he is such a bad actor that he even gets the unwitting Reviewer #2 blamed for his bad behavior.


https://onlinelibrary.wiley.com/doi/full/10.1111/ssqu.12824

Reply to this post

Back to top Alert abuse Link here Permalink


Response to CatLady78 (Original post)

Wed Jul 1, 2020, 06:17 PM

3. I love Retraction Watch.

My default position used to be to believe most of what is published. When some experiments appeared to not be reproducible, I doubted myself.

Growing old is a useful thing. I have to remind my son and his girlfriend of that site.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to NNadir (Reply #3)

Thu Jul 2, 2020, 04:41 AM

4. ...

I can't be too judgemental myself where there is no actual misconduct or malice. Honest errors do occur.
I was a pretty rotten scientist myself when I was younger. When I look back I am often shocked by how clueless I was. Fortunately I had good colleagues throughout (so I could trust them even when I could not trust my own competence). I am still mired copping my own work :-/.

I try to be extra vigilant/honest in the reporting of my own work which is all I can do.

I liked this oped I saw years ago:

https://www.nature.com/news/redefine-misconduct-as-distorted-reporting-1.12411

It really made me think about how one presents one's work whether it is to superiors or colleagues or to the larger science community. It is important to explicitly spell out any lingering unease (about your own assumptions) one might have about a data set to superiors or colleagues and not to put that aside to mull over later till you have a comprehensive picture. Comprehensive pictures can take time.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to CatLady78 (Reply #4)

Thu Jul 2, 2020, 09:11 AM

5. I completely understand what you are saying.

We all have seen and experienced work that is merely sloppy, or poorly performed, incorrectly interpreted, etc,.

Poor record keeping is another frustrating thing. I once knew an organic chemist who when asked to make a new compound, could produce the compound in a way that was analytically of high purity and with spectra that was absolutely consistent with the desired structure. Nevertheless we had to fire him because of behaviors having nothing to do with his scientific skills and everything to do with his personality and attitudes.

I was asked to go through his notebooks to reproduce a particular synthesis. This was very, very, very complex chemistry, very sensitive to conditions, as I knew, since I had synthesized many analogues myself.

His notebook had entries like this:

"Charged x grams of Compound A and y grams of compound B. Stirred 3 hours. Yield 82%"

Well, then...

In industrial chemistry, of course, one often faces the horror of having to reproduce work in a patent (as well as a way to invalidate or get around a patent). Patent literature is purposely designed to be as much like that guy's notebook as is possible and get away with it.

I have a good friend who is a patent examiner. He read a submitted patent's procedure and called his boss in Washington and said, "This chemistry is extremely dangerous. Someone could get killed." His boss said, "That's not your problem; you are simply to determine if it's novel."

Many people, particularly lay people, forget that science is a very human activity, subject to all human frailty. We see these oracular statements in the media sometimes that state, "Scientists say..." as if a revealed and unquestionable truth is about to be announced.

You and I know better.

Understanding error sources is critical; every good scientist knows this, and practically every issue of a good journal has a few errata papers at the end. Most often they're minor, but I've seen some that are serious.

As a person who follows Joan Brennecke's work on ionic liquids (which is nothing to do with the field in which I work), I loved the firm stance she and her co-editors took at the Journal of Chemical and Engineering Data some time back:

New Procedures for Articles Reporting Thermophysical Properties (Brennecke, Goodwin, Mathias and Wu, J. Chem. Eng. Data 2011, 56, 42794279)

My feeling is that every good scientist, like you, looks back at his or her career and thinks about how clueless one was at some point. This of course, only means that you are learning which is a good thing, of course.

My son is finishing his undergraduate degree, and has had the good fortune to have been exposed to research almost from his freshman year. Last year, at his Oak Ridge internship he had the pleasure of making the wrong compound for a neutron diffraction experiment, only to learn that the compound he made by mistake was one they'd been trying to make for years and had long given up on making. His disappointment translated into unintentional success. Good things happen, sometimes randomly, as do bad things. He's very smart, but I keep trying to gently remind him that a truly valuable person is one who is merely educated enough to understand how little he or she actually knows.

I'm an old man, nearing the end of my life. Looking back, I can say, that 95-99% of the time that I thought I had a new or original idea, when I looked into it, I found that what I was thinking to be novel had already been done or had already been shown to be unworkable by experiment. When thinking in areas that have nothing to do with my professional work, where I've had new ideas that don't seem to be under development, if I wait around long enough, usually someone else will think of it and do it. This gives me a sense of peace.

I'm also aware of many times where I mislead myself or was frustrated that things were not working without realizing that this was also good thing. Sometimes things are discovered at the wrong time too. It is unfortunate that nuclear fission was discovered in a time of justifiable war against truly evil people.

Sometimes, one has the pleasure - I think a great deal about nuclear engineering, a field in which I have no formal training - of having a "new idea" only to discover that someone thought of it in the early 1950's and it's been forgotten. That's pretty cool too.

One considers that perhaps Archimedes discovered calculus in Greco-Roman times. Certainly some of his surviving methods suggest this possibility. His world was overwhelmed by the force of ignorance, of course, but sometimes I reflect that perhaps humanity was not mature enough to have safely advanced with a knowledge of calculus given the brutality of his age. In light of the world we are experiencing now, where ignorance is often celebrated, I sometimes question whether we are mature enough, yet.

Overall though, life, particularly a life in science, is beautiful, and then you die. It is thrilling to the imagination to understand how unimportant one is, while simultaneously, as Herman Hesse put it, being that "remarkable point, at which views of the universe, intersect, never to be repeated."

Anyway.

There is a difference in doing good science badly, or bad science well, and in lying. Some errors are legitimate of course, but I like "Retraction Watch" because mostly it focuses on the bad guys and bad gals, the Trumpians in science; there are some. As the world is often found to embrace lies, little things like the "Retraction Watch" website are ever more important.

Thanks for your comment.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to NNadir (Reply #5)

Thu Jul 2, 2020, 09:22 AM

6. I am bookmarking your reply

for a careful read/response.

Thanks for posting...it is obscurely encouraging..
I am currently struggling with a fairly dreary scientific task...which is why I am here so much ;-/.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to NNadir (Reply #5)


Response to CatLady78 (Original post)

Fri Jul 3, 2020, 04:55 PM

7. I've seen people reviewed and act really differently.

The wife-thing (just leave that alone) worked hard and prepared a really good article.

It came back with a really bad review. The reviewer, even to me, was out of his (her) depth.

Wife shelved the article. It was the last straw, instead of rewriting it or revising it appropriately and including a reason why the one reviewer had his (let's say) head far up his butt she fled the field.

She should have known better. We co-wrote an article, it came back with a questionable review, and a month later the editor called to make sure a revision was in the works. It wasn't. He said to revise in accordance with a couple of the reviews, not the mean spirited one, but the editorial board didn't have a stable of reviewers with a lot of depth in the topic at hand.

Saw a student in phonetics stare at a horrible review. His advisor laughed, said to revise appropriately and include a rebuttal saying why he was going to ignore that review. Rob was in horror, challenging a Holy Reviewer. He did as instructed and got an email back from the editor saying he agreed, the one reviewer was pathetic, and saying it was good that Rob was ballsy enough to just say, "No, I'm right."


Saw one review written where the writer said he knew who the writer was--small universe, not a lot of people working on that topic--and that he was trashing the submission to "put him in his place" for other things he'd done. It was a grad student who was being hazed.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Igel (Reply #7)

Reply to this thread