Wikipedia talk:Identifying reliable sources (medicine)/Archive 8

Archive 5

Archive 6

Archive 7

Archive 8

Archive 9

Archive 10

→

Archive 15

Acupuncture and TCM

Acupuncture (edit | talk | history | protect | delete | links | watch | logs | views)

Traditional Chinese medicine (edit | talk | history | protect | delete | links | watch | logs | views)

Just a heads-up. Acupuncturists and believers in traditional Chinese medicine have taken to crying WP:MEDRS as a way to shelter their pet statements that acupuncture is better than a placebo? They are trumpeting a source that has been roundly impeached here: [1]. This Vickers et al. source is being spammed throughout Wikipedia as evidence that acupuncture is better than a placebo and that's very, very bad. Please help.

jps (talk) 02:51, 31 October 2013 (UTC)

I could care less about acupuncture and have never used it. I think it's clinical effect is questionable and that it remains possible that its effects are due to placebo, and I think the sources largely support this view. However, the sources clearly do not say that evidence shows conclusively that it is entirely placebo. I do care about is misrepresentation of scientific sources and therefore I draw a bright line which jps is bent on ramming his way through. What initially spurred this discussion was an edit by jps (formerly ScienceApologist, who was banned until recently) at Traditional Chinese Medicine. I criticized his edit a couple hours ago (which came late, after an edit war I wasn't around for) with this:

Dude. This is your edit. You have added "Review articles discussing the effectiveness of acupuncture have concluded that its outcomes are due to the placebo effect" and cite 4 sources, the earliest from 2005 and the latest from 2008. The first, Ernst 2006, ends with "Some findings are encouraging but others suggest that its clinical effects mainly depend on a placebo response". Your summary of his conclusion is not even a half-truth and would likely earn you a very dark scowl from Ernst himself. It might be similar to calling the sky green and not blue, and the colors are not as far apart as red and blue, but it's still objectively untrue. The other reviews are similar; your summary contradicts Cochrane review Furlan et al 2005 perhaps most clearly, as it concludes that "For chronic low-back pain, acupuncture is more effective for pain relief and functional improvement than no treatment or sham treatment immediately after treatment and in the short-term only". It's also sad that you've dragged a couple people who really should know better (IRWolfie- and Alexbrn) into making this flagrantly false edit, the latter of which was a participant of a similar debate at acupuncture not long ago. Misrepresenting scientists who have carefully worded their conclusions is deeply offensive; it's probably most deeply offensive to the researchers themselves but it's also completely unacceptable to our readers and our reputation. As I mentioned when I saw this earlier, it reminds of me an incident that I helped out with. Now, if you want to work on rehabilitation, you should probably self-revert and then take a look at Vickers et al 2012 in JAMA. II | (t - c) 01:45, 31 October 2013 (UTC)

Rather than an apology or understanding, this provoked the following comment from jps:

Oooh... you guys really have gone down hill. Using Vickers? I will working on systematically removing Vickers from this encyclopedia on the basis of this: [2]. Thanks for the heads-up. And please don't POV-push so much!

So he is now on a campaign against the 2012 meta-analysis by Andrew Vickers and his nine colleagues, published in JAMA Internal Medicine (formerly Archives of Internal Medicine). The fact that this is his response to an accusation of misrepresenting scientific sources is most troubling. In my view, it doesn't really matter whether he removes Vickers (and I don't care all that much if Vickers is cited or not), since the conclusions are roughly consistent with the other reviews. But it seems clear that he is not capable of hewing to the nuanced wording of these reviews. II | (t - c) 03:15, 31 October 2013 (UTC)

Josh, if there's a windmill to tilt at, I wouldn't start here, at least not on Wikipedia. I don't know anything about acupuncture or Vickers but the basic parameters of the article (it's a recent MEDLINE-indexed meta-analysis and review published by well-degreed, university-associated authors in a well-respected journal with a high impact factor) would make it very hard to unseat from a Wikipedia article, if it would really be a good idea to try doing so in the first place. Over time our sourcing guideline has become rather prescriptive, and that's probably the only way to make it work when most editors are well-meaning but not very well trained amateur volunteers. There are more outlandish claims supported by much weaker sourcing in other articles... time would be better spend on those. Zad68 03:47, 31 October 2013 (UTC)

Really? The claim that acupuncture is effective has been the rallying cry of the alt med community for a while, and their insistence that researchers receiving funding from their alt med advisory boards pump out credulous papers in out-of-the-way journals is getting in the way of major points that are published in high-quality journals. This is a deplorable state of affairs similar to what was going on at homeopathy a few years ago when Dana Ullman took over and tried to "balance" all the facts about homeopathy being simply inefficacious nonsense with credulous studies from out-of-the-way journals. It's the same story. And we have part of the MEDRS guideline set-up to counteract that: the bit about independent sources. The Vickers article is not independent. The Ernst articles are. So why are we giving them WP:GEVAL? If you guys can't help, this guideline becomes toothless. jps (talk) 12:03, 31 October 2013 (UTC)

The JAMA article appears to meet MEDRS, the blog is definitely not MEDRS and should not be used for anything on Wikipedia, especially removing other sources. Suggest jps is the one with non NPOV, and is heading for a minimum topic ban on acupuncture given the history of previous behavior. Lesion (talk) 10:07, 31 October 2013 (UTC)

The "JAMA" article is not from the actual high-impact JAMA but from an out-of-the-way attempt to compete with trade journals. It is not reliable as a source because it doesn't demand independent confirmation and thus is in contravention of our MEDRS rule for independent sourcing. jps (talk) 12:03, 31 October 2013 (UTC)

It appeared to me that you were basically saying "I'm going to remove this source because of what this blog said" (paraphrase). This is inappropriate. On a related matter, I am interested in your comments on this section: Temporomandibular_joint_disorder#Acupuncture, which I think I wrote and I believe represents the mainstream opinion. Lesion (talk) 12:59, 31 October 2013 (UTC)

The attempt to say, "it's only a blog" is a lot of red herring. Consider the source. Ernst is the pre-eminent (one might argue ONLY expert) in acupuncture efficacy research. His point about this source is that the interpretations being used on it should be, "There is no evidence that acupuncture is better than a placebo". That's not being communicated when this source is being spammed across Wikipedia.

Here's my comment on what you wrote: you are using improperly vetted, poorly controlled, and out-of-the-way studies in poor-quality journals to essentially make Wikipedia recommend the use of acupuncture for this condition. Terrible thing you've done, actually. The issue is that acupuncture has no mechanistic basis and the claimed "placebo" being administered wasn't done properly. Yes, there's a shit-ton of bad papers out there on this subject. Wikipedia should either avoid them completely or give proper context. This was not done in that section. jps (talk) 13:41, 31 October 2013 (UTC)

Respectfully, regardless of who writes this blog, it does not meet MEDRS. Surely you have to agree? It follows that we should not use it as a source, or as justification to remove other sources.

100% disagree. To give a comparable example, if the most famous heart surgeon in the world is quoted as saying, "Please do not read Journal of XXX as it is not a reliable source for research about heart surgery" Wikipedia should take that kind of statement EXTREMELY seriously. jps (talk) 15:20, 31 October 2013 (UTC)

This is not a comparable example. Such would be "the most famous heart surgeon in the world said on his personal blog ...". One the one hand you appear to say that evidence is everything, on the other you say that self published expert opinion trumps sources published in peer review journals. At least quote him from a proper source, not a blog. Lesion (talk) 15:32, 31 October 2013 (UTC)

What you're failing to understand is there is essentially no such thing as a WP:MEDRS quality source that would include the evaluation "Please do not read Journal of XXX as it is not a reliable source for research about heart surgery" because that's not what journals are for. The BEST SOURCE we can have for such an evaluation is a direct quote from an expert. Just because the expert says it on a blog, in an interview, or at a conference does not make the statement unreliable. jps (talk) 15:35, 31 October 2013 (UTC)

Re. the section Temporomandibular_joint_disorder#Acupuncture, I have used one up to date systematic review to form the basis of this section. Following your comments I have searched for other suitable sources, and found 2 other systematic reviews/meta analyses which have similar conclusions (i.e. limited evidence of efficacy) [3],[4]. I also found this source which I believe is what you are talking a bout above Acupuncture may be no more effective than sham acupuncture in treating temporomandibular joint disorders.. I will add these sources to the section, but apart from adding the conclusion of the 4th systematic review, I do not see a need to change the "limited evidence of efficacy" statement of this section. The other additional secondary sources are not cherry picked, and add other opinions for and against the use of acupuncture for TMD. I also point out that the section receives due weight relative to other sections in the treatment section. I therefore disagree with your interpretation "Wikipedia is basically recommending this for this condition", as it is in the context of a long treatment section. Lesion (talk) 14:54, 31 October 2013 (UTC)

But consider the sources you are using. High-quality journals should be availed and the presumption with any medical procedure is one of the null result, which isn't taken seriously by these source. "Limited" efficacy is actually giving too much weight to the studies that are not demonstrating the gold standard of control. So while you can mention that people have tried to study the efficacy using uncontrolled studies, there is no evidence that acupuncture is better than a placebo. That's the kind of writing that I would give a gold star. jps (talk) 15:20, 31 October 2013 (UTC)

I am uncomfortable adding my own interpretation of MEDRS sources. We should follow closely what the sources state. This section has 5 recent systematic reviews now (all meet MEDRS imo), I think it is fine as I have given voice to both sides of the argument. Lesion (talk) 15:32, 31 October 2013 (UTC)

Writing Wikipedia is an act of translation, but you have to be careful when translating the words of fringe theorists. They can only be used, even when published in what look to be otherwise respectable publications, to show what they think. As you have currently written the section, it reads as though their studies are evidence when in fact it is just the authors (acupuncturists all, I might add) who are saying that. This is the very reason we ask for independent sources whenever we do analysis. You are including analysis from primary sources and it should probably be removed or replaced with Ernst. jps (talk) 15:37, 31 October 2013 (UTC)

This section contains no primary sources. How can a systematic review be a primary source? It is by definition a secondary source because it is based on primary sources... Until a Cochrane review comes out on this topic, which hopefully we can more sure is independent, these will have to do. I certainly do not think it is appropriate to remove this section and place a link to some blog instead. You have a strange idea of how the encyclopedia should be. Your recommendation does not represent MEDRS. Please feel free to suggest how you feel the guideline should be changed, but while it stands in its current form, you have no basis to place this blog as a source or use it to replace any content. This is nonsense. Lesion (talk) 15:58, 31 October 2013 (UTC)

Here is my understanding of how you would need to change MEDRS: "Please use only recent, reliable secondary and tertiary sources. Do not use primary sources to refute the findings of mainstream sources found in peer review journals, UNLESS it is the self published material of an expert whose opinion jps agrees with, in which case, ignore any reliable sources because they carry less weight. If you disagree with the conclusions of said sources, feel free to liberally apply your own unsourced interpretation of their inadequacies." Lesion (talk) 16:22, 31 October 2013 (UTC)

Q: "How can a systematic review be a primary source?" A: If it is written by a fringe proponent, it can easily be a vector to promote a parochial or fringe opinion. When a cold fusion advocate writes a "systematic review of cold fusion", they are not writing a secondary source, they are writing a propaganda piece. Likewise with an acupuncturist writing a review of the evidence for acupuncture. You have to consider the source, not just the kind of work. I am not arguing to replace the section with a link to a blog. I'm arguing for using the Ernst reviews (which ARE properly independent) as replacements and to not mischaracterize them. Sorry if that wasn't clear. It would indeed be silly to replace Vickers et al. with blogs that dispute Vickers et al., for example. jps (talk) 17:16, 31 October 2013 (UTC)

You say "acupuncurists all", but as far as I know none of the authors of these numerous reviews are acupuncturists. Andrew Vickers is certainly not an "acupuncturist" - he is a PhD career researcher at the Memorial-Sloan Kettering Cancer Center. I highly doubt he moonlights as an acupuncturist. As I said above, even if Vickers et al is completely removed from the picture, the conclusions of the other reviews are basically similar. And certainly, you can add published criticism of the Vickers review or any other review; although I know there are some people who've regarded letters or commentary articles "rebutting" reviews as automatically "not meeting MEDRS", I think that's an uncommon position. II | (t - c) 18:13, 31 October 2013 (UTC)

Just read the guy's CV. He does not hide his support for societies, groups, and practitioners who use acupuncture and "integrative oncology" as a method. The conclusions being trumpeted in Wikipedia are that "A 2012 meta-analysis found significant differences between true and sham acupuncture, which indicates that acupuncture is more than a placebo when treating chronic pain." and "A 2012 meta-analysis concluded that acupuncture is effective for the treatment of chronic pain." This is in line with the fringe-promotion of Vickers et al. but is simply at odds with Ernst reviews, for example. Trying to lean on "published" reviews is a red herring. This is not how the life-cycle of identifying bad studies goes. Poor studies just suffocate from lack of exposure. jps (talk) 18:40, 31 October 2013 (UTC)

It is not a case of not being clear, at a few points in this thread you appear to clearly state that you think this blog is a suitable source. Now you are reversing your position and claiming you were talking about other sources by the blog author. Per Imperfectly's point above, published criticism (e.g. at least a letter to the editor in a peer review journal) wouldn't be removed by most editors I think, but a personal blog is another matter. Indeed, some of the systematic reviews now in the TMD#acupuncture section appear as "comment on" in the pubmed listing. You could say this is published criticism, although perhaps it is the worst criticism possible since the authors felt the need to start over and perform their own systematic review. Lesion (talk) 18:33, 31 October 2013 (UTC)

You have misunderstood me, I think. Blogs are certainly reliable sources for what the person who wrote them said. That's uncontroversial. When the person writing the blog is an expert, they are excellent sources for a particular topic. Did I ask for personal blogs to be included in the article? No. I just pointed out that a problematic source was being used and the experts who have commented on the subject have explained why the source is problematic. Then suddenly you proposed we remove the problematic source and replace it with the blogs. That's not even close to what I said. Go ahead and quote me where you think I said something to that effect. jps (talk) 18:40, 31 October 2013 (UTC)

Briefly, because this is getting stupid, the comments you make which start with the phrases "The attempt to say", "100% disagree" and "What you're failing to understand" all appear to state that you think the blog source in question is a suitable source, in the context of the respective immediately preceding comments (from me). Agree I must have misunderstood when you said "should probably be removed or replaced with Ernst", an understandable mistake in the context. Lesion (talk) 19:04, 31 October 2013 (UTC)

jps you do not have a leg to stand on, to exclude Vickers as a source per se, as others have said here - your hand-waving at the independent sources paragraph will not do, when (as Lesion writes above) the investigators are with universities, and funding came from the NIH (*yes, some funding also came from a nonprofit dedicated to altmed) and the journal is peer reviewed with an impact factor of 11.46. I would be more interested in seeing what content people are adding based on Vickers. The effect size they found, while statistically relevant, is pretty small (just .5 SD) and if people are blowing this up to say that accupuncture cures chronic pain, rather than "may give clinically relevant relief" then there is a problem... Jytdog (talk) 13:10, 31 October 2013 (UTC)

The last sentence you state gives the problem as essentially that, so it's not very encouraging when you claim I "do not have a leg to stand on". Adopting blind and, frankly, incompetent credulity with respect to the interpretations of Vickers et al. is exactly the kind of nonsense going on in acupuncture and TCM. Two options: remove the source or properly contextualize it. To be clear, I actually don't care which, but the biggest issue here is that the reliable sources who have discussed this work (Ernst, for example) have taken direct issue with the interpretations being leveled in our articles -- namely the unfounded contention there is plainly evidence that acupuncture works better than sham acupuncture. Now, we could go on about the fact that this article has been raked over the coals and criticized in multiple ways by experts on acupuncture evidentiary studies, but this is often beyond the level of analysis necessary for Wikipedia. If you want to help providing context, that's fine, but to claim that this is an acceptable source and to leave it at that is essentially playing into the hands of pseudoscientific charlatans and makes a mockery out of Wikipedia's attempt at neutrality. jps (talk) 13:36, 31 October 2013 (UTC)

Whether a given source is valid, and what content one crafts based on a source, are two entirely different issues. You started this by challenging the validity of the source itself. On that, you have no leg to stand on and you are discrediting any arguments you want to make about content by making it seem that you cannot think clearly and that you do not understand Wikipedia guidelines and policies. I know what it is like to be in the trenches of very contentious issues, and making sloppy arguments only makes things worse for you. By the way, are there publications in MEDRS-compliant sources (please do not bring up the blog again - doing so only further discredits you) in which the Vickers study has been criticized? You do not actually cite any. If they exist and are at least as good as Vickers (which is very good) they can be used to craft "however..." content.14:21, 31 October 2013 (UTC)

No, they are NOT entirely different issues. The "validity" of the source according to WP:GEVAL is pretty close to being "invalid", so I don't see where you are coming from there. I'm not disputing the fact that the source exists, I'm disputing its use (which is the only thing that matters for sources at Wikipedia... we aren't DMOZ). Further, attempts to claim that rejoinders to how Vickers et al. is interpreted should be in "MEDRS compliant sources" which, I assume, is code for, "has anyone peer-reviewed a criticism of Vickers et. al?" is a terrible request. That's 100% NOT how discourse in science works. Negative and null results or criticism letters are not published in peer-reviewed journals. That's not where such critique happens and the currency of scientific discussion happens in the context of how the experts in the subject handle a given source. The reason for this should be clear: journals are subject to the editorial whims of the publisher, the chief editor, the editorial board, and the general philosophy of the publication. They often cannot and do not accommodate such back-and-forth, and in the case of this "JAMA" article (note that it's only under the JAMA header due to a poorly executed Internal medicine resource rescue attempt by the JAMA publishing board), it is clear that the editorial board is predisposed to accommodate acupuncture believers and promoters. The blog, which you snidely attack but is written by an acknowledged EXPERT in this subject, points this out rather plainly. MEDRS is fine for situations where the normal process of scientific discourse is taking place, but in the case of WP:FRINGE claims like those relating to acupuncture, that's not a functional alternative. jps (talk) 14:35, 31 October 2013 (UTC)

First you show know that based on what you have said here so far, I ~think~ I am on the same "side" as you, in terms of reigning in claims that go too far (it is hard to tell because you are all over the place). So please don't bring your battle to me. But you are not thinking clearly. When you write things like "that's 100% not how science works" so emphatically, you are showing that you don't understand how Wikipedia works. You are hurting your own argument. Content needs to be closely based on what reliable sources say and for health-related content, that means MEDRS-compliant sources. And you cannot shoot down a study that is our gold-standard level source under MEDRS with a lesser source. If you could, then all kinds of hell would break loose the other way. All that can be done, is to tone down any content based on this source that goes too far.Jytdog (talk) 14:53, 31 October 2013 (UTC)

Here's the problem with your claim. You have evaluated the source as being "gold-standard level". It's plainly not because the interpretations that the authors included in their text have been roundly criticized by every single independent expert who has evaluated the paper. Now, you seem to think our hands are tied here, but they clearly aren't. This is a primary source and it hasn't been used by others or cited by anyone but by acupuncturists because it is a poor source. That's a WP:REDFLAG and it is our duty to either figure out how to responsibly characterize the claims or remove them entirely. The idea is that Wikipedia asks editors to evaluate sources. Accordingly, this Vickers source is poor not because of the work it cites but because of the interpretations it offers (see below for more on this). However, Wikipedia is using its interpretations to promote claims of efficacious acupuncture. This is not only wrong, it's outrageous. AND, I might add, MEDRS has a very coherent explanation of this very problem. We need to be prepared to apply the axe when it is necessary and it is clear to me that in studies that are poorly executed by acupuncturists and their supporters that claim efficacy are not good studies. Incidentally, don't be confused by the "JAMA" header. It's not the flagship journal: it's a low-impact internal medicine journal that JAMA gobbled up wholesale and left to fester in out-of-the-way places. If this is a "gold standard" source, then you guys need to re-evaluate how you are judging "gold standard", because you're not doing a good job. jps (talk) 15:01, 31 October 2013 (UTC)

I understand Vickers to be our gold standard "meta-analyses of randomized controlled trials (RCTs)". I don't see how you could call it a primary source and not that. I know you don't like its conclusions but I don't understand how you are categorizing the source. Please explain.Jytdog (talk) 15:08, 31 October 2013 (UTC)

"Rules are for the Guidance of the Wise and the Obeisance of Fools" - the intent of the WP:MEDRS guideline is to ensure good sources are being used. Exceptionally, a guideline will guide us wrong. The question is: is it doing that here? (add: I agree though that it is a secondary source). Alexbrn ^{talk|contribs|COI} 15:17, 31 October 2013 (UTC)

See, when primary sources masquerade as secondary sources, we get confused. Vickers et al. is a secondary source only for the previous work they cite. Their interpretations of those works they cite (the interpretation of their meta analyses) are absolutely WP:PRIMARY. This should be obvious to anyone reading the work. jps (talk) 15:25, 31 October 2013 (UTC)

Well I haven't read it. I now will ... (later: well, it looks like a secondary to me; it may be a bad secondary but I don't think that makes it a primary). Alexbrn ^{talk|contribs|COI} 15:37, 31 October 2013 (UTC)

I was afraid that is what you would say, jps. That is not a valid description but is rather a WP:TENDENTIOUS one. I am going to step away from this; it is not a rational conservation.Jytdog (talk) 15:48, 31 October 2013 (UTC)

"Valid description" of what? You were afraid that I would point out that when a source makes a novel or original claim they are acting as a WP:PRIMARY source? Have you read Vickers et al.? jps (talk) 17:18, 31 October 2013 (UTC)

Yes. Every meta-analysis creates a "new" data set from the gathered data, and is therefore able to make new syntheses of that data. That is what every meta-analysis does, and is exactly what our gold standard is. By your definition, there is no such thing as a meta-analysis, and that is just plain crazy. Sorry. I get it that you don't like the result (it appears to be a disaster in your world, and I am sorry about that). You will notice, I hope, that no one here is joining you so far in your idiosyncratic and errant classification of this source... it may be that some will do and I am shown wrong, but to me you seem to be really twisting things to try to knock this source out. I don't believe you will succeed. Jytdog (talk) 17:53, 31 October 2013 (UTC)

There are plenty of meta-analyses, and, in fact, Vickers et al. can be uncontroversially used if we just quote their plain results without their layer of interpretation. It's the interpretation that is the problem and that's what's being paraded around on the "gold standard" platter, perhaps not by you, but certainly in the articles that are currently using Vickers et al. as a source. It's not even that I don't "like" the result. It's that the result that they get does not support their interpretation. That's why the Ernst quote is so useful here. He basically explains that their result is consistent with the exact opposite of what they conclude (in the sense of null hypothesis testing). So, I'm at a loss for what you're claiming here. Perhaps you haven't read through the sources that criticize the paper? Perhaps you think because they aren't on pubmed they don't deserve reading? I'm not sure. But explain to me how one impeaches a source other than doing what Ernst does. He can't exactly publish a paper on the subject of someone's interpretation being cock-eyed. So what more do you want to show that the meta-analysis is being problematically interpreted by the authors? jps (talk) 19:08, 31 October 2013 (UTC)

To be clear, I don't want WP:MEDRS to be toothless. I actually think I'm close to the last in line for wanting to weaken that guideline or its application. I'm on board with making sure that Vickers is represented accurately and appropriately in articles. Zad68 13:57, 31 October 2013 (UTC)

I think that'd be fine, but it's a bit much for Wikipedia. It's been somewhat scandalous that Vickers et al. has included the far-reaching interpretations of the plain results that it did, and we are parroting the author's credulity throughout the encyclopedia. This needs to be at least hedged and I often think removing problematic sources is better than trying to fairly describe complicated back-and-forths. jps (talk) 14:01, 31 October 2013 (UTC)

Not sure if it's been raised yet but this newspaper article gives Ernst's verdict on the Vickers paper: that it "impressively and clearly" showed that the effects of acupuncture were mostly due to placebo. Alexbrn ^{talk|contribs|COI} 14:03, 31 October 2013 (UTC)

Exactly. However, this is not the interpretation of the work that is being trumpeted in our articles on the subject, and, worse, referencing this study is being used to claim that we need to claim that acupuncture is effective in places like the lead of acupuncture or the evidence section of traditional Chinese medicine. I'm appalled that MEDRS is being used by such advocates as a smokescreen, and it's clear from the reaction that I'm getting here that this is a big risk with the way this particular guideline is being monitored by some. jps (talk) 14:28, 31 October 2013 (UTC)

Well, I'll take Ernst's steer on how to interpret an acupuncture paper as being reliable over any random wikipedia editor (even me) any day. It seems to me there is an unusual complicating factor here in that there is an agreed mainstream context in which these studies need to be interpreted (where, for example, the precise nature of the blinding is critical) - and Wikipedia needs to be sophisticated enough to reflect that. Or maybe that's just too sophisticated to be workable here? Alexbrn ^{talk|contribs|COI} 14:37, 31 October 2013 (UTC)

I think you summarizing the confounding factors well. The issue is one of WP:FRINGE proponents pocketing a low-impact journal for their own promotional use and Wikipedia being unable to handle it because, essentially, Wikipedia cannot understand that low-impact journals publish garbage ALL THE FRICKIN' TIME and the community simply ignores that garbage. Here's where it gets even more interesting: it appears Vickers has been able to publish a "peer-reviewed" rejoinder to his detractors [5]. This is plainly indicative that the journal is corrupt and any competent researcher worth his or her salt would plainly just say, "alright, this is just too rabbit-holy -- stick with the high-impact Ernst studies and move on: this controversy can't be meaningfully described without explaining a context in academic publishing that is entirely peripheral to the subject at hand." So I said, "remove the source". But the problem is many editors here say, "the source is fine, can't we contextualize it?" The answer is, sadly, not without becoming a clearinghouse of original research, which, I might add, one could plainly see is the problem with the Vickers et al. paper in the first place (there are no citations to it by independent groups). jps (talk) 14:49, 31 October 2013 (UTC)

Yes, and that's just an example of the meta-problem with editing fringe topics: there are not enough good editors in this area (and sadly one fewer today, it seems) so it feels like an eternal last-ditch defence of Helm's Deep against the forces of endarkenment. For understandable reasons, the level of grip on sourcing issues is generally very low even among some editors with many thousands of edits (I'm still learning).

That said I think with acupuncture, Wikipedia can't get ahead of itself and we need to lag the implications of the science a bit. As Ernst says in that Guardian article, "I fear that, once we manage to eliminate this bias from acupuncture studies, we might find that the effects of acupuncture exclusively are a placebo response". The day is nigh, but not here yet. That is not to say that we can't improve the way the Vickers study is framed here ... Alexbrn ^{talk|contribs|COI} 15:01, 31 October 2013 (UTC)

The way acupuncture efficacy should be characterized is with preference to the null result. Namely, "the extant evidence is consistent with acupuncture's efficacy being due to the placebo effect". That is 100% in line with what Ernst is saying in all of his works. jps (talk) 15:25, 31 October 2013 (UTC)

MEDRS and NPOV aren't broken and don't need to be fixed. We can address whatever problems or disagreements there are with Vickers the same way we deal with anything else. WP should lag the science, and the science is obviously in flux. Maybe sci consensus, or majority opinion is lagging the science bloggers, but it's not our place to get ahead of the debate. (I worry about both Type 1 and Type 2 errors in the research; it's frustrating.) I suspect that JPS isn't serious about deleting the source, but is just proposing something extreme in order to move the goalposts and cause others to cry foul so he can say they're anti-science; it's happened before. JPS, aka ScienceApologist, has an almost unparalleled history of being tendentious and disruptive, and in his reincarnation here again tending in that direction. --Middle 8 (talk) 03:54, 2 November 2013 (UTC)

More: I see a lot of prevarication from JPS about secondary sources really being primary sources, and perfectly fine journals really being bad ones -- all of these topsy-turvy assertions premised on the idea that acupuncture must be fringe, and unambiguously so, just like e.g. homeopathy, so that any mainstream appearance of taking it seriously is subversive. A simpler explanation -- one that that doesn't require us to go down JPS's logical rabbit-hole -- is that acupuncture isn't as fringe as he says it is. Aspects of it are; it is used in fringe ways; the non-fringe aspects of it are co-opted by highly fringe people. But there is also a mainstream effort to study it, a process that is unfolding. And its practice spans the fringe and the mainstream, probably on average sitting in the grey zone, leaning fringe. Surely we're smart enough to be able to discriminate among these different things, and not use what's legitimate about it to elevate the bullshit, or conversely.

JPS's approach is seriously flawed, and ironically is perhaps a more extremely mixed bag than acupuncture. On the one hand, anti-science is corrosive at multiple levels and at its worst threatens civilization as we know it; OTOH, as Zad says above, JPS is tilting at the wrong windmill, insulting our intelligence, and wasting our time. He wouldn't need to contort logic and fact so badly if his case were as strong as he claims. He's trolling and bluffing, and evidently enjoys the attendant gaming and drama. His approach here (let alone historically) is all wrong for Wikipedia: too much heat and not enough light, in fact so much heat that fire tends to break out, generating smoke that actually obscures the light. --Middle 8 (talk) 06:20, 2 November 2013 (UTC)

It's all getting a bit forum-like, maybe time to close this thread? (Meanwhile, on the TCM article I have made a specific edit proposal and apart from a couple of responses it's tumbleweed. Maybe folks could re-convene there?) Alexbrn ^{talk|contribs|COI} 06:33, 2 November 2013 (UTC)

I think there is a clear enough consensus. Agree with Middle 8. One POV-pushing editor creating drama for drama's sake. Lesion (talk) 10:07, 2 November 2013 (UTC)

• Just one comment. JAMA Internal Medicine, formerly Arhcives of Internal medicine is generally regarded of equal scientific quality to JAMA, as perhaps more so, because it is purely a research journal, while JAMA contains other material also. (As with all scientific journals, even of the very highest quality, that does not mean that every individual article ever published in it is reliable. just that in general unless there is reason to think otherwise, articles published in it can be assumed to most likely be reliable.) DGG ( talk ) 04:52, 6 November 2013 (UTC)

Good point. This is indeed a top journal. We shouldn't be judging this source based on something said by an anonymous blogger. If this blogger wants to become part of the dialogue of science, he should attempt to publish his criticism as a letter or commentary in the same journal. Then it could cited in the article. TimidGuy (talk) 11:59, 6 November 2013 (UTC)

Well, as long as we're dealing with rumors of how this particular journal is received and considering the valued opinions of those who have been sanctioned for promoting the quack medical advice of the Maharishi, perhaps we should consider the rumors I now report which is that this journal's editorial board has been sympathetic to so-called "holistic medicine" due to some rather unseemly financial relationships between those studying holistic internal medicine and the corrupt NCCAM system set-up by Congress. In that case, I say dismiss all the articles on alternative medicine published there. And, incidentally, this journal does not publish null results nor accept letters contradicting its credulous papers, but does seem willing to print letters from the authors who have done poor work. Does one dismiss the entire journal? No. But one sees and hears what one sees and hears and one reports on these problems here on Wikipedia talk pages where we seem to give free reign to woo-woo believer and skeptic alike! jps (talk) 13:29, 7 November 2013 (UTC)

Query - ENSSER statement about safety of genetically modified food

Hi, back with some more GM food stuff. An anti-GMO organization called ENSSER recently released a statement claiming that there is no scientific consensus that genetically modified food is as safe as food from conventional sources, here. That link is to their website; you can see that they are a small and fairly young organization, and that their board is environmentalists and anti-GMO activists (e.g. vice chair is from CRIIGEN, Seralini's fundraising organization).

An editor, User:Gaba p, has proposed to include a whole paragraph in the Controversies section of the Genetically modified organism article like this (which he created and then self-reverted to show how he wanted to deploy it). The controversies section is currently based tightly on the lede of the Genetically modified food controversies article, which currently makes no reference to the ENSSER statement. (quick note - the GM suite of articles had been over-run with anti-GMO commentary until about a year and a half ago, when a group of editors cleaned them up so that, for instance, the Genetically modified food article actually describes what GM food is, how it is made, what foods you find in the store are GM, etc; before we did that work there was little/no actual content in that and other articles. We took all the content on controversy and put it in one article, organized it, and left a stub based on the lead of the Controversies article, with a link to main, in each of the others. It was a big job.)

The lead of the Controversies article, and the Controversies section in all the related articles, state the scientific consensus as follows: "There is broad scientific consensus that food on the market derived from GM crops poses no greater risk than conventional food." sources:

1. 2012 American Association for the Advancement of Science Board of Directors' statement describing the scientific consensus on relative safety of GM foods, citing Royal Society, AMA, and NAS, among others
2. a book put out by the EU describing A decade of EU-funded GMO research (2001–2010} which says "The main conclusion to be drawn from the efforts of more than 130 research projects, covering a period of more than 25 years of research, and involving more than 500 independent research groups, is that biotechnology, and in particular GMOs, are not per se more risky than e.g. conventional plant breeding technologies." (p. 16)
3. A review article by Pamela Ronald

The body of the Controversies article cites about 7 more sources.

We recently (August of this year) had an RfC on the statement of the scientific consensus and its sources, which confirmed both. It is here.

In any case, I am here to make three statements and ask for comment:

1. content concerning the safety of eating genetically modified food is an issue about health, and so sourcing is subject to MEDRS. Please comment.
2. This source fails MEDRS and so cannot be used as a source to support health-related content, nor can other sources that simply describe what this source says. The reason, is that this is a SPS, which would be OK if it were published by a "reputable major medical or scientific body", which ENSSER is not, especially compared with AASS, Royal Society, AMA, NAS, etc. Please comment.
3. It goes without saying then, that no content based on this source should be in Wikipedia to support content about health, and it certainly cannot be used to challenge content describing the actual scientific consensus on the relative safety of GM food. Please comment. Thanks! Jytdog (talk) 15:25, 15 November 2013 (UTC)

Activist groups are not reliable to determine scientific consensus. This is akin to global warming skeptic groups saying there is no consensus on global warming. That source appears to fail WP:MEDRS as an objective determination of what is medically/scientifically valid. Yes, MEDRS applies here since it discusses health effects; this seems straightforward to me. It might be useful in a subsection detailing what political activist groups say about the controversy, if there are independent sources that discuss it (which would give it WP:WEIGHT to enter the page). Although not a sourcing issue, it seems odd to me that the proposal changes the parent (GMO) article which should be a summary of the daughter (GMO controversy) article, without showing it deserves weight and therefore a prominent place in the daughter article first. Yobol (talk) 15:32, 15 November 2013 (UTC)

thanks for your quick input! Jytdog (talk) 15:43, 15 November 2013 (UTC)

I request that people interested in discussing this issue please do so at the original thread here to avoid fragmentation. Thanks. Gaba ^(talk) 15:46, 15 November 2013 (UTC)

I request that this be discussed here, so it remains in the history of the MEDRS talk page, where it belongs. We can bring the result back to the relevant pages in the GM suite, as we have done and continue to do with the discussion about the validity of the Entropy article on glyphosate. I anticipate that in the future other editors will seek to generate content based on the ENSSER statement, and discussion of whether MEDRS applies, and whether the statement is a valid source under MEDRS belongs here. Jytdog (talk) 16:02, 15 November 2013 (UTC)

Please don't. The full (lengthy) discussion is over there and that would force us to entirely reprise it here. The discussion belongs where the edit that initiated it was made and that is not here. Jytdog this only serves to fragment the original discussion and I would sincerely appreciate if you could stop creating multiple instances of it. A simple link to an archived discussion can be copy/pasted whenever you feel it is appropriate. Regards. Gaba ^(talk) 16:40, 15 November 2013 (UTC)

Again, discussion about whether MEDRS applies belongs here. This is the place to determine that. Also, I did not seek to open a discussion at the Controversies page, but simply notified editors who watch there of this discussion and our original discussion at the GMO Talk page. I do not anticipate any discussion happening on the Controversies page and would also encourage editors who comment there, to come here to discuss the application of MEDRS and then come to the GMO page to apply the findings. Enough, let's await responses. Jytdog (talk) 16:58, 15 November 2013 (UTC)

An example

Currently we have this sentence "Individual primary sources should not be cited or juxtaposed so as to "debunk" or contradict the conclusions of reliable secondary sources." I suggest that we add an example such as "For example if a review article state that there is not good evidence for or against X we do not add primary research making a claim thereis." Doc James (talk · contribs · email) (if I write on your page reply on mine) 04:08, 24 November 2013 (UTC)

I think the example you give is aligned with the intent of the advice. Blue Rasberry (talk) 12:02, 28 November 2013 (UTC)

I recommend "we should not" instead of "we do not". Axl ¤ [Talk] 12:50, 28 November 2013 (UTC)

• Is this needed? We should be looking for ways to reduce the overall length of the guideline imo... Lesion (talk) 18:25, 28 November 2013 (UTC)

Can we just say "primary sources must not be used to contradict secondary sources" ? Alexbrn ^{talk|contribs|COI} 18:58, 28 November 2013 (UTC)

Some people do not know what "contradict" means. They believe that if a high quality secondary source says the evidence is inconclusive, one can add a primary source which says it works as the secondary source did not say it didn't work. Doc James (talk · contribs · email) (if I write on your page reply on mine) 19:30, 28 November 2013 (UTC)

Suggested reword to MEDDATE section

Propose changing:

"These instructions are appropriate for actively researched areas with many primary sources and several reviews and may need to be relaxed in areas where little progress is being made or few reviews are being published."

To

"This guideline applies mainly to Wikipedia articles concerning topics for which there is ongoing and active research. For articles on topics for which there are fewer available sources, it may be appropriate to include older sources if nothing recent can be found."

Rationale:

• the intro could be interpreted as stating that primary sources are desirable as Wikipedia sources. I am not greatly fussy about the wording, I just feel that it is sending out contradictory messages if we are not very clear. If anyone wants to reword please feel free. Mainly I thought the reference to primary sources needed to go... Lesion (talk) 12:39, 24 October 2013 (UTC)
  • For clarification, this is under the "Use up-to-date evidence" section. - Sidelight12 ^Talk 12:44, 6 November 2013 (UTC)

• Support-- 100% more readable. I support this change. LT910001 (talk) 22:03, 24 October 2013 (UTC)
• Support - I think it's better as well. Seppi333 (talk) 23:29, 24 October 2013 (UTC)
• Oppose I believe language was selected intentionally to mention primary sources to provide a little push against the human tendency to become extremely rigid and rule-based. My impression here is that in the past few years this guideline has been interpreted in an extremely rule-based manner which is fairly hostile to newcomers and even a turnoff to some experienced people (at least myself). People can agree to disagree, but I guess it basically comes down to: is Wikipedia a place which has a whole lot of information, including pretty new stuff, or just information which has attracted a sufficient amount of high-profile interest that there's a review article or a few out there? I come down on the former side. Others can disagree, and that's OK, but I doubt my position will change. II | (t - c) 01:46, 25 October 2013 (UTC)

I would point out that the change of wording does not explicitly say to use primary sources or not to use them. I feel it is better to leave this discussion to the dedicated part of the guideline, (which I suspect is the part of MEDRS that you actually want to reword?). Here we are talking about how to select recent sources, and I feel it is confusing to drag primary sources into it. Agree with you that MEDRS is interpreted rigidly by some. It is just easier to say "don't do that please" to a new user who has placed content based on small, in vitro studies which contradict the consensus with a strongly worded guideline than one which says "primary sources are OK sometimes, and other times they are less OK" and not clearly define when they are suitable ... Agree it is difficult for new users, but I assume most of us went through having our first edits reverted. Also, you have to question whether we want to encourage the kind of editors who want to cherry pick primary sources to back their own non-NPOV to stay. I feel that we need a guideline which is clear to this kind of editor that this is unacceptable. It is a bit of a compromise between the quality of the encyclopedia and a user-friendly environment in some respects. Lesion (talk) 12:37, 25 October 2013 (UTC)

I noticed that it doesn't specify. I think this tends to get more used by experienced editors and suggests a "tone". I'm not going to hold it up if people want it (which it appears they do), and I agree that it doesn't fundamentally change the guideline. Reasonable people can disagree and I see why you want to tweak it. But I still think it's better to specifically note a relaxation of rules. As far as whether we want new editors, I think that anyone who has gotten to the point of figuring out how to navigate the system, and starts out their edits with actual medical literature, should be treated with respect and mentorship rather than accusatory hostility and suspicion. Wikipedia has always been pretty much a motley crew and I don't see that changing. Incidentally, when this guideline was being hashed out, one of the few editors I've encountered who has professed to be an actual researcher (Paul gene, although evidence was of course lacking) was one of its most vociferous opponents, and apparently left the project after losing the debate. My bigger concern with the guideline is that mostly when I see a primary source removed around here, there's a secondary source or several which could just as well support the information. II | (t - c) 23:46, 26 October 2013 (UTC)

Suggest leave this proposal stand for longer to get more opinions. Lesion (talk) 10:00, 31 October 2013 (UTC)

• Support - it's clearer, and since this is a guideline there is still scope for flexibility in truly exceptional circumstances. Alexbrn ^{talk|contribs|COI} 07:35, 25 October 2013 (UTC)
• Oppose - it is clearer, but the meaning changed to where primary sources if there is little research in the topics got dropped out of the original text. - Sidelight12 ^Talk 12:46, 6 November 2013 (UTC)

Sidelight, please could you rephrase your rationale for opposition? Lesion (talk) 12:54, 6 November 2013 (UTC)

The original text was written to relax the rules on older and primary sources on under-reviewed/under-researched topics. Without this, Wikipedia wouldn't have information that readers are inquisitive about. Furthermore, the omission made, pertaining to primary sources, could be wp:creep. - Sidelight12 ^Talk 13:51, 6 November 2013 (UTC)

MEDDATE is not intended to state that it is ok to use primary sources if no other sources are available. If there are no secondary or tertiary sources, the topic should not be in the encylopedia. MEDDATE is not a "loophole" to bypass such fundamental guidelines as notability, and RS. MEDDATE should be purely about the age of sources. Let us leave the discussion on primary and secondary sources to the dedicated part of MEDRS, which this should not be. I don't understand how you think this rewording is CREEP, since it is providing more vague guidance than the previous wording? Doesn't this mean that the old wording was more CREEP? Lesion (talk) 10:51, 7 November 2013 (UTC)

The original wording wasn't creep. The proposed change possibly is creep, because its bumping out something that had a purpose. I'd be funny that people would expect to find information on Wikipedia that is available elsewhere. Notability is not the case for inclusion into articles. "MEDDATE should be purely about the age of sources" - everything in the original wording has to do with date. - Sidelight12 ^Talk 23:21, 9 November 2013 (UTC)

Usually, when something is called WP:CREEP it's running in the direction of micromanagement and rigid rules. Vague guidance is the opposite of micromanagement, and it is appropriate for guidance. II | (t - c)

• Oppose. MEDDATE is not intended to state that it is ok to use primary sources if no other sources are available is not commentary I agree with. Biosthmors (talk) pls notify me (i.e. {{U}}) while signing a reply, thx 07:41, 10 November 2013 (UTC)

So, you are saying MEDDATE is intended to state that it is OK to use primary sources? Lesion (talk) 12:58, 10 November 2013 (UTC)

Of course. Biosthmors (talk) pls notify me (i.e. {{U}}) while signing a reply, thx 14:28, 12 November 2013 (UTC)

MEDDATE is intended to be a loophole in RS and MEDRS? Am I missing something here? Lesion (talk) 14:49, 12 November 2013 (UTC)

Yep. Something big. WP:MEDPRI. Biosthmors (talk) pls notify me (i.e. {{U}}) while signing a reply, thx 09:14, 20 November 2013 (UTC)

To me, MEDDATE should essentially say: "if there are not enough recent sources (e.g. in the last 5 years), it may be appropriate to use some older ones." Instead, we have a long-winded paragraph, making a very simple message unnecessarily complicated. This situation just about sums up any Wikipedia guideline. What is needed is for all the rules and guidelines to be reduced to their core message, and ideally merged into as few pages as possible. The result would be a greater level of understanding among editors and new users would learn the ropes more effectively. Instead what we will get is increasing complexity and increased number of guidelines. Should we be surprised that the number of active new users is dwindling? Lesion (talk) 12:58, 10 November 2013 (UTC)

Yes, I wish something was done about the poor drafting here: there has been a spate recently of editors claiming to have read MEDRS and saying it gives them an opening to use primary sources. Alexbrn ^{talk|contribs|COI} 08:15, 20 November 2013 (UTC)

I'm not persuaded. There are many ideas for improvements out there. I don't see this one as being particularly important in any way. Biosthmors (talk) pls notify me (i.e. {{U}}) while signing a reply, thx 13:49, 20 November 2013 (UTC)

• Support The previous could be interpreted as an invitation for editors to due WP:OR and write about topics with WP:UNDUE weight. Most primary research is so tentative that it should not be used. And most primary research papers way over interpret their results as positive. We need secondary source to keep Wikipedia WP:NPOV. Doc James (talk · contribs · email) (if I write on your page reply on mine) 19:38, 28 November 2013 (UTC)

German Acupuncture Trials

These were clinical trials run in the early 2000s in Germany. There is some debate about sourcing and whether/how their suggestions about the effectiveness of acupuncture should be included. More eyes welcome. (Also posted to WP:FT/N.) Alexbrn ^{talk|contribs|COI} 08:18, 20 November 2013 (UTC)

MEDRS has been undermined. Editors claim they do not have to follow MEDRS. I propose MEDRS should become policy. QuackGuru (talk) 19:10, 28 November 2013 (UTC)

Definitely support that. WP:MEDRS is in line with WP:V and WP:RS. They all say one should use high quality secondary sources. We just specify what these are for medicine. For non medical topics all the other guidance also says secondary sources should be used. Doc James (talk · contribs · email) (if I write on your page reply on mine) 19:33, 28 November 2013 (UTC)

Yup, long overdue. Now who's going to put together the wp:PROPOSAL? LeadSongDog come howl! 20:24, 28 November 2013 (UTC)

Might an easier route be to keep to MEDRS as a guideline, but introduce its core precepts into a new paragraph in WP:RS, which MEDRS than expanded on? Alexbrn ^{talk|contribs|COI} 08:35, 29 November 2013 (UTC)

Question about source credibility/conflict of interest

I am trying to read through Wikipedia's guidelines before I make any edits to make sure I understand them. Does the credibility of a source play into its reliability? Does any conflict of interest a source may have affect its reliability? I noticed these terms aren't mentioned and didn't know if they were implied in the concept of "reliability". — Preceding unsigned comment added by AuburnMagnolia (talk • contribs) 03:03, 7 December 2013 (UTC)

Part of the guideline is to ensure that sources are credible, by discouraging sources of which the credibility might be doubtful (e.g. small primary studies that are hyped up by press releases). A degree of editorial judgement is sometimes required, and ultimately consensus will be needed where doubt exists. JFW | T@lk 22:46, 7 December 2013 (UTC)

So of particular interest, to me, is conflict of interest. Medicine is not a pure science and there are a number of motives that can affect medical research (e.g. financial, reputation, etc.). Does Wikipedia give consideration to these factors? If an author serves to gain financially from an outcome of a study, does that affect the weight that Wikipedia gives to that study? For example, if a study is of a high quality type and published in a reputable journal but there are conflict-of-interest issues with the author, is that study given equal weight to those without COI? AuburnMagnolia (talk) 02:34, 8 December 2013 (UTC)

What you're looking for is the conflict of interest guideline, which applies project-wide.

One of the reasons we discourage primary sources is because selective citation from primary sources makes it easy to (over)emphasise a particular point that suits someone's personal theories or interests. If a source is of impeccable quality, but an editor stands to derive benefit from its inclusion (an unusual scenario with secondary sources) then the editor would do well to declare this conflict of interest on the relevant article's talk page, and allow other editors to weigh in on the appropriateness of including the article as a reference. JFW | T@lk 07:44, 8 December 2013 (UTC)

Thanks for your clarification! AuburnMagnolia (talk) 16:17, 8 December 2013 (UTC)

Concern about MedRS date guidelines and wiping out sources

I have seen recently on several articles editors removing sources and related content based on MedRS because, though the removed citation was reliable, and otherwise met medRS (ie from a peer reviewed medical journal), is 10+ years old. For some areas of medicine and science which get relatively little scholarly attention, I'm concerned that the 'Look for reviews published in the last five years or so, preferably in the last two or three years.' is arbitrary, and if no peer reviewed sources exist for the time range, can lead to the exclusion of valuable content. Dialectric (talk) 23:46, 22 November 2013 (UTC)

As the guidance says: "These instructions are appropriate for actively researched areas with many primary sources and several reviews and may need to be relaxed in areas where little progress is being made or few reviews are being published." Alexbrn ^{talk|contribs|COI} 06:04, 23 November 2013 (UTC)

It's not clear to me that Dialectric understands MEDRS: "(ie from a peer reviewed medical journal), is 10+ years old." And not a primary source, unless used appropriately. Plenty of peer reviewed recent medical journal articles are primary sources. SandyGeorgia (Talk) 15:49, 23 November 2013 (UTC)

I don't understand your point, SandyGeorgia. I didn't mention primary sources in my original statement. Could you clarify? Dialectric (talk) 22:45, 23 November 2013 (UTC)

Which edits are you concerned about? Doc James (talk · contribs · email) (if I write on your page reply on mine) 22:52, 23 November 2013 (UTC)

Dialectric, you said "(ie from a peer reviewed medical journal)" which led me to believe that you (may) think that anything in a peer reviewed medical journal may meet MEDRS. I may be misunderstanding your intent, which is why I started my post with, "It's not clear to me ... " Are you suggesting that anything that is in a peer-reviewed journal meets MEDRS? That's how I read your post. SandyGeorgia (Talk) 00:28, 24 November 2013 (UTC)

RE:SandyGeorgia, no I am not suggesting that, as that is obviously wrong. For clarity, I am concerned with source removal where the sole justification provided for removal is MEDDATE, and prior discussion has not taken place.

Re:Jmh649, rather than call out specific edits / editors, I hoped to raise a general issue I have with Med:RS. Though the wording Alexbrn pointed out, and I missed, isn't ideal, ('may need to be relaxed' invites argument over every such inclusion), it does address my issue.Dialectric (talk) 16:11, 24 November 2013 (UTC)

Old wrong stuff should be removed. Whether or not it is out of date depends partly on what the material is. Doc James (talk · contribs · email) (if I write on your page reply on mine) 19:28, 28 November 2013 (UTC)

Of course old, wrong information should be removed. The issue is one of burden. That is, the current wording allows anyone to remove any Med citation because it is old, without having to provide evidence of, and a ref for, newer reviews. Dialectric (talk) 22:32, 28 November 2013 (UTC)

Our policies allow anyone to remove anything regarding of how it is supported Doc James (talk · contribs · email) (if I write on your page reply on mine) 22:34, 28 November 2013 (UTC)

In my opinion, old references/information should not be deleted just because they are old. If they are inaccurate/incorrect, then of course they should be removed—but the age of the material is irrelevant. If the references are still accurate but newer (otherwise equally reliable) sources are available, then they should be replaced with the new references. If there are no new references available, the old material should be retained. Axl ¤ [Talk] 22:44, 28 November 2013 (UTC)

Agree Doc James (talk · contribs · email) (if I write on your page reply on mine) 22:46, 28 November 2013 (UTC)

Also agreed. There's also a lack of understanding that sometimes reviews are just classic and they are highly cited for a particular topic or issue, and shouldn't be just replaced by the first 2013 review to come out. For example, when I was reviewing back pain for good article status, there was resistance from User:Zad68 to adding the 2007 American College of Physicians (ACP) current back pain guideline review article - even though this is basically a landmark publication and replaced a previous guideline from 1994 or thereabouts. These types of articles have a significant influence on US physicians. Also I do think we need to hear names. There are people who are responsible for things like this and they need to be told if people don't approve. I mention Zad68 because from my experience he was one of the people with this very rigid view, altho I don't know if it continues - would like to hear a comment. II | (t - c) 05:01, 7 December 2013 (UTC)

These are difficult judgement calls. In an active area of research I would say a 2007 review is too old. But back pain is not a very active area of research so I would not see a concern with a 2007 review. But without some expertise in the subject area at hand these are difficult calls to make (is it an active area of research or not). I also reviewed the article in question and did not see any striking differences between the 2007 and 2013 reviews. Doc James (talk · contribs · email) (if I write on your page reply on mine) 19:19, 8 December 2013 (UTC)

Responding to the ping from II here, I don't think what I said is being represented accurately. We were talking about low back pain and mattresses on Talk:Low back pain. In the particular instance of the use of the 2007 ACP guideline, my primary issue wasn't the use of the ACP 2007 Guideline per se but rather that we had a newer secondary source that's a global review of back pain guidelines produced by medical organizations worldwide. Here's the relevant bit of what I said:

And I saw you added a bit to the article sources to Chou 2007 (PMID 17909209). The addition of that content sourced to that source is not something I really agree with, honestly, as it's a bit old per WP:MEDDATE, and more importantly that's a guideline from only one medical organization. What's critical here is that we have a 2010 review of guidelines from many medical organizations worldwide (Koes 2010, PMID 20602122), and that source, surprisingly enough, does not mention mattresses at all either. I think it is a WP:WEIGHT problem to mention something from the 2007 guideline of only one medical organization when we have a 2010 review of guidelines worldwide that does not.

So it wasn't as much of a WP:MEDDATE concern as much as it was a WP:WEIGHT concern. Hope that clears that up... Zad68 03:18, 10 December 2013 (UTC)

Adding: ...and I ended up not arguing against the use of the ACP 2007 Guideline strongly enough to keep the material out of the article; Low back pain ended up with the mattress material in it. Zad68 03:22, 10 December 2013 (UTC)

Zad68: First off, I'm sorry if I misrepresented you somewhat - I was thinking of the comment as it's a bit old per WP:MEDDATE. And since you didn't directly address the topic at hand I'll assume you have a similar view that things shouldn't be trimmed out just purely due to age. I also noticed that it doesn't seem that anyone mentioned the language "do not automatically give more weight to the review that happens to have been published most recently, as this is recentism" in the policy, which may address Dialectric's concerns. Glancing at your contribs, you do cite MEDDATE sometimes in your edits e.g. [6]. Not saying it is wrong, but it can seem perfunctory. And I don't want to be discouraging even if I'm singling you out because you're the only one who came to mind, and I do recognize that you do a lot of great work around here, much more than me. However, with regard to the back pain situation, I got resistance for adding a guideline which was cited by AMA News as producing "what are considered by many to be the first comprehensive [United States] national guidelines since 1994 on the diagnosis and treatment of acute low back pain" in "a process took more than two years". Yes, I was using it for a specific, somewhat underreported purposes. However, if I get resistance for that and have to debate it, it concerns me a little bit that the "weight" inclusion standards are just too restrictive and people are excessively discouraged. It's also somewhat discouraging to think that if I add something, it'll probably be gone in a couple years due to aggressive updating. Further, this aggressive standard of inclusion can result in an extremely 'safe' article which is broad and vague more like an undergrad textbook than the surprising detail and intrigue that I expect on Wikipedia. Personally I'm not sure why orthotics (something which someone added after finding a "systematic review" on it which found they are *surprise* ineffective) would be considered more relevant (or "weighty") to back pain than mattresses, but it's not something I was going to bother arguing about and besides a tidbit about orthotics is one of those surprising little details. As sort of an aside, I have in the past specifically cited old articles after noticing it cited in a modern textbook or review as the best source. This is more common with the more pure scientific side rather than the clinical side. It's slightly disappointing but not really surprising to me to think that this would up being peremptorily swapped out for something which is "newer" even if it is simply a cursory look mainly citing old stuff.II | (t - c) 08:08, 13 December 2013 (UTC)

II, regarding "things shouldn't be trimmed out just purely due to age." - It think it depends on the subject and how much is being published regarding it. I actually do think it is appropriate in well-studied subject to age out things that no longer receive attention from the reliable sources being published on it. This is a weight issue combined with the fact that the scientific consensus of thinking on a topic can move away from one position to another position over time, although nobody may have published something specifically stating "Although Theory X used to be popular, it isn't accepted any more." If there is a well-studied subject, and there are reviews from 10-20 years ago stating that there's support for Theory X, but up-to-date reviews don't mention Theory X at all, the article should not mention Theory X. For subjects that are not so well studied, articles should mention Theory X but mention the age of the evidence, "Several reviews from 1989 state that..." I think correct application of WP:WEIGHT is very important, but unfortunately generally not done very well. There is a tremendous danger especially in WP:MED-scope content of presenting outdated minor or fringe theories or aspects of subjects as if they were equally valid or important as the topics that are actually treated uniformly across the sources as important. This may be just a difference in perspective we have between you and me: I think we should be writing articles that, if done well, will not be changing significantly over time, and might be able to appear as a chapter in college-level intro textbook. If someone were to say that about an article I made significant contributions to, I would take that as a huge compliment! For a well-studied, established topic, a GA or FA article should look a year from now largely like it looks now, unless some tremendous breakthrough happens... and even then I'd prefer the article wait until the hype dies down so that proper context can be given. Zad68 17:02, 13 December 2013 (UTC)

Another try at proposal for addition for identifying possible poor journals

An earlier proposal for addition to MEDRS was archived without conclusion here. I am resurrecting this proposal, which has only been more relevant given recent published literature. Proposed addition:

An integral part of finding high quality sources is avoiding articles from journals without a reputation for fact-checking and accuracy. A red flag that a journal article is probably not reliable for health claims includes publication by a publisher that has a reputation for exhibiting "predatory" behavior, which include questionable business practices and/or peer-review processes that raises concerns about the reliability of their journal articles. (See "Notes" section below for examples of such publishers.) Other indications that a journal article may not reliable is its publication in a journal that is not indexed in the bibliographic database MEDLINE, or the publication of an article whose content is outside the normal scope of that journal (for instance, a psychiatric nursing journal publishing an article on the treatment of cancer). Determining the reliability of any individual journal article should take into account whether the article has garnered significant positive citations in sources of undisputed reliability, suggesting wider acceptance in the medical literature despite any red flags suggested here.

Notes

I think something like this is a necessary addition, and hope to get some useful feedback on how to improve it or whether to go ahead and add it to the guideline. Yobol (talk) 19:14, 9 December 2013 (UTC)

Support. I agree that we should try to develop a process to minimise citations to journals that fail standards of basic editorial practice. The drive to publish means that journals set up shop to accomodate this. In the UK, selection for medical training posts now includes the provision that publications will only be taken into consideration if they have appeared in PubMed indexed journals. Can we agree on similar parameters? JFW | T@lk 22:10, 9 December 2013 (UTC)

Journals not indexed in MEDLINE are red flagged in my proposal above (PubMed indexing is in my opinion too liberal as it includes numerous open access journals through PubMed Central that are not MEDLINE indexed). Yobol (talk) 22:39, 9 December 2013 (UTC)

Support. There are actually some dubious open access journals, such as Hindawi and Bentham Science journals, that are indexed on Medline; so Yobol is right about how PubMed indexing is rather liberal. But there is no question non indexed journals, like JPANDS, fail MEDRS rather spectacularly. Jinkinson talk to me What did he do now? 17:07, 10 December 2013 (UTC)

Support a common sense proposal that will help increase the quality of sourcing for our articles. Zad68 17:31, 10 December 2013 (UTC)

Comment This addition generally seems important, but it creates a contradiction. It says that sources should have "garnered significant positive citations...." It could take a couple years for a research review to achieve that. And in practice it would disallow using any current research reviews. This contradicts the MEDRS advice to use up-to-date evidence. I suggest we change the wording in the last sentence from "should" to "may also." I'm going to make that change; feel free to revert. TimidGuy (talk) 11:35, 11 December 2013 (UTC)

Agree with the change. For older sources, this is an acceptable demand. For recent reviews in core journals, I don't think this should be mandatory and indeed may restrict the use of up-to-date sources. JFW | T@lk 13:01, 11 December 2013 (UTC)

Agree, this is an improvement. Yobol (talk) 16:10, 11 December 2013 (UTC)

Comments. Note that in rare circumstances, it may be appropriate for us to draw citations from journals that are not MEDLINE indexed simply due to their newness. (From what I gather, MEDLINE won't index anything that hasn't been in print for at least 12 months, plus several months for them to get around to reviewing the journal and indexing it.) In most cases, such new journals are going to be low-impact specialty ones, but every so often we'll get a new 'heavy hitter' from a respected editorial board and publisher, something that's going to be high-caliber right out of the gate. Think PLOS Medicine or Nature Methods, to pull a couple examples from the last ten years. These examples won't come along very often, but we should be aware of them.

For reference: The difference between MEDLINE, PubMed, and PMC. TenOfAllTrades(talk) 16:36, 11 December 2013 (UTC)

• Support I support this language being inserted, I acknowledge the language proposed as good enough, but I feel that this is too complicated for me to endorse entirely. One concern that I have is that the predatory open access publishing article cited is a critique of open access journals. While the phenomenon of predatory publishing is confined to open access publications, there are lots of other reasons why other journals may not meet WP:MEDRS, and I do not want to single out open access journals as the biggest or only problem when there is not evidence that this is so. I have looked into this issue and while I find a lot of contemporary critique on open access publications, I am unable to find critiques of systematic bias in traditional publication even though I know that people make such critiques. I am happy to support Yobol's proposal but I would be more happy if someone had good ideas for developing it further after it were established. Blue Rasberry (talk) 17:20, 11 December 2013 (UTC)
• Comment weak Oppose I don't buy it. This is relying on one author, who may have a conflict of interest. It is discriminatory against journals where there is no proof against their quality compared to other journals. - Sidelight12 ^Talk 19:19, 11 December 2013 (UTC)

  How is this really any different from our standards for non-academic magazines? We also avoid magazines whose reputation for fact-checking and editorial independence is also unknown. WhatamIdoing (talk) 19:53, 11 December 2013 (UTC)

  The first sentence can't be argued with. Calling out journals as predatory is going too far. I'd think most who submit their publications as open access want to contribute or want their contributions to be known. There is nothing wrong with open access in itself, as the author criticizes. Also problematic is, "or the publication of an article whose content is outside the normal scope of that journal (for instance, a psychiatric nursing journal publishing an article on the treatment of cancer), " as there's nothing wrong with that. There are way better and recent examples of journals dropping the ball than the claims made by the author. The claims by the author are there to target open access, are not reliable, and look to have a conflict of interest. - Sidelight12 ^Talk 18:28, 15 December 2013 (UTC)

  I just realized a case where a journal covering a subject outside of its scope could be a problem. That, if this subject is a potential competitor to the specialty of the selected journal. Other than that obvious conflict of interest, there's nothing else wrong with a journal covering a subject outside of its normal scope. We should be careful to not single out open access articles. Also, we shouldn't rely on Beall's two publications, but instead rely on different sources/reasoning for the decision. My worry was of useful articles that couldn't be used to expand knowledge. - Sidelight12 ^Talk 01:53, 17 December 2013 (UTC)

  The problem with "out of scope" articles is that the journal's editors don't know anything about the subject and therefore can't do a proper review of the content. They end up approving "incredible" papers that people who are familiar with the field instantly find serious flaws in. There have been some spectacular failures along those lines.
  Now, the example needs to be considered carefully. A psychiatric journal can certainly publish about the psychiatric issues associated with cancer, especially post-chemotherapy cognitive impairment and lots of things associated with brain cancer. But they shouldn't be publishing papers on, say, the hot new treatment for skin cancer. Accepting that would be akin to using a political magazine to talk about trends in children's clothing, or a fashion magazine to talk about civil engineering. WhatamIdoing (talk) 00:11, 18 December 2013 (UTC)

• Support I share Bluerasberry's concerns about excessively targeting open-access journals which are painted with a broad brush, so I hope that doesn't happen. Some decent open-access journals actually disclose the peer review (e.g., BMC's Nutrition Journal which isn't really great otherwise) versus traditional journals being more secretive. I very much support efforts to raise the bar in scientific publishing. II | (t - c) 08:24, 13 December 2013 (UTC)
• Support There has been enough published discussion c.f. to inform a reasonable policy on this, we are certainly not relying on a single author as @Sidelight12: suggests. Of particular note are PMC 3763246, doi:10.1186/1741-7015-10-73, PMC 3804522, and Šipka P. Legitimacy of citations in predatory publishing: The case of proliferation of papers by Serbian authors in two Bosnian WoS indexed journals. CEES Occasional Paper Series. 2012. (June 15th, 2013). pp.25–26. To be sure it is easy to criticize the pay-to-publish model, but the reality is that the problem of journal quality is not that simply determined by OA practices - some journals will game the citation system to boost their ISI ratings. Some will cut corners on the review process to gain publication fees. Some new journals take a few years before getting the wrinkles out, while others ride on an established reputation while using weakened editorial practices. Some conventional journals will even have the nerve to charge readers for a retraction notice for their own errors! Our best bet is to rely first and foremost on journals that have been considered by independent assessors such as PubMed to be of quality for several years running. Even then, we should ask ourselves whether an assertion in the review paper we want to use is itself citing primary papers from related or independent publishers. JCR does not do all our work for us.LeadSongDog come howl! 14:20, 13 December 2013 (UTC)
• Advisory I just notified Wikipedia_talk:WikiProject_Academic_Journals#Notice_of_related_discussion that this discussion was ongoing. Anywhere else that should know? LeadSongDog come howl! 19:46, 13 December 2013 (UTC)

That seems diligent. I added a notice to WT:OPEN. Blue Rasberry (talk) 20:07, 13 December 2013 (UTC)

Best before dates on primary cites?

At present we have "If no review on the subject is published in a reasonable amount of time, then the text associated with the primary source should be removed." As this often gets overlooked, I 'd suggest we need some way of flagging a best-before date on primary source citations. Surely a little template magic can cure this... LeadSongDog come howl! 14:39, 9 January 2014 (UTC)

Only very few primary sources should be used. That should keep this issue to a minimum. I did use one for therapeutic hypothermia in cardiac arrest though. Doc James (talk · contribs · email) (if I write on your page reply on mine) 00:34, 28 January 2014 (UTC)

Proposal to add language to "Respect secondary sources" section

I'm proposing we add the following:

Another reason to avoid primary sources in the biomedical field - especially papers reporting results of in vitro experiments - is that they are often not replicable and are therefore unsuitable for use in generating encyclopedic, reliable content about health. A paper published by drug discovery scientists at Bayer in 2011 reported that Bayer scientists were able to replicate results in only ~20–25% of the prominent papers they tried to reproduce; scientists from Amgen followed with a publication in 2012 showing that they were only able to replicate 6 (11%) of 53 high-impact publications and calling for higher standards in scientific publishing. The journal Nature announced in April 2013 that in response to these and other articles showing a widespread problem with reproducibility, it was taking measures to raise its standards.

What do you think? Jytdog (talk) 03:48, 29 December 2013 (UTC)

This seems to be a good change to me. The revision begs the question and makes me wonder if this should be stated explicitly: Do studies by professional organizations and for-profit companies deserve less weight than purely academic or scholarly studies? It is not unheard of for studies to cut corners, choose inclusion criteria to produce the desired outcome, or outright lie in order to achieve a predestined conclusion---especially when profit is involved. AuburnMagnolia (talk) 21:23, 31 December 2013 (UTC)

Support Jytdog's suggestion, well-supported as proposed. Zad68 21:27, 31 December 2013 (UTC)

Thanks Zad! Actually in my experience, basic science at biotech companies is perhaps more rigorous than in academia - that is kind of the point of the studies by Bayer and Amgen that I cited - the papers they could not replicate were generally from academics. (Scientists at companies don't have room to be as creative as academic scientists, for sure, but I think their context makes them more rigorous) We recently had a guy join our faculty who was head of R&D for a pharma company and I have talked to post-docs who work for him who are just gob-smacked at the level of rigor he requires (double checking reagents, measurements, N of 10, etc). He won't even look at data with an N of 3. And this is generally what I have found, with regard to the basic science. Companies care about reliability of results very much, since they pour millions of dollars into stuff that works and want to stop putting money in, or change course, if things aren't working. When it comes to publishing (especially MD-level stuff like clinical trials), those papers get put under other lenses (e.g. marketing) before going out the door and so it does make sense to me to look at such papers with a grain of salt. Basic science, not so much. Jytdog (talk) 22:02, 31 December 2013 (UTC)

That's great to hear! My comment was referring to secondary studies such as meta-analysis, for example. It seems common practice to include only 10% of the studies on a particular subject in a meta-analysis. The inclusion criteria is up to the researcher. Wikipedia doesn't allow editors to omit studies due to inclusion criteria. I've seen studies in which the inclusion criteria is carefully crafted to obtain a specific result. If a professional organization, for example, has a vested interest in a specific outcome, shouldn't we be able to give less weight to a meta-analysis it produces? Consider the COI? AuburnMagnolia (talk) 14:20, 2 January 2014 (UTC)

Support. I've recently needed to refer to this phenomenon. This is a powerful argument for sticking to secondary sources. JFW | T@lk 23:18, 31 December 2013 (UTC)

• Support I worry about the length of the sections but since this issue is so much discussed, I think it hurts nothing to bolster the section which especially good content like this and if the section ever gets too long, it would harm nothing to even fork this section into its own article. I appreciate what is being proposed and only good can come from adding it, even though it is becoming longer and more discussed than many other policy sections. Blue Rasberry (talk) 17:17, 2 January 2014 (UTC)
• Support A good point well made. Alexbrn ^{talk|contribs|COI} 17:26, 2 January 2014 (UTC)
• Support. Yes, it looks fine. Axl ¤ [Talk] 12:05, 4 January 2014 (UTC)
• Support. Yes this is excellent and just the information many want regarding our strong recommendation to use secondary sources. Doc James (talk · contribs · email) (if I write on your page reply on mine) 00:33, 28 January 2014 (UTC)

Thanks all. It has been a month, and since there is no opposition I went ahead and added it. Jytdog (talk) 11:32, 28 January 2014 (UTC)

• Comment - I disagree with how the first sentence is written. It should be "A reason to avoid primary sources in the biomedical field with papers reporting results of in vitro experiments is that they are often not replicable." saying 'another' conflicts with the rest of the section before it, and this needs to specifically be about in vitro considering the reasoning is about in vitro. - Sidelight12 ^Talk 18:22, 28 January 2014 (UTC)

Thanks for your comment! Can you please say why you understand that this caveat doesn't apply to animal studies? ThanksJytdog (talk) 18:28, 28 January 2014 (UTC)

I have no dispute against that, as this can be extended to apply to animal studies. It should be reworded for clarity, and we need to add the connection of why it applies to animal studies. However, for human studies, I'd like to see an addition that permits primary studies that show concrete evidence. - Sidelight12 ^Talk 18:59, 28 January 2014 (UTC)

This needs to be more widely applied

I am somewhat dismayed that many psychology articles seem to ignore this guideline altogether and even benefits to physical health are claimed based on single studies. I am particularly offended by Optimism and Health. Yes, there is a decent systematic review in Psychological Bulletin for cardiovascular health. This review is discussed in NHS Choices. However, the relationship between optimism and cancer is purely speculative, as even Seligman admits in Flourish, and many people are complaining about the irrational optimism of cancer patients, such as the opinion piece in the NY Times entitled "The Cancer of Optimism". In light of this, it is rather reckless to make the claims we make about cancer based on only three primary sources.

The point I want to make here is that social psychology has had its own share of failed replications, debunked papers, outright fabrications, and sensationalist gimmickry, so why is this guideline limited to biomedical articles? No conclusions about mental health should be made based on single studies either! Vesal (talk) 18:32, 27 January 2014 (UTC)

If you're worried about optimism and cancer, perhaps WhatamIdoing might be able to help! JFW | T@lk 20:09, 27 January 2014 (UTC)

I have cheerfully checked the two primary studies and one spammy website, and none of them support the claim that being optimistic prevents cancer.

The study about head and neck cancer could be used (but shouldn't) to claim that after you've already got cancer, optimistic people are more likely to survive longer. But it shouldn't, because it's a decade out of date, has only 101 patients, and most importantly, because it's not "optimism" itself that is believed to cause survival, but the fact that people who are sicker (and more tired) are automatically less optimistic than healthier people. Optimism is a marker, not a cause. WhatamIdoing (talk) 00:18, 28 January 2014 (UTC)

Thank you! This certainly raised the ratio of MEDRS-compliant sources to primary sources in that article, although it is still firmly below the critical Losada line. However, my question was more generally about this guideline, wouldn't it make sense to expand its scope to explicitly include statements about mental health in addition to biomedical information? Of course, I am not worried about claims that writing a happiness journal makes a person happier, but when people start claiming that such things can improve or prevent clinical conditions, and these kinds of claims are appearing uncritically in Wikipedia articles based on single studies, I believe this guideline needs to be advertised more widely. Vesal (talk) 15:32, 28 January 2014 (UTC)

It's my understanding that the authors of this guide, and the community that applies it, interprets it as covering everything health-related, and in my view, mental health is definitely part of that. I don't know how to advertise anything in Wikipedia; in my mind the best way to get MEDRS known more, is to explicitly cite it when you edit relevant articles and be clear about deploying it in disputes. Jytdog (talk) 16:04, 28 January 2014 (UTC)

Ditto that. SandyGeorgia (Talk) 16:06, 28 January 2014 (UTC)

We specifically called out "alternative medicine" in the lead a couple of years ago, and since then, then "You mean it applies to my herb o' the week?!" comments have dropped nearly to zero. We could add "mental health" on the same line, if you wanted. WP:Nobody reads the directions in the short-term, but eventually it might help a little bit. WhatamIdoing (talk) 20:06, 28 January 2014 (UTC)

It's not like Wikipedia:WikiProject_Psychology/About is at odds with MEDRS, but it could be more directly linked. Also, comparing template:Wikiproject Psychology to template:WPMED, there's a rather obvious difference... LeadSongDog come howl! 03:44, 29 January 2014 (UTC)

Would anyone be opposed to...

The following discussion is closed. Please do not modify it. Subsequent comments should be made in a new section. A summary of the conclusions reached follows.

No consensus to add the ref

---

me appending this ref to the following sentence?

However, in vitro and animal-model findings do not translate consistently into clinical effects in human beings.

Added: the ref was used in animal testing - I needed to find something that said WP:MEDANIMAL in a nutshell.

Regards, Seppi333 (Insert 2¢) 05:55, 4 January 2014 (UTC)

It looks reasonable. Axl ¤ [Talk] 12:03, 4 January 2014 (UTC)

References

User:Seppi333 went ahead and added to the guideline in this dif and i just reverted in this dif, with edit note " please wait for consensus to build". To say more now, while the text in the guideline is important and true, this ref seems ax-grindy to me. Jytdog (talk) 18:38, 4 January 2014 (UTC)

"this ref seems ax-grindy to me." I understand your point. I was initially suspicious about that as well. However there doesn't seem to be any obvious conflict of interest. Also, the BMJ is a respected peer-reviewed journal. Axl ¤ [Talk] 20:00, 4 January 2014 (UTC)

Great source. I'm not sure if we need to copy its conclusions verbatim. JFW | T@lk 22:37, 4 January 2014 (UTC)

what I mean by "ax-grindy" is the title and the way the article is used by anti-animal research people (probably because of the title), and for the selection of topics in the article (e.g. stroke drugs) which are for notoriously difficult incdications - failure in Phase III trials in those indications can be used to indict the entire science and art of drug discovery and development - there is no reason to use them to pick on the animal testing part except to grind axes. Jytdog (talk) 00:45, 5 January 2014 (UTC)

I'm not sure that this is the mainstream view. There are, for example, some strong complaints that there are too few "disparate animals and strains" being used for research. Black 6 is used for just about everything, despite being a pretty odd mouse (the only one that drinks alcohol voluntarily, for example). Eleven out of twelve pain studies are performed in rodents. There's a reason that researchers joke, "If you were a mouse, you'd be cured already." WhatamIdoing (talk) 00:04, 28 January 2014 (UTC)

User:WhatamIdoing, I think when you say "this" (as in "not sure this is the mainstream view") you mean the proposed reference, and not my comment, correct? Jytdog (talk) 16:06, 28 January 2014 (UTC)

Probably so. It makes sense. Is this covered in the recent edit? - Sidelight12 ^Talk 18:35, 28 January 2014 (UTC)

Yes, I mean that the proposed footnote may not represent the mainstream view among experts on animal testing, and specifically the first item in the list, "Disparate animal species and strains" is debatable. Certainly it can be complicated to compare rat vs dog vs rabbit studies (especially when trying to assert something about humans), but there are good reasons to believe that there are too few species involved in testing. You are far more likely to encounter a situation in which you are comparing Black 6 against Black 6 against Black 6, and then discovering that Black 6's response has no correlation with the human response. WhatamIdoing (talk) 20:01, 28 January 2014 (UTC)

I don't think it's debatable at all. For certain drugs, it's obviously not the case. For others, it's so relevant as to render animal testing laughably useless (the following was my motivation for creating WP:MEDANIMAL and adding this reference). E.g.:

 • THE MAJOR ROUTES OF METABOLISM /OF AMPHETAMINE/ ARE HYDROXYLATION OF AROMATIC RING TO P-HYDROXYAMPHETAMINE & DEAMINATION TO BENZYL METHYL KETONE, FOLLOWED BY OXIDATION TO BENZOIC ACID. A CONSIDERABLE SPECIES DIFFERENCE IN METABOLISM OCCURS. IN MAN A LARGE AMOUNT OF THE DRUG IS EXCRETED UNCHANGED (30%) & THE MAJOR METABOLITES ARE BENZOIC ACID (20%), BENZYL METHYL KETONE (3%) & P-HYDROXY-AMPHETAMINE (3%). IN THE RAT THE MAJOR METABOLITE IS P-HYDROXY-AMPHETAMINE (60%), IN THE RABBIT IT IS BENZYL METHYL KETONE (22%) & BENZOIC ACID (27%), & IN THE DOG IT IS BENZOIC ACID (32%) & UNCHANGED AMPHETAMINE (38%).
 • CARBONYL REDUCTION ... /CAN INVOLVE/ COMPOUNDS WHICH ARE THEMSELVES DRUG METABOLITES. OF PARTICULAR SIGNIFICANCE ARE THE CARBONYL DERIVATIVES RESULTING FROM DEAMINATION. ... WHEN THE PRODUCT OF DEAMINATION IS A KETONE, IT /IS/ OFTEN RECOVERED IN SIGNIFICANT AMOUNTS TOGETHER WITH ITS REDUCED METABOLITE. A CLASSIC EXAMPLE IS THAT OF AMPHETAMINE; THE TWO METABOLITES, PHENYLACETONE & PHENYLISOPROPANOL ARE EXCRETED IN A RATIO WHICH SHOWS A MARKED SPECIES DEPENDENCY. ...

Considerable species differences in amphetamine clearance are indicated by a biological half-life of 30 minutes in goats compared with over 1 hr in pigs & almost 5 hr in dogs. ...

The minimal lethal amphetamine dose varies with age & animal species. ...

There was considerable species difference in biotransformation, but not in the excretion of (14)C, after admin of (14)C-amphetamine. ... Much of the (14)C was excreted in the 24 hr urine after ip admin of amphetamine to dogs & guinea pigs, & oral admin to other species. Three days after a dose of (+ or -)-amphetamine, human subjects had excreted 91% of the (14)C in the urine, rats 86%, rabbits 86%, & dogs 78%. The last 3 species excreted 5%, 7%, & 0% of the dose respectively in 3-day feces. Since excretion of (14)C over broad time intervals its optical isomers was given, the above data can be compared with results obtained when an optical isomer was given to monkeys, mice, & guinea pigs. Thus, after a dose of (+)-amphetamine, (also known as dexamphetamine ...), guinea pigs excreted 88% of the (14)C in the 2-day urine, mice excreted 87% in the 3-day urine, & a monkey excreted 80% in the 3-day urine. Another monkey excreted less than 58%. Of interest was that man, monkey, dog, & mouse excreted about 30% of the (14)C as unchanged amphetamine in 24 hr, whereas guinea pig excreted 22%, rat excreted 13%, & rabbit only excreted 4%. These results confirmed an earlier study of the excretion of (14)C & of unchanged amphetamine in rats given (+ or -)-amphetamine at 4 different dose levels orally & at 1 level sc. ... These earlier studies showed that rates of excretion of (14)C were similar after oral or ip dosing, thus indicating comparable & rapid absorption of amphetamine in these animals by two routes.
— Pubchem Compound entry on Amphetamine

The most notable difference between humans and animals, which isn't indicated on Pubchem, is that amphetamine is neurogenerative in humans and neurotoxic in many animals. Hence, the importance and relevance of the first bullet. Seppi333 (Insert 2¢ | Maintained) 22:20, 28 January 2014 (UTC)

Yes, I agree that trying to translate non-human testing into human results is a problem. But you've just inadvertently proven my point. This exceptionally well-researched compound has been tested in rats, mice, guinea pigs, rabbits, dogs, goats, pigs, and monkeys. That's eight families and probably about ten species. That means that there are more than 5,400 other mammal species that this exceptionally well-tested compound wasn't tested in. "Tested in 0.18% of non-human mammal species" does not sound like an impressively diverse research program. The normal process for a drug—"Tested in just 0.05% of non-human mammal species"—is even worse. WhatamIdoing (talk) 23:41, 28 January 2014 (UTC)

I think the gist of that bullet has more to do with how the particular enzymes that a drug is metabolized on vary between species as opposed to how a specific drug is metabolized. The only thing my example shows is that drugs metabolized on FMO3 and CYP2D6 may have significant pharmacokinetic differences between lab animals and humans. Seppi333 (Insert 2¢ | Maintained) 04:41, 29 January 2014 (UTC)

So can someone come to a conclusion on the consensus for this proposed addition? I obviously have a COI in doing this myself. Seppi333 (Insert 2¢ | Maintained) 21:36, 7 February 2014 (UTC)

I see no consensus to add it; in fact most comments were against. Without consensus to add it, it should not be added. There may be better sources. Jytdog (talk) 21:44, 7 February 2014 (UTC)

I've added discussion tags with that result then. Seppi333 (Insert 2¢ | Maintained) 09:53, 8 February 2014 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Better explanation of primary vs secondary sources

Wikipedia and virtually every other source defines primary research as that done by the researchers. Secondary research draws conclusions from other studies. I have seen many studies that draw new conclusions from existing studies. These are clearly not primary studies, but Wikipedia considers them so. Wikipedia considers only review and meta-analysis studies secondary studies.

Where is it said that only review and meta-analysis are secondary studies? Surely, there must be a source somewhere. A few Wikipedians have asked me to take this on faith, but I prefer something more concrete. In any case, it is quite clear that the policy needs clarification. AuburnMagnolia (talk) 05:48, 5 February 2014 (UTC)

We describe it in the nutshell at the top ideal sources are "general or systematic reviews in reliable, third-party, published sources, such as reputable medical journals, widely recognised standard textbooks written by experts in a field, or medical guidelines and position statements from nationally or internationally recognised expert bodies" Doc James (talk · contribs · email) (if I write on your page reply on mine) 14:38, 5 February 2014 (UTC)

I think we need to make WP:MEDRS more clear about the status of sources like retrospective studies and general epidemiological data-mining, where a hypothesis is investigated by searching an existing database of raw data. For Wikipedia sourcing purposes, these are primary sources. Zad68 14:53, 5 February 2014 (UTC)

AuburnMagnolia, it might be helpful if you could give us a handful of examples. WhatamIdoing (talk) 19:12, 5 February 2014 (UTC)

WhatamIdoing, I'm pretty sure Auburn is referring to this conversation, where Auburn was suggesting to add content based on PMID 23656698. The finding of this source is the result of a data-mining exercise against the CDC's prevalence databases. Auburn appears to feel that because the study make reference to existing data, it's a secondary source. (Auburn originally referred to it as a meta-analysis but corrected that in a later statement.) The CDC data is what Auburn appears to be referring to as "existing studies". Everybody else commenting there is looking at it as a primary source. To me it looks like a retrospective study. TenOfAllTrades provided what I thought was a good explanation of the differences between this sort of retrospective study (primary) and a meta-analysis (secondary) but Auburn hasn't accepted this. Zad68 19:41, 5 February 2014 (UTC)

I can easily see why a study that uses a "systematic search" to determine (three) factors for its analysis would seem like a systematic review or similar secondary source. I believe it's a primary source, but it is a difficult source to classify. I'm not sure that it would be easy or necessarily even helpful to clarify the "rules" to cover the most complicated cases. WhatamIdoing (talk) 18:22, 6 February 2014 (UTC)

I think this is exactly the point. I come here to Wikipedia, fairly green, and attempt to learn the rules. I spend a lot of time reading and asking questions. And then I get it wrong. That's frustrating. I absolutely think the rules need clarifying to cover these cases. The fact that you say "I believe it's a primary source" causes me concern. This means that there is some amount of editorial discretion and not just policy. This leaves open the door to editor bias. And I think this is one of the advantages of a strong Wikipedia policy—to reduce as much as possible or eliminate editor bias. AuburnMagnolia (talk) 00:11, 7 February 2014 (UTC)

The world is never as black and white as we like. And this is a publication of an innovative study and classifying it does need judgement. I agree it is primary and I think it is pretty obviously so; they are not synthesizing the results and studies of others and presenting a consensus view. Instead, the authors mined the data collected in other publications, and in other databases, pooled it, and presented an original analysis of that data. It is a piece of original research, not a synthesis of original research of others. Sometimes you need to consider the spirit of the thing. The goal of Wikipedia is not to present cutting edge research, but rather the consensus in the field, and that is what secondary and tertiary sources do. Jytdog (talk) 00:56, 7 February 2014 (UTC)

So why not codify what you and others have told me? That way new Wikipedians won't make the same mistake that I did. People criticize Wikipedia for not being friendly to newcomers. Maybe we can help them out by giving them better information before presenting sources they think are OK only to get shot down. AuburnMagnolia (talk) 04:17, 7 February 2014 (UTC)

Hi Auburn. People here have worked long and hard (long before I came around) to make this guideline useful - comprehensive enough and short enough, so that people will actually read it and understand it and use it. I am sorry you find these efforts (volunteer like yourself) so unhelpful and frustrating. But rather than just complaining, please propose a targeted amendment that would make the guideline better.Jytdog (talk) 18:36, 7 February 2014 (UTC)

I really don't get the attitude. What you said is exactly what I'm trying to do. I'm not complaining at all. I am highlighting a weakness in the policy exposed by my attempt to edit after trying to read and understand the policies. If you go back and re-read my comments, you'll see that it is quite clear how I am suggesting to improve the current policy. I honestly have no idea why there is so much resistance to improving things on Wikipedia. I also don't understand the animosity to newcomers. Very strange. AuburnMagnolia (talk) 02:26, 8 February 2014 (UTC)

I believe that what Jytdog meant was something like, "Please tell me the exact words that you would like to add to this guideline". By the way, have you seen Wikipedia:Identifying and using primary and secondary sources yet? WhatamIdoing (talk) 05:41, 8 February 2014 (UTC)

I was training to gain support for my idea before proposing text. I didn't want to put the cart before the horse. I have read Wikipedia:Identifying and using primary and secondary sources. AuburnMagnolia (talk) 12:58, 13 February 2014 (UTC)

Proposal re retrospective studies and general epidemiological data-mining

Following User:Zad68's suggestion above and addressing User:AuburnMagnolia's request, I suggest that we add the following to the definition of "Primary source": "Publications that present the results of retrospective studies or the results of epidemiological data-mining, in which a hypothesis is investigated by searching publications or databases, are primary sources; these publications are different from secondary and tertiary sources that present the scientific consensus." I have run into the issue of people pushing studies like the one discussed above as secondary, and it could be useful to have explicit language on it. Jytdog (talk) 17:52, 8 February 2014 (UTC)

Thanks, Jytdog, I think this is a great idea. AuburnMagnolia (talk) 13:00, 13 February 2014 (UTC)

External links in refs

Hi, now I think that where possible when we reference open-access articles like those in PMC we should write down the url of said article in PDF format for easy access. I've tried this on the Huntington's disease page, but for some reason it was reverted. Thoughts?

Example:

Baughman, RP (March 2011). "A concise review of pulmonary sarcoidosis" (PDF). American journal of respiratory and critical care medicine. 183 (5): 573–81. doi:10.1164/rccm.201006-0865CI. PMC 3081278. PMID 21037016. {{cite journal}}: Unknown parameter |coauthors= ignored (|author= suggested) (help)

Fuse809 (talk) 13:22, 19 February 2014 (UTC)

Toxicity and MEDRS

I'm sure this issue has been covered before, but I'm having trouble finding it. My question is a general one, which I will illustrate using some examples from the Glyphosate article. Specifically these questions revolve around issues of how to take into account that much/most toxicity data that is available, and that is used by regulators, seems to fall outside the requirements of MEDRS as I have understood it.

"The EPA considers glyphosate to be noncarcinogenic and relatively low in dermal and oral acute toxicity.[19]" Are regulatory reviews by national environmental and medical authorities normally secondary sources? Tertiary?

"A study published in 2000 found that Roundup interfered with an enzyme involved in testosterone production in mouse cell culture.[80] A study by the Seralini lab published in 2005 found that glyphosate interferes with aromatase, an estrogen biosynthesis enzyme, in cultures of human placental cells and that the Roundup formulation of glyphosate had stronger such activity.[81]" Again, these seem to be excluded by MEDRS, but any such result observed as part of pharmaceutical development program would absolutely have to be reported to FDA, and would be considered a major black mark in the evaluation process. So again I find myself leaning toward deleting a statement as not sufficiently proven for Wikipedia that regulatory authorities would take very seriously. Another interesting question that arises here is that these studies were done at unrealistically high concentrations - how to note that sort of thing without being guilty of injecting original research?

"A study on rats published in 2010 found that administering Roundup Transorb orally to prepubescent rats once a day for 30 days, reduced testosterone production and affected testicle morphology, but did not affect levels of estradiol and corticosterone.[83]" Same issue. Seems to be excluded by MEDRS, but one does not perform such experiments on people and most of what is known about the toxicity of many substances comes from animal models. And again in this particular case the study is very poor quality because the doses used were massive and vastly out of proportion to any likely human exposure. I don't like killing it based on the fact that it is an animal study, but its original research if I mention that the doses used in the study are above the LD50 of table salt. How to proceed?

"Various other studies suggest that glyphosate may be mutagenic.[87]" - Just illustrating once again that genotoxicity studies are an important part of EPA and FDA decision making, but as in vitro results, seem to be excluded by MEDRS.

No example of this in the glyphosate article, but noting here that the FDA also makes decisions to put warnings on drug labels and even to withdraw drugs based on aggregate case reports to AERS. But case reports are excluded by MEDRS. Some adverse events are important enough to result in regulatory action even if too rare to be seen in clinical trials. I have mixed feelings about this case report rule, and wonder if there might be some exception for aggregated case reports as covered in a review.

Any clarification and thoughts would be appreciated. I'm really struggling to understand this issue. Formerly 98 (talk) 13:07, 27 February 2014 (UTC)

I have not given the information you are presenting much review, and have not looked at it at all in context, but generally, the idea is to exclude primary sources and cite secondary sources. MEDRS sources typically will cite and be derived from non-MEDRS sources. For issues of toxicology, what happens is that someone reports small-scale research which is not MEDRS compliant. No conclusions can be drawn from this research alone. However, when an organization like the EPA or FDA cites those research findings, then because of that their credibility increases. That may or may not make the government paper a MEDRS source, but one cannot be certain without examining the document because there are a range of quality among government sources. If the government gives an opinion or has a conclusion then that is worth mentioning only by virtue of it being from a major stakeholder and that has nothing to do with MEDRS at all, although often major organizations will back their statements with research which would also confirm it applicable under MEDRS. Is that a start toward helping? Blue Rasberry (talk) 15:20, 27 February 2014 (UTC)

Yeah, I think that's good. It just seems troubling to me that in a lot of ways the MEDRS rules seem designed mostly to address issues of efficacy (where most subjects are expected to show an effect of exposure) and not so much toxicity (where a serious adverse effect in 0.1% of exposed persons is significant, but difficult to detect via the usual RCT gold standard). Because of the lower frequency of events in toxicology and because of ethical considerations, toxicology knowledge is much more likely than efficacy knowledge to be based on lower quality evidence such as in vitro studies, animal studies, and case reports. Furthermore (and possibly because it is lower quality data), my experience is that reviews of tox data are fewer and farther between than efficacy reviews in most cases.

So I find myself worrying that MEDRS, if systematically enforced (its often not), would systematically bias Wikipedia toward an unrealistically high benefit/risk ratio of drugs, ag chemicals, etc. So maybe this is a philosphical concern more than an interpretation one. Would you delete any or all of the specific items above if only primary references could be found? Formerly 98 (talk) 15:53, 27 February 2014 (UTC)

I totally support adding language to MEDRS specific to toxicity as per my previous bellyaching on this. I was invited to create an essay for consideration which I haven't done yet. my bad. i very much appreciate Formerly bringing this to everyone's attention, as it remains a problem. Jytdog (talk) 16:28, 27 February 2014 (UTC)

We want sources that take into account all the primary sources and give them due weight. This is often a regulatory agency or a toxicology textbook. Both of these would be good sources. We do not want to have people selecting their favorite primary sources as than they could write anything they want as there are nearly always sources that go both ways. Doc James (talk · contribs · email) (if I write on your page reply on mine) 22:46, 27 February 2014 (UTC)

Reviews by from regulatory agencies are still reviews. Reviews of case studies are still reviews.

Some regulatory documents are primary, but you may WP:USEPRIMARY. It'd be silly to demand a systematic review on whether the FDA approved a drug or not: you can perfectly well cite a primary source, such as the FDA's announcement of their decision for this non-biomedical fact. WhatamIdoing (talk) 01:01, 6 March 2014 (UTC)

Question about a journal

Hey all, I am looking to get some wider feedback on a journal. The journal is Acupuncture In Medicine. Briefly, AiM is owned by the British Medical Acupuncture Society and published by BMJ; it is MEDLINE-indexed, and peer-reviewed. Its stated aim is "to promote the scientific understanding of acupuncture and related treatments by publishing scientific investigations of their effectiveness and modes of action as well as articles on their use in health services and clinical practice....The Western understanding of neurophysiology and anatomy is used to interpret the effects of acupuncture." Arguments are being made on the German acupuncture trials Talk page (here and here) that the journal itself is an unreliable source under Wikipedia:FRINGE#Independent_sources and, most relevant to this Talk page, under Wikipedia:MEDRS#Use_independent_sources. The claim is that all articles related to acupuncture from this journal should be considered unreliable sources, because the journal is not independent of the field of acupuncture. All the other usual issues would apply (what is the specific content being supported, and is the specific article used as a source a review, etc). It is the wider question I am coming with, however, as it is a conversation stopper. I can see how reasonable arguments can be made on either side of the question, but my sense is that the claim is invalid. Input is welcome! Input grounded as carefully on policies and guidelines as possible (not wikilawyered, just grounded) is very welcome. Jytdog (talk) 13:25, 5 March 2014 (UTC)

The editorial board is clearly made up of only acupuncture believers, and they are promoting an editorial bent which is clearly promotional of acupuncture in a very seedy way. This list is particularly damning. They are preferentially recruiting acupuncturists and pseudoscience/altmed-supporters of various sorts. Notice how, for example, Edzard Ernst is prominent in his absence on that list as are Bed Goldacre, David Colquhoun, and Steven Novella. The journal is systematically excluding WP:FRIND. jps (talk) 13:44, 5 March 2014 (UTC)

I'm not so concerned about the absence of Ben Goldacre on the editorial board (he plays pretty fast and loose with the facts himself at times), but on first pass roughly 9 of the first 10 clinical trials I found published in this journal gave positive results. This is a dramatically higher rate than is found in the mainstream literature. I don't know how exactly we exclude this journal however given that it is Medline indexed. Formerly 98 (talk) 14:12, 5 March 2014 (UTC)

Curiosity: with which facts has Ben Goldacre played fast and loose? jps (talk) 18:22, 5 March 2014 (UTC)

A journal owned by acupuncturists whose editorial control is by acupuncturists cannot possibly be what WP:FRINGE and WP:MEDRS has in mind regarding independent sourcing for an objective look at acupuncture. I would suggest that articles from this journal are reliable for what acupuncturists think but are probably not reliable as an independent, objective source to see if it is effective. As always, there is an "out" here; if the journal article in question has been positively cited in the medical literature, this would go a long way to establishing it as reliable for MEDRS purposes, regardless of where it is published. If it has not garnered much support in the medical literature, I would avoid its use and use only those sources clearly independent of the topic. Without knowing the background, if the only journal articles stating that acupuncture work are coming from acupuncture owned and edited journals, that is a massive red flag.

I would think an analogous situation would be the journal Homeopathy (See WP:PARITY), which although MEDLINE indexed, is specifically disallowed by WP:FRINGE, for what I assume is the exact same reasons. Yobol (talk) 15:37, 5 March 2014 (UTC)

You can't condemn the entire journal out of hand merely because it has an editorial position. This is on par with saying that (your choice: Fox News or The New York Times) should always be rejected as a source. Shall we reject all specialist journals, unless we can prove that the editorial board has regular fights? I bet you'll find that the editorial board of NEJM has only people holding MDs. Anyone can see that this is obviously "biased" against DOs and nurses, and promotes MDs. Do you think you'll find many neurologists on the editorial board of JAMA Psychiatry? I don't. How about endocrinologists, to make sure that thyroid disorders that mimic psychiatric conditions are given fair attention? I'm guessing they've got zero. Shall we toss them out, too, since we've just proven that their boards are "biased" and "non-independent"?

When we talk about the need for an WP:Independent source, we are talking about whether that the author or publisher is likely to directly benefit from it. Publishing information about how you are the best _____ in the world or that people with this condition should pay you for treatment is not independent. Publishing what you did in a clinical trial, with no reason to believe that you'll benefit from this, is sufficiently independent.

Or, to put it another way, whatever the game, whatever the rules, the rules are the same for both sides. If it's "independent" for a surgery journal with a bunch of surgeons on its board to let a surgeon publish what happened to his surgery patients, then it's "independent" for an acupuncture journal with a bunch of acupuncturists on its board to let an acupuncturist publish what happened to his acupuncture patients. SPOV was rejected. We don't apply different rules to subjects that editors decide are "scientific" or "not".

Additionally, nearly any source can be reliable for some statements, per the FAQ (which you'll find towards the top of WT:V). WhatamIdoing (talk) 01:39, 6 March 2014 (UTC)

The publisher of this journal is in the business of promoting acupuncture for essentially any and all ailments. Acupuncture, I would remind you, is a non-protected term. Anyone can call themselves an acupuncturist. What surgery journal are you referencing which excludes articles that show evidence that surgery for certain conditions may not be advisable? Most surgery journals allow for non-surgeons to review clinical studies, for example. Do you see the problem? jps (talk) 03:30, 6 March 2014 (UTC)

The publisher of this journal is in the business of promoting acupuncture for essentially any and all ailments. Citation needed? Original research?

The publisher actually says that their intent is "to promote the scientific understanding of acupuncture", which is importantly different from saying that they're "promoting acupuncture".

Anyone can call themselves an acupuncturist.

If we assume that this "anyone" doesn't mind spending a year in jail, then I guess you're right. (Guess what? That's the same jail term given to people who pretend to be physicians in California.) It may not be a protected term everywhere, but it certainly is protected in some places. WhatamIdoing (talk) 22:47, 7 March 2014 (UTC)

Also, I checked and many of the members of the editorial board of JAMA Psychiatry are not psychiatrists. One is, in fact, a neuroanatomist. Others are statisticians. In fact, the board looks exactly like what one would hope a board would look like for a reputable journal. It's a good study in contrasts. jps (talk) 05:56, 6 March 2014 (UTC)

thanks, WAID. this is pretty close to how i see it. this is such an interesting question to me. under your (our) reasoning, however, any group of fringe-y practitioners can set up a journal and even have critical reviews in it that are perfectly MEDRS compliant. right? would you please address that? am very interested in your thoughts on the hard part :) Jytdog (talk) 02:23, 6 March 2014 (UTC)

It's difficult. Wikipedia allows sources to be biased without impinging on their reliability. What we have to do is consider whether it's WP:DUE, considering all the sources that exist in the world. It's not good to over-represent the word of people in the field against people outside of the field—or the other way around. But you can't completely exclude either side or toss out all of the sources that are from one "side". WhatamIdoing (talk) 22:47, 7 March 2014 (UTC)

@WAID: Not sure if you're replying to me or not, but I did say specifically say what it would be reliable for, in my opinion. Regardless, let's take this a step back and use a more apt analogy. Let's say a group of orthopedic surgeons say they came up with a "revolutionary" new toe surgery that cures low back pain. The technique is so lucrative that they teach a select group of other surgeons who promote this across the country. There is no medical/physiological reason for such a surgery to work, and studies conducted by outside groups find negative results. This group then sets up a journal specifically dedicated to this surgical technique, the editorial board is comprised of surgeons who perform this technique and surprisingly, the results in this journal are markedly more positive than the studies done by, and published in, journals not affiliated with this technique. I would suggest to you that this journal has a clear conflict of interest with regards to the technique, and is not an independent source to objectively discuss the effectiveness of this technique, though it is reliable for what proponents of the technique believe. Is it your suggestion that we should take such a journal at face value? When you have a fringe group, publishing a fringe journal specifically about the fringe topic they are advocating, they are probably not independent of the topic, no matter what degree they hold. If Wikipedia's rules say that we need to treat this particular journal as equally valid as other journals, then there is something systemically wrong with Wikipedia's rules. Yobol (talk) 02:49, 6 March 2014 (UTC)

User:Yobol, I love what you wrote there - that faces that prong of the dilemma headon. OK now I am going to put the hard part of the question to you. It seems entirely plausible (to me at least) that there are scientists who (for whatever reason) want to test acupuncture interventions to see if they work and who do so rigorously and publish good science, and others who might write rigorous reviews. Where do they publish that, but for journals that at least aspire (as per the mission statement) to be solidly scientific, like the one I posted about? There are hardcore skeptics among us who will cut the legs out from an under an effort to use such a source - calling the journal FRINGE stops the conversation; and Wikipedia loses what may be good sources that way (and good sources seem to be hard to find in acupuncture). This is why I am on the fence and brought the question. Maybe this is a case where the fifth pillar comes in, but I am very uncomfortable doing that in alt med articles that are fringey and where we need to reason closely based on PAG. do you see what i mean? thx Jytdog (talk) 03:04, 6 March 2014 (UTC)

Let me be clear, I do not want suggest that there is any intentional wrong-doing on any part of any of these journals. I suspect that the vast, vast majority of these journals, editors, authors have nothing but the best of intentions to do good science; however we all know how easy it is to have unintentional biases, and how many different ways biases can creep into any work, despite the best of intentions. I also do understand your point; we all want to follow WP rules, be as fair to all sources as possible. However, I think WP rules already give us guidance here; we use independent sources (both in fringe and in medical topics), and I would not consider these are not independent sources. There is no need to invoke WP:IAR here. There are enough studies about acupuncture in non-acupuncture journals that we do not need to use the dubious ones. Yobol (talk) 03:25, 6 March 2014 (UTC)

Yobol nails it. The problem is that they just exclude independent review. That's inimical to our rules about independent sources being necessary. jps (talk) 03:27, 6 March 2014 (UTC)

Our rules about independent sources do not demand that the sources not be in the field. We accept specialized string-theory physics journals for writing about string theory; we accept specialized vaccination journals for writing about vaccination; we accept specialized acupuncture journals for writing about acupuncture. WhatamIdoing (talk) 22:47, 7 March 2014 (UTC)

I'm not sure why we're having this discussion. WP's rules are quite clear: whether a journal can be trusted as reliable mainly depends on whether it is peer reviewed; cf. WP:FRINGE#Peer-reviewed sources:

One important barometer for determining the notability and level of acceptance of fringe ideas related to science, history or other academic pursuits is the presence or absence of peer-reviewed research on the subject. While a lack of peer-reviewed sources does not automatically mean that the subject should be excluded from Wikipedia, there must be adequate reliable sources to allow the subject to be covered in sufficient detail without engaging in original research. Care should be taken with journals that exist mainly to promote a particular viewpoint. Journals that are not peer reviewed by the wider academic community should not be considered reliable, except to show the views of the groups represented by those journals.[5] Peer review is an important feature of reliable sources that discuss scientific, historical or other academic ideas, but it is not the same as acceptance by the scientific community. It is important that original hypotheses that have gone through peer review do not get presented in Wikipedia as representing scientific consensus or fact. Articles about fringe theories sourced solely from a single primary source (even when it is peer reviewed) may be excluded from Wikipedia on notability grounds.

. --Mallexikon (talk) 03:58, 6 March 2014 (UTC)

Peer review is a necessary, but not sufficient criteria for most MEDRS. In this case, you cannot take one part of FRINGE or MEDRS out of context to the rest (i.e. the need for independent sources). Peer review is not a magic pixie dust you can sprinkle on a source and say "reliable!". Not all peer-review is the same, and not all journals/editorial boards are the same. Yobol (talk) 04:02, 6 March 2014 (UTC)

You're right that peer review is not a sufficient criteria for MEDRS (sources also would have to be secondary at up-to-date), however, by WP's rules, "Peer reviewed medical journals are a natural choice as a source for up-to-date medical information in Wikipedia articles" (WP:MEDRS#Biomedical journals). That means that WP recognizes peer review as the litmus test concerning the question whether a journal is independent/reliable or not. Of course there are some fringe journals that fake peer review (cf. WP:PARITY: "Note that fringe journals exist, some of which claim peer review. Only a very few of these actually have any meaningful peer review outside of promoters of the fringe theories, and should generally be considered unreliable. Examples of unreliable journals include, but are not limited to: The Creation Research Society Quarterly, Homeopathy, and the Journal of Frontier Science (which uses blog comments[6] as its supposed peer review)"). But I don't see any evidence that Acupuncture in Medicine constitutes this kind of fraud. Let's keep in mind that it is MEDLINE-indexed and published by BMJ. Their reviewer list comprises more than 100 people, and the editorial board clearly does not mainly include acupuncturists. I think it is regrettable that they have 2 people from the "Royal London Homeopathic Hospital" (regrettable that this kind of hospital exists at all) on their editorial board, and I would agree with jps that it is regrettable that Ernst is not on their reviewer list. But that is not evidence enough to label their peer-review a fraud and throw this source out. --Mallexikon (talk) 05:29, 6 March 2014 (UTC)

I have gone through the entire editorial board and reviewer list and see all alt-med professionals or acupuncturists. Can you point to one who isn't? jps (talk) 06:08, 6 March 2014 (UTC)

The reviewer list comprises more than 100 people... You checked them all? --Mallexikon (talk) 07:35, 6 March 2014 (UTC)

It's not that hard to copypaste them into a google search. Try it out. You'll get the hang. I could be mistaken, I suppose, since I didn't run it very quickly, so if you find one who isn't do let me know. jps (talk) 11:32, 6 March 2014 (UTC)

If I recall correctly, the last time we discussed acupuncture (on this page) you called Andrew Vickers and all the others authoring this meta-analysis acupuncturists, so it's difficult to take you, ScienceApologist, at your word. It's really, really sad that I have to say that I question your ability to present information faithfully; you're sharp enough that this means I question your basic integrity. On the overall point, I lean towards allowing Wikipedia to present both sides (especially in, say, the acunpuncture article) but basically agree with Yobol when it comes to the overall evaluation: "There are enough studies about acupuncture in non-acupuncture journals that we do not need to use the dubious ones". If someone feels strongly that there is a niche gap that a specialist reference fills, perhaps a a nuanced perspective of a hypothesized theory, a specialist journal might be appropriate particularly if it can be balanced appropriately. Yes, in my view even in homeopathy the journal Homeopathy likely should be referenced to allow the article to present the proponents' strongest case as such. In my opinion, allowing the fringe view a little breathing room is a fundamental and incontrovertible requirement for NPOV, as much distaste as it leaves for those with hardcore passion. II | (t - c) 07:53, 6 March 2014 (UTC)

Homeopathy is specifically called out as unreliable at WP:FRINGE. You're asking us to ignore our own guideline. jps (talk) 11:32, 6 March 2014 (UTC)

No, we're asking you to pay careful attention to the policy that says "The appropriateness of any source depends on the context". There are statements for which even a garbage journal could be perfectly reliable, or even authoritative, source. WhatamIdoing (talk) 23:02, 7 March 2014 (UTC)

Thanks for everybody's comments - some are more careful and reasoned and generalizable than others. Things are getting wander-y, so I want to pull up. My goal here is to try to understand the basis, grounded in policy and guidelines, for excluding a journal that has many of the trappings of a potential publisher of legit MEDRS-compliant sources, except its subject matter. WAID's post above says "you cannot exclude it." Which is very useful for preventing editors from arbitrarily excluding a journal because they don't like its subject matter, but doesn't address the hard part of the question - that there is FRINGE out there. Yobol's answer calls on Wikipedia editors to examine the reviewers and editorial board of a journal to see if those lists include a wide range of expertise. This has good support in a guideline, as User:Mallexikon quoted above from WP:FRINGE, namely "Journals that are not peer reviewed by the wider academic community..." This seems well generalizable to me, and would be as useful for screening out a journal by a "cabal" of orthopedic surgeons promoting some funky procedure, as for excluding a journal "by acupuncturists, for acupuncturists." Applying this will lead to questions/debates of what "wider" means, of course, and indeed that discussion has broken out above. But interpretation is always debatable. OK, so re-posing the question: is that it then? Or are there relevant bits of policies or guidelines, other than reviewing editorial board/peer-reviewers, that can help rationally exclude or include a journal on a fringey topic as a publisher of MEDRS sources? Thanks again! Jytdog (talk) 13:26, 6 March 2014 (UTC)

Two more things to consider: publication record and citations. If journal articles are not cited outside of the journal itself or outside of the insular community, that's a redflag. Additionally, if the journal has a questionable publication record (for example, only publishing positive studies or only publishing negative studies on competing ideas) that should be considered as well. jps (talk) 01:16, 7 March 2014 (UTC)

are those criteria somewhere in policy or guideline? Jytdog (talk) 01:56, 7 March 2014 (UTC)

I think some editorial judgement is required in the use of sources such as the AIM journal. For references within acupuncture articles and sections, I'd say it is probably an acceptable source ("acupuncture is thought by its proponents to be beneficial for condition X") but you wouldn't want it as a source for general content in the way we would cite Cochrane or non-CAM journals. It is not dissimilar to citing the Bible - it is a good source if what you're saying is that the Bible says one thing or another, but you wouldn't use it as the definitive source on a current-day issue. I'm sure there will be discussions about the scope of AIM as a source, but I wouldn't rule it out on principle provided there is consensus for each individual use. JFW | T@lk 14:30, 7 March 2014 (UTC)

You'll want to check for the usual features of reliable journals:

• Does it have a reputation for fact-checking and accuracy: Do they issue corrections? Do they withdraw bad papers?
• Is it published by a reputable publishing house, rather than by the author(s)? Association with a reputable group is good. Do other sources name it as an example of a bad or captive journal? Do other sources cite it (as much as might be reasonable—niche subjects don't get cited much in general journals)?
• Is it a third-party or independent source, with no significant financial or other conflict of interest? Do they publish only papers written by their own reviewers (or employees)? Will the authors or journal's owners make money off of this (more than the typical academic or medical paper)? Is this part of somebody's get-rich-quick scheme?
• Does it have a professional structure in place for deciding whether to publish something, such as editorial oversight or peer review processes? Do their "instructions to authors" indicate a proper, preferably anonymous, peer review process? WhatamIdoing (talk) 23:02, 7 March 2014 (UTC)

Thank you, particularly to WAID, Yobol, and JFW for taking time to write careful responses. Not a simple question and I appreciate the time and thought. I don't want to drag the battles on the GERAC page out here; you have given us plenty to work with over there. Thanks again.Jytdog (talk) 00:36, 8 March 2014 (UTC)

General comment: I'm a bit late to this party due to a short break, but will post here what I posted at Talk:GERAC: The "independent sources" part of WP:FRINGE has been construed more narrowly than in academia. See Independent sources & refs therein. Dependence isn't a function of a journal's topic in the real world; it actually has to do with external relationships (e.g., a reviewer being the parent of an author). No important conflicts -- according to prevailing standards in academia -- are declared by its editorial board (most/all of whom practice acu, not surprising given the subject matter, and have additional qualifications as well). The journal appears to fulfill WP's other tests for sources, such as peer review and indexing; furthermore, studies published in it have been used by the Cochrane Collaboration and in other accepted MEDRS's, and MD's and PhD's publish in it (e.g. Edzard Ernst). "Dependence" seems to have been misinterpreted by editors unfamiliar with what it actually means. It doesn't have to do with a journal's topic area.

That said, User:Yobol makes a good analogy, but given that Cochrane evaluates papers from this journal on their merits -- and AFAIK they do publish findings of no efficacy, since that's what clinical trials show for most conditions -- it seems to be that we should as well, rather than dismiss it as fringe, or necesarily and inherently biased, outright. IOW, Yobol's hypothetical example is fair, but this real-world case isn't considered conflicted or inherently biased in academia, including among those who write the best of the best MEDRS's (Cochrane). Middle 8 (leave me alone • talk to me • COI?) 17:45, 15 March 2014 (UTC)

Whoa, just caught the thing above on peer review. Their review board doesn't include people outside of alt-med research/practice? If so, obviously it's not reliable. (This is true in any field but especially alt-med; alt-meds shouldn't stay alt, but rather undergo scrutiny and then join real-med or go away. You can't do that by being selective about who reviews the research. I'm disappointed in AIM if they lack the balls & integrity to do it right.) --Middle 8 (leave me alone • talk to me • COI?) 07:47, 17 March 2014 (UTC)

1. There is no policy basis for that statement. We don't get to completely reject FOX News' political coverage even if their editorial board is all Republicans. We don't get to completely reject an academic journal even if all of their reviewers are in the same general branch of medicine. See WP:BIASED: reliable sources are allowed to be biased.
2. You seem to have been misled by the description above. There's a dubious claim that the board is "made up of only acupuncture believers", but there's no real basis for that claim, and certainly no basis for assuming that there are no people outside of the alt med field. This biostatistician, for example, doesn't appear to be an alt med practitioner of any kind, and if you look through his publications, the most prominent subject is surgery for prostate cancer—more than 100 papers on surgery. Now perhaps that's the only one; I just picked a name from an institution I recognized and I haven't checked all the others. But it's clear that the editorial review board is not merely altmed practitioners. WhatamIdoing (talk) 20:35, 28 March 2014 (UTC)

Journal of the American Psychiatric Nurses Association

I am wondering why the Journal of the American Psychiatric Nurses Association would not be an acceptable source when used to discuss psychiatric care of cancer patients. Thoughts? Gandydancer (talk) 15:40, 28 March 2014 (UTC)

I am wondering why you think that treatment of psychiatric conditions = treatment of cancer. Thoughts? Yobol (talk) 19:35, 28 March 2014 (UTC)

Cancer patients and cancer survivors commonly have psychiatric conditions as a result to diagnosis and treatment. It might range from PTSD to physical brain damage. Mixed anxiety and depression is common. It would not be "a treatment of cancer", but it would certainly be "a treatment that is necessary because of cancer". WhatamIdoing (talk) 20:16, 28 March 2014 (UTC)

Of course, numerous chronic medical conditions are associated with depression and other psychiatric conditions, not just cancer. Treatment of the associated psychiatric conditions is still != treatment of that condition. This seems to be going astray of the intended use of this talk page, though. Should've known better than to respond to a loaded question. Yobol (talk) 19:12, 29 March 2014 (UTC)

Thanks Waid. You (and Doc James) are one of the few editors around these parts that do not treat "outsiders" with contempt and I appreciate it. I believe that this language: Other indications that a journal article may not be reliable are its publication in a journal that is not indexed in the bibliographic database MEDLINE,[16] or its content being outside the journal's normal scope (for instance, an article on the treatment of cancer in a psychiatric nursing journal) needs to be changed. Surly you guys can find a better example, and I will bite my tongue rather than say more. Gandydancer (talk) 21:35, 29 March 2014 (UTC)

I doubt that anyone reads that (in good faith, and I know that's not always an entirely accurate description of people reading this page) and decides that something in that journal would not be useful if it's about psychiatric care of people with brain tumors—after all, "psychiatric care" of anybody is within the remit of that journal. But if instead you're seeing an article in that type of about whether this or that cytotoxic drug produces a higher treatment-free survival period, then I think you should be concerned. WhatamIdoing (talk) 01:21, 1 April 2014 (UTC)

You seem to be unable to see my position and why I brought this up. I had hoped for a discussion with perhaps a change in wording. Since you all seem to actually believe that the Journal of the American Psychiatric Nurses Association would carry an article outside of their expertise, cytotoxic drug treatments for example, there is no reason for further discussion. Gandydancer (talk) 01:33, 4 April 2014 (UTC)

I think WhatamIdoing hit the nail on the head, and your response was way too harsh Gandy. Unbelievable that a journal would ever publish an article that was out of scope and outside the expertise of its referees? Here's an article on the costs of chemical synthesis of sofosbuvir. It was written by pharmacologists and an x-ray crystallographer without the participation of a single synthetic organic chemist, refereed by MDs, and published in the Journal of Clincial Infectious Diseases. Here's an article on the safety of driving while using a cell phone. The authors are transportation safety experts and statisticians, but it was referreed by MDs who lack the expertise to evaluate this type of research and published in the NEJM. Here's an article on the economics of drug development, written by an MD without industry experience, refereed by MDs, and published in the BMJ. No sign of any one with finance expertise anywhere in the process. Sadly, this sort of thing happens all the time.Formerly 98 (talk) 03:04, 4 April 2014 (UTC)

More to Gandy's point, this journal might not, but some journals do. WhatamIdoing (talk) 03:16, 4 April 2014 (UTC)

@ Waid, no that's not my point at all. Formerly, your examples really do not fit with what I'm saying. The three examples, use of cell phones while driving, the cost of the manufacturing of a drug, and a third on drug economics are not topics that for our purposes here on WP are considered medical topics as far as the guidelines are concerned. My point is that I do not believe for one minute that any of the peer reviewed psychiatric nursing journals would overstep their boundaries and publish an article that was outside of their area of expertise, in other words an article about physical care, and Wikipedia need not give dire warnings about the possibility of this supposed danger. Please again read the way this comment about nursing journals is used and see why nursing professionals might roll their eyes and groan wondering why Wikipedia would still be using this sort of thinking/language. Gandydancer (talk) 04:36, 4 April 2014 (UTC)

Apologies Gandy, I seem to have missed your point, perhaps you could repeat it succinctly and very directly? My point was that if the NEJM and Clinical Infectious Diseases, both of which are high quality medical journals, occasionally publish articles that are outside the expertise of their reviewers, it seems a fair bet that others do from time to time also. But it seems that I was arguing off-point? Formerly 98 (talk) 14:39, 4 April 2014 (UTC)

Formerly, one need not have been the retired director of nursing of a large teaching hospital (hint, hint) to see that it would be best to change the wording that I have pointed out. It should be obvious regardless of one's background, IMO. But if perhaps you still don't understand where I'm coming from, you should be aware of "The doctor-nurse game" [7] Gandydancer (talk) 15:54, 4 April 2014 (UTC)

Or, as I said above, the problem isn't that this journal—or that any nursing journal—would do this; the problem is that some journals publish things outside of their expertise. This example is probably picking on nurses unfairly (psychiatric nurses, even: we're using one of the lowest-status medical specialties, nursing, and women, too, as our example of potential malefactors). We could just as easily pick on "winners" with an example like, "an article about diagnosing prostate cancer published in a cardiology journal" or "an article about treating dandruff in a radiology journal". WhatamIdoing (talk) 16:07, 4 April 2014 (UTC)

I have no qualms in changing examples as suggested above. Not sure that we are "denigrating" anyone as this is just an example, if we change examples, we aren't "denigrating" cardiologists or radiologists in those examples, either. (As an aside, there are plenty of male nurses as well, so I'm not sure how this can be construed as picking on women). Yobol (talk) 16:28, 4 April 2014 (UTC)

Added another example, so it doesn't seem like we're "picking on" anyone in particular. Hopefully this will meet with everyone's satisfaction. Yobol (talk) 16:34, 4 April 2014 (UTC)

Thanks Waid, you got it exactly right. While the present wording, "efficacy of a new cancer treatment in a psychiatric journal or the surgical techniques for hip replacement in a urology journal" certainly takes care of my complaint, I'd actually be very, very surprised if one could find an example of that happening. But that's another argument and not mine. BTW Yobol, men make up less than 10% of RNs and they have, incidentally, improved the profession of nursing in the same way that the addition of a large number of female physicians has improved that profession. Gandydancer (talk) 15:13, 5 April 2014 (UTC)

I hope you're right. Of course, the point of the example is to give people an idea of what we're talking about, not to document specific past problems. I think that making them more extreme or silly will help people figure out what we're talking about (the ones who didn't instantly understand and start mentally reciting their own examples). WhatamIdoing (talk) 15:59, 5 April 2014 (UTC)

Agreed that the point of the example(s) was not to document prior incidents or to "denigrate" any particular group, but used as a point of illustration for those unfamiliar with medical journals and that some have specific scopes. I hope we don't soon find some urologists taking umbrage... Yobol (talk) 16:35, 5 April 2014 (UTC)

Yobol, the wording was changed because it is important that Wikipedia not use ignorant and prejudicial thinking/wording, not because I, or nurses, or psychiatric professionals, or "women", would feel unreasonably insulted. I'd suggest that you save your sarcasm for another day. Gandydancer (talk) 18:19, 5 April 2014 (UTC)

I see nothing "ignorant" or "prejudicial" about the previous wording. I will leave it at that, as everyone seems satisfied with current wording and that is the reason for this talk page. Yobol (talk) 20:03, 5 April 2014 (UTC)

The stifling impact of secondary sources on the flow of information.

I have to admit that there is particular behavior that is annoying me. I understand MEDRS considers secondary sources as a golden standard. However, most members of MEDRS use the "ideal" standard as an excuse to remove primary studies at their own discretion, especially if they are the primary editor of an article. In most instances there is little attempt at qualifying the information that is currently available. In fact, I've noticed several instances of individuals using outdated literature/systemic reviews as a way of reaching their wanted conclusion. To name names, Ashburn used a literature review on lipoic acid's impact on dementia to state that there is not good evidence of it's efficiency. However, when one takes time to read the cited article it simply stated that there were no randomized/controlled trails using lipoic to treat dementia and that it was not recommended currently. Ashburn then reverted multiple attempts at stating this fact simply saying that he was making it "plainer". This isn't me holding a vendetta against Ashburn specifically; I have noticed this behavior from many people within the MEDRS community.

This sort of behavior is bad enough, however, it is even worse when one realizes that conceptions such as statistical significance and clinically beneficence are arbitrarily defined social conventions. Reading many articles as they are currently written guarantees that readers will reach unfounded conclusions.

I have few ideas that could remedy this situation. 1. Extensively qualifying primary and animal model studies instead of reflexively deleting them. There is no need to ignore all evidence that is not ideal. This is especially true with newer and behavioral treatments.

2. Instead of using terms like "weak", "no good" or "good" evidence actually state the numerical benefit derived by a particular treatment compared to active or passive placebo. This is probably the best way to avoid systemic biases.

3. Recognize that the current used hierarchy of scientific journals is again a social construct with it's own particular problems. It's not reasonable to pick and choose which article to use based on the impact factor of the journal that published it nor altmertics.

4. Understand that there are other Wikipedians with differing editing styles. And that there are few rules that allow carte blanche authority within this faction.

TL;DR: Don't let the perfect be the enemy of the good.

I read and edit Wikipedia to discover, learn, share new and interesting information. I am also by nature an inclusionist and eventualist. I find it dismaying to be beholden to the arbitrary whims of councils and bureaucracy. I would much rather allow the free flow of information than win internet points and attain a "good" article status. I sincerely hope we can find a way to bridge this impasse. Khimaris (talk) 19:33, 31 March 2014 (UTC)

Hmm, you've been mass removing secondary sources from Lipoic acid‎ with misleading edit summaries[8] and and now repeatedly adding bigged-up language ("emerging evidence") and re-interpreted conclusions to omit their negativity. I don't know how that helps the flow of information; rather it helps the flow of misinformation. Alexbrn ^{talk|contribs|COI} 20:54, 31 March 2014 (UTC)

I'm perfectly fine with all results as long as they are verifiable and substantially qualified. Your edits in that particular article had a negative bias( NEGATIVITY IS A BIAS ) that needed to be addressed. In one source that was cited you prevented the addition of information that there were no known trials of Lipoic acid and tried to use that to say that there was "no good evidence". In another there was a cited review article that simply restated the existence of a trail of lipoic acid on metabolic disorders. Except it stated a negative review whilst the original article did not. This is what I mean about lot letting the perfect be the enemy of the good and not being bogged down by arbitrary definitions. It's simply lazy to use reviews in the manner that you have been using them. Some amount of detective work is required.

And now of course you've sent me a warning. As if any sort of addition I could make that deviates from what you've written would ever not be reversed by you or anyone else in your faction. This sort of behavior is the greatest continuing dick move being pulled by the Wiki powers that be. Khimaris (talk) 21:26, 31 March 2014 (UTC)

Khimaris What you are proposing is deeply antithetical to Wikipedia in at least two ways and displays an ignorance of biomedical science.

1. (Wikipedia reason 1) It requires a ton of original research which has no place at Wikipedia.
2. (biomedical science reason) Much of the primary biomedical literature is preliminary and turns out to be unreplicable - much of it is noise, not information.
3. (Wikipedia reason 2) Wikipedia has nothing to do with "the free flow of information." Our mission is to provide reliable and verifiable information to the public, which is why both WP:RS and WP:MEDRS emphasize using secondary sources (which also guards against the introduction of original research in the selection and weighting of primary sources). The importance of secondary sources is even greater in the biomedical field, as per #2; secondary sources are essential for determining what actually is information and is not noise.

You need to understand these things if you are going to have a happy and productive time at Wikipedia. If you don't agree with these things, you are going to have change the clear preference for secondary sources in WP:OR and WP:RS which are each policies; I don't know what you would do about #2 as that is just reality. Jytdog (talk) 21:37, 31 March 2014 (UTC) (correct errors, Jytdog (talk) 10:38, 1 April 2014 (UTC))

No, Jytdog it is antithetical to YOUR personal view of Wikipedia and the existing culture of some within the MEDRS community.

(Wikipedia reason 1) It doesn't require original research to state the numeral benefit derived from a treatment nor to state the age of a study. Neither does avoiding arbitrarily derived terms. Am I the only person here who finds it unacceptable to use an old article stating no currently existing trials to state that a particular treatment is not effective whilst there are exiting (newer) trials on pubmed? That should give you at least an ounce of pause. You say that Wikipedia is for the dissemination of reliable (sourced and independent) information but in many instances it is very lacking.

(biomedical science reason) Your point? Wikipedia is not an academic product nor a place to seek personal medical information. Given time wrongs will be righted. That is the basic premise behind the scientific method. This is why we know that vaccines don't cause autism ffs.

(Wikipedia reason 2) Every edit that I have made that was eventually removed by a member of MEDRS met every gold standard. I'm calling bullshit on that one. It doesn't matter how "reliable" and "verifiable" the information posted is. If it is disagreeable to an extensive editor of an article it gets removed and other members of MEDRS fall in line with agreement.

Basically you've just told me to shut up, fall in line and nod my head to you and the other members. Somehow all the rules, regulations and culture put in place by the current and previous members of MEDRS are infallible and unquestionable. How dare some new comer question anything that is currently in place. And people wonder why the active membership of wikipedia is in decline.Khimaris (talk) 22:09, 31 March 2014 (UTC)

I haven't told anybody to shut up. I don't think you understood what I wrote - I tried to tell you that you are barking up the wrong tree. MEDRS cannot change until the mission and policies change. Really -- wrong tree. Happy to discuss on your Talk page or mine, but I am not willing to take up more of Talk:MEDRS with this. Jytdog (talk) 22:22, 31 March 2014 (UTC)

I think Jytdog's position is pretty much that of the community. That's how the rules that currently exist came into existence, because they reflect the values of the community as a whole. The rules are not "infallable and unquestionable", but they're not all wrong just because you disagree with them either. Majority rule falls fall short of ideal, but it beats the hell out of minority rule.

I'm a relative newcomer here, and have had some influence on the MEDRS rules. You have to start with the idea of what the community values and try to show how changes will advance the values of those who are already here and have worked for years on this before you got here. Coming in and saying its all wrong because the rules exclude some data that you want to include is the wrong approach. The rules cause some problems and they prevent other problems. Everybody's trying to make the system as good as it can be, and its natural that those who have committed effort for many years will have more say than those who started editing last month.Formerly 98 (talk) 23:03, 31 March 2014 (UTC)

Hi Khimaris, and welcome to Wikipedia.

The thing about using secondary sources isn't something created on this page. It's actually in the main policies at WP:No original research and WP:Neutral point of view. MEDRS isn't allowed to contradict that. WhatamIdoing (talk) 01:24, 1 April 2014 (UTC)

Per WP:OR "Policy: Unless restricted by another policy, primary sources that have been reliably published may be used in Wikipedia; but only with care, because it is easy to misuse them.[4] Any interpretation of primary source material requires a reliable secondary source for that interpretation. A primary source may only be used on Wikipedia to make straightforward, descriptive statements of facts that can be verified by any educated person with access to the primary source but without further, specialized knowledge. "

Per WP:UNDUE "Neutrality requires that each article or other page in the mainspace fairly represents all significant viewpoints that have been published by reliable sources, in proportion to the prominence of each viewpoint in the published, reliable sources.[3] Giving due weight and avoiding giving undue weight means that articles should not give minority views as much of, or as detailed, a description as more widely held views. Generally, the views of tiny minorities should not be included at all, except perhaps in a "see also" to an article about those specific views. For example, the article on the Earth does not directly mention modern support for the Flat Earth concept, the view of a distinct minority; to do so would give undue weight to it."

I fail to see the issue. You can hold secondary sources as idea whilst using primary sources when necessary. Khimaris (talk) 06:27, 1 April 2014 (UTC)

yep, you are indeed failing to see the deeper issues here, how this all fits together. When you see that, you will recognize it as a beautiful thing. We cannot make you see it. I am not sure you are trying to and indeed you have asked no questions, but rather argued. We can lead you to water....Jytdog (talk) 10:42, 1 April 2014 (UTC)

Wikipedia articles should be based on reliable, published secondary sources and, to a lesser extent, on tertiary sources and primary sources. Secondary or tertiary sources are needed to establish the topic's notability and to avoid novel interpretations of primary sources....Do not base an entire article on primary sources, and be cautious about basing large passages on them. WP:SECONDARY

Neutrality assigns weight to viewpoints in proportion to their prominence. However, when reputable sources contradict one another and are relatively equal in prominence, describe both approaches and work for balance. This involves describing the opposing views clearly, drawing on secondary or tertiary sources that describe the disagreement from a disinterested viewpoint. WP:BALANCE

Base articles largely on reliable secondary sources. While primary sources are appropriate in some cases, relying on them can be problematic. WP:V

Articles should rely on secondary sources whenever possible. For example, a review article, monograph, or textbook is better than a primary research paper....Primary sources are often difficult to use appropriately. While they can be both reliable and useful in certain situations, they must be used with caution in order to avoid original research. While specific facts may be taken from primary sources, secondary sources that present the same material are preferred. Large blocks of material based purely on primary sources should be avoided....Ideal sources for biomedical assertions include general or systematic reviews in reliable, third-party, published sources, such as reputable medical journals, widely recognised standard textbooks written by experts in a field, or medical guidelines and position statements from nationally or internationally reputable expert bodies. WP:RS

The preference for secondary sources is very clear. You may WP:USEPRIMARY sources in some limited circumstances. Howver, those limited circumstances do not include:

1. "to support large passages",
2. "to debunk or contradict secondary sources", or
3. "to present WP:The Truth" (which involves cherry-picking primary sources based on your personal assessment of whether they're any good). WhatamIdoing (talk) 17:46, 1 April 2014 (UTC)

It's limited beyond that now. Khimaris, I'm the only one who voted against it, if you see above. I think some compromise could have been made. I think I am just frustrated that science lags behind where it could be, and secondary sources take forever to finally say the same thing primary sources say. There is a lack of medical reviews, and that's a problem independent of Wikipedia. I've seen people cherry-pick information from secondary sources, and bias them, until I got a hold of them behind paywalls, when it wasn't as biased as the editors wanted others to believe. Those secondary sources were much closer in content as the primary sources, than what was being told. - Sidelight12 ^Talk 22:23, 1 April 2014 (UTC)

User:Sidelight12 only with a crystal ball, which none of us have, can you say which of the many primary sources will ultimately be accepted by experts in the field, and what weight they will be given, when reviews come out. You, like Khimaris are not grappling with the unreliability of primary sources in the biomedical field, and the OR that any editor must do to select and give weight to any given primary study when it is brand new. Please consider the deeper issues. Thanks. Jytdog (talk) 00:55, 2 April 2014 (UTC)

That is a well thought out response, Jytdog. I also think primary sources can be used without OR, if they manifest a concrete result. For, an abstract result, or questionnaire yes, what you are saying is definitely correct. I still think we should think everything out, for possible exceptions that primary sources could be used. - Sidelight12 ^Talk 05:03, 3 April 2014 (UTC)

What's a "concrete result" though? Isn't the problem that what seems to be a concrete result as stated in a primary is often discounted when a secondary review identifies flaws in the research? Alexbrn ^{talk|contribs|COI} 06:27, 3 April 2014 (UTC)

Alex,

If you see a new study that says, "We surveyed ten million people and found two with naturally green hair", then it does not require any violation of WP:OR at all to write, "One study claims to have found two people with naturally green hair" (or to remove an outdated statement saying, "No scientific report has ever claimed to find anyone with naturally green hair"). You might later find that it's a WP:DUE violation, but there's no OR problem here. WhatamIdoing (talk) 15:03, 3 April 2014 (UTC)

Close, WhatamIdoing. A survey often requires opinion from the participant, so that is abstract, but the source's results are a fact, still open to interpretation. A concrete result would be, this can be quantitatively measured, or this can be seen visually. It doesn't even have to change or contradict the secondary source, it can be an add-on. The case you pointed out, where there can be flaws in research is valid, Alexbrn. It may be true most of the time, but could there be exceptions, where there a result is plain to see, and requires no interpretation? - Sidelight12 ^Talk 07:22, 4 April 2014 (UTC)

In practice, I'm not sure. Something that was an obvious fact would hardly need sourcing, and such things are unlikely to be stated as the result of a piece of research. I think context is a key influencing factor: statements from primary research which might seem innocuous in isolation are often included in a "Health effects" section or an article lede in a way which implies they are medically significant. The very fact something has been selected for inclusion in an article implies it has a certain significance - and it is this implication where the editorial OR arises. My question is always: why is Wikipedia elevating this piece of primary meterial to wider notice when no other secondary or tertiarty source has? (For if it had, we should use that instead.) Alexbrn ^{talk|contribs|COI} 13:56, 4 April 2014 (UTC)

Sidelight, you are going way beyond what an editor is permitted to do. We don't get to decide whether their results are correct, concrete, quantitative, or anything else. We are prohibited from adding our own analysis or opining on whether their methods were valid. If the reliable source says, "We found that the incidence of naturally green hair is one in five million", then it is an undeniable fact that they found this result. We can report what they say: "One study in 2014 surveyed ten million people and reported two cases of naturally green hair". This is a statement of undeniable, non-opinionated, uninterpreted, un-analyzed fact.

Alex, sometimes we report these things because it's better not to be wrong while the academic press churns. If we've got an elderly secondary source saying "Nobody's ever run a clinical trial on this", and you know this is false—say, because the results of the Phase II test are all over the media this week—then it's bad to keep pushing false information out to our readers. It hurts them (by providing false information), and it makes us look stupid. This is explicitly addressed and permitted in MEDRS. (See the stuff with the autism example.) WhatamIdoing (talk) 16:18, 4 April 2014 (UTC)

No I'm not. That was an interpretation, I didn't say. I'm not looking to create OR or analysis. I'm saying what you just said, by concrete I meant "a statement of undeniable, non-opinionated, uninterpreted, un-analyzed fact," without relying on a questionnaire, which is opinion. The survey example you gave wasn't an abstract questionnaire, so ok. I was looking for, a skin tumor shrunk by 40%, undeniable fact that can be seen, and doesn't require any analysis to state it. - Sidelight12 ^Talk 17:13, 4 April 2014 (UTC)

You're looking for "objective". Wikipedia is looking for "published in a reliable source". See WP:ASF: We assert (as an objective fact) that the primary source said this. The primary source might be nothing more than the authors' opinions; that doesn't matter, because we are allowed to assert facts about opinions ("Politician Paul says that he believes he looks good in a blue suit" or "The Supreme High Order of Colorific Sky Scientists says that they believe that the sky is a paler blue than it was when they were all children"). We aren't asserting that the opinion is a fact; we're only asserting the objective fact that the primary source said these things. WhatamIdoing (talk) 17:58, 4 April 2014 (UTC)

Yes we need to use secondary sources. If the secondary source says the evidence is not clear adding a primary source that makes a statement one way or the other "evidence of benefit / evidence of harm" is contradicting it. This is not allowed and appears to be what you are attempting to do. Doc James (talk · contribs · email) (if I write on your page reply on mine) 05:54, 2 April 2014 (UTC)

That doesn't seem like a reasonable response. - Sidelight12 ^Talk 05:03, 3 April 2014 (UTC)

but Sidelight12, doc james was almost quoting MEDRS which says: "Individual primary sources should not be cited or juxtaposed so as to "debunk" or contradict the conclusions of reliable secondary sources." I can't tell you how many times we run into people who have some fringe idea about health (usually they want WP to make strong statements about supposed toxicities or benefits of something) who try to push it with primary sources that contradict secondary sources, and start crying "censoring" and "conspiracy" when they cannot get consensus for it. Jytdog (talk) 10:48, 3 April 2014 (UTC)

It's not reasonable to repeat over and over, we should use secondary sources. We know the rule, we are discussing why. At least, saying something is inconclusive is better than saying there is nothing more. A newer primary source doesn't necessarily contradict that. In a different scene, a secondary source may be incorrect due to newer research. - Sidelight12 ^Talk 07:22, 4 April 2014 (UTC)

In general, I agree with the arguments above about using secondary sources. However, there can be an element of paternalism here which I don't like and which I think brings Wikipedia into disrepute – a view that readers must be protected from primary sources because they won't be able to evaluate them. Suppose an old secondary review supports a particular conclusion, but more recent primary studies, not yet included in reviews, disagree. It would clearly be wrong to write the article to say that the recent primary studies have overturned the older secondary review. On the other hand, to suppress all mention of the more recent primary studies (which may well have been publicized in less careful media) just because they are primary is equally wrong in my view. It's censorship and Wikipedia does not censor. There is a fine line here; it's a matter of careful judgement by responsible editors. I understand all too well that there are irresponsible editors pushing fringe views, but the solution is not blanket bans or blanket supports either way; they are too blunt an instrument. Peter coxhead (talk) 13:17, 2 April 2014 (UTC)

hmmm Peter coxhead I see how you could look at it that way, but it is kind of backwards. The strong preference for secondary sources is not about readers - it is about editors. For editors to choose specific primary sources over others and to give weight to them - and even to judge that the results of the study they want to use are valid and will stand up to replication - requires a level of judgement and even prognostication (!) that in general goes beyond the scope of authority (for lack of a better word) that Wikipedia gives its editors (not authors, not scientists, nor Nostradamuses... editors). So there needs to be a really high bar against using a primary source, and really valid reasons for using it, that the editor who wants to use the primary source can get consensus for. You see that, yes? Jytdog (talk) 13:57, 2 April 2014 (UTC)

Peter, you bring up the word "disrepute." Which will bring more disrepute: allowing editors to interpret primary sources according to their personal biases, with the inevitable flame wars over how different editors are interpreting the same primary sources differently; or requiring editors to stick to secondary sources, where there is less room for editorial bias and, thus, more stable and generally more reliable content? The Wikipedia is an encyclopedia, not a source of breaking news. We can wait until some new piece of research has been analyzed and vetted, and I think we should. TechBear | Talk | Contributions 14:00, 2 April 2014 (UTC)

Yes we sometimes do very rarely use primary sources. These however are not the 20 petra dish studies that "proves" X cures cancer in fibroblasts. Or the 40 person trial of foxtrot versus jive for Parkinson's. These are major trials of high quality with consensus developed around their use created on the talk page first. Doc James (talk · contribs · email) (if I write on your page reply on mine) 16:56, 2 April 2014 (UTC)

It's all a matter of emphasis and judgement. I'm not disagreeing so much as saying that sometimes the pendulum has swung too far one way.

Of course we cannot possibly allow editors to "interpret primary sources according to their personal biases", no sensible person would propose this, least of all me. Of course it's right to strongly prefer secondary sources, and especially to use secondary sources to judge importance. But sometimes these sensible propositions are interpreted not as guidance but as rigid rules that primary sources should not be cited, and this I cannot accept. Jytdog's comment that there should be "a really high bar against using a primary source" (in this context for medical claims) and that the editor who wants to use the primary source needs to seek consensus among other editors is fine, provided that those other editors don't then cite WP policies and guidance as if they imposed a blanket ban on primary sources. And this has sometimes happened.

"Wikipedia is an encyclopedia, not a source of breaking news. We can wait until some new piece of research has been analyzed and vetted" – this is simply not true of Wikipedia as a whole. Just as one tiny example in the science area, the mid-March news of the possible detection of inflationary gravitational waves is already in the lead section of Inflation (cosmology). As another example, whenever I've read in the popular press of the discovery of some new dinosaur fossil, I've always found that the Wikipedia article has already been created. Trying to make medical articles an exception to the rest of science in Wikipedia may or may not be right, but it will be an exception.

Also please take seriously the issue of censorship. Refusing to mention something altogether just feeds conspiracy theorists. Peter coxhead (talk) 18:15, 2 April 2014 (UTC)

2nd paragraph is reasonable, third and fourth are deeply wrong. 3rd: WP:NOTNEWS is policy, and as I've said already, primary studies in our field are generally not reliable. If primary sources are more often reliable in other fields, so be it. 4th: don't even know where to begin. "censorship" is what newbie FRINGE POV-pushers cry, in ignorance and violation of most Wikipedia policies. I have no idea why this is being brought up in an otherwise reasonable conversation.Jytdog (talk) 18:35, 2 April 2014 (UTC)

Jytdog, you're mis-interpreting or mis-reporting WP:NOTNEWS. Another short cut to it is WP:NOTNEWSPAPER. It does not say that new information should not be reported; it basically says that it must be encyclopedic new information – relating to enduring or notable events, information or people.

I can only say that I'm sad that you don't see why reasonable editors should be concerned about censorship, real or imagined. The relevant parts of the opening sentence of Censorship are "censorship is the suppression of ... public communication which may be considered objectionable, harmful, ... as determined by a ... controlling body." This is precisely and exactly the point of WP:MEDRS, namely to suppress the communication of medical information which is considered objectionable and/or harmful (because of its unreliability) as determined by the community of Wikipedia editors. "Primary studies in our field are generally not reliable, so we won't even mention that they exist even when they are widely reported elsewhere" is censorship. It may well be justified censorship; in many cases it is.

Maybe you're assuming that censorship must necessarily be a bad thing. On the contrary, censorship is often highly desirable when the censored communication is harmful or hurtful. But refusing to report the existence of a primary source is censorship and refusing to have a reasonable discussion about whether it is desirable in specific cases isn't helpful. Peter coxhead (talk) 19:01, 2 April 2014 (UTC)

I should note that attempting to devise our guidelines to satisfy conspiracy theorists would be an incredibly futile proposition. We should make our guidelines based on what works to have the best information, not to cater the lunatic fringe. Yobol (talk) 18:47, 2 April 2014 (UTC)

Of course it would be. Only an idiot would suggest otherwise, and I didn't and don't. Peter coxhead (talk) 19:01, 2 April 2014 (UTC)

Your text seemed to indicate that we should beware "censorship" because it would "feeds conspiracy theorists". I was opining that we shouldn't care one way or another about conspiracy theorists at all. It is not "censorship" to not have something in an encyclopedia, it is editorial discretion of what deserves WP:DUE weight in an article. Cries of "censorship" are remarkably unconvincing in this context, and is certainly not appropriate to accuse others of this when they are just trying to put together a guideline that is reliable and accurate. Yobol (talk) 19:12, 2 April 2014 (UTC)

The discussion has become theoretical and unmoored from any concrete question or proposal to change MEDRS. I'm done. Happy to continue theoretical discussions on anybody's Talk page. Jytdog (talk) 19:26, 2 April 2014 (UTC)

Jytdog that's pretty fucking weak. You keep linking to wikispaces touting rules that we should follow: WP:OR, WP:NOTNEWS, and so on. However the content of the pages have nothing to do with what we're talking about. PLEASE talk about the actual issues instead of linking. Consensus building is a two way street. This is not reasonable behavior.Khimaris (talk) 19:55, 2 April 2014 (UTC)

Jytdog It would do us all great justice for you to dispense with the paternalism. Right now you're trying to speak from place of authority without addressing any singular issues. In one instance you say that the use of a primary source requires the use of original research on the part of an editor. That simply isn't true and I've explained the reasons why. On the other hand, I've seen people say that, "If we use of primary sources we'll be flooded with studies saying different things." The same follows for literature reviews and meta-analysises. Every year new papers come out using arbitrary criteria for inclusion, differing definitions of statistical, clinical, and practical significance. Do anti-depressants work? Is saturated fat going clog your ticker? The best "we" can do is shrug (because I'm not even allowed to state how a study interprets it's data!).
And to your response to Peter coxhead: I've had extensive arguments with Doc James over the inclusion, use and interpretation of studies that have met every gold standard of WP:MEDRS. His sources were "better before". And his written interpretation of studies were "plainer"... Nope, these problems are just endemic of using primary sources. /s
TechBear What do you think the peer review service is for and what does that have to do with "breaking news"?
Doc James A scenario for you: lit review from 5 years go says there are no trials for a particular treatment and thus the treatment cannot be currently recommended. Am I not allowed to mention a singular trail from 3 years ago nor it's stated mechanism of action?
Sidelight you can check out r/Scholar if you're ever stuck behind a paywall. If want more consistent access you could register at a local community college or even a four year if you're an America. Online library services there typically allow you access to a great deal of journal publishers.
Lastly you told me that hell is other people. However I'm perfectly willing to raise my hand high to get my point across. Cheers. Khimaris (talk) 18:49, 2 April 2014 (UTC)

You should use secondary sources rather than primary ones. Doc James (talk · contribs · email) (if I write on your page reply on mine) 03:56, 3 April 2014 (UTC)

Re your statement "you told me that hell is other people"..."I'm perfectly willing to [raise the Sign of the horns]..." You appear to be promising to continue engaging in unacceptable behavior 'get your point across'. If you do that, all that will happen is that you will be blocked again for your bad behavior, just as you were blocked for your edit-warring behavior. Wikipedia is a collaborative working environment and you are required to be civil in your interactions with others. Zad68 19:35, 2 April 2014 (UTC)

I personally don't care what you do Zad68, as long as you're in good health. But I do find it curious how you singled out my reverts; as if the reverts done by everyone else were somehow perfect, valid and unquestionable... You're more than free to do as you will. But in the future I'll just make sure that everyone else is inline with all the rules as well. Cheers. Khimaris (talk) 20:12, 2 April 2014 (UTC)

There are a handful of appropriate issues above, so I'm going to talk about some of them.

1. Some editors are too quick to deny primary sources. Or non-scholarly sources. Or sources that don't fit their POV. Or sources that they think don't fit their POV. Some of this is because people are biased (you can be quite openly biased here if your bias is the "right" one.) More often this is the result of people moving too quickly and thinking too slowly. It's so much faster to hit the undo button because "it's primary, and primary is bad" than to figure out whether a secondary source says the same thing, or whether this is one of the times that it's appropriate to use a primary source.
2. Using primary sources does not inherently require original research. Choosing sources—even cherry-picking them—is not necessarily original research. But as a practical matter, when people use primary sources for biomedical information, they often do engage in original research in addition to using primary sources.
3. Peer review by a journal, especially unpublished peer review, is not a substitute for someone publishing an analysis of your work. Peer reviewers can tell you whether the study meets the (perhaps comically low) standards of the journal in question. It doesn't tell you whether the results are typical, whether better studies have been performed, or anything else that is very useful to us.
4. There are times when it is appropriate to cite primary sources. There are a couple of ways that we handle that: we sometimes omit everything, or contrast the two ("Foo is has been generally reported as 42%, but one 2014 study reported the prevalence to be 62%.") However, when you are citing a review about non-biomedical information (e.g., whether organizations have conducted trials), then MEDRS doesn't apply anyway. As a quick rule of thumb, if you can assign a date or place to it, then it's not biomedical information. Events happen on dates and at places: On 01 April 2014, WhatamIdoing started a clinical trial in her garage. Biomedical information is timeless: Which chemical binds to which gene does not change with the calendar. WhatamIdoing (talk) 20:52, 2 April 2014 (UTC)

Thank you for a very clear and helpful contribution. Peter coxhead (talk) 21:38, 2 April 2014 (UTC)

If the point was notable / significant it should be supportable by a secondary source. If the only support is a primary source what is being added may be of questionable notability and may not be accepted generally. Not every peer reviewed article needs to be somewhere in Wikipedia. Doc James (talk · contribs · email) (if I write on your page reply on mine) 03:56, 3 April 2014 (UTC)

It depends what you mean by "supportable". If the only mention of the topic is in a primary source, then, sure, it's highly likely to be of questionable notability. I've never disputed that. It's not an issue. The issues are well set out in WhatamIdoing's points above. In particular, while some editors not familiar with WP:MEDRS or not respectful of it have been too ready to add inappropriate material based on primary sources, equally some editors familiar with it have been too quick to remove primary sources without analysing whether their use was appropriate in the context (and have sometimes appeared to say that primary sources are never acceptable). All I ask is that WP:MEDRS is treated sensibly along with other WP guidelines and policies within the framework of how the English Wikipedia actually operates. Anyway, the points I wanted to make have been made, so enough from me. Peter coxhead (talk) 12:31, 3 April 2014 (UTC)

It's not just about adding inappropriate material; good material gets reverted. We have real-world scientists who assume that the rules are the same as most academic journals, meaning that they must cite the primary sources. And their work gets reverted without anyone stopping to think about whether it's possible to find a secondary source for that material.

There are reasons listed in MEDRS for using primary sources. At least those reasons ought to be accepted. WhatamIdoing (talk) 15:11, 3 April 2014 (UTC)

This may be a matter of perspective. If you are working on relatively non-controversial articles with a lot of new, cutting edge science, I can see where the secondary source rules would be frustrating at times. But I work mainly on articles focused on fairly controversial ones, or at least those for which there are activist groups with outlying opinions. What Wikipedia has taught me is that if you search hard enough, you can find a paper saying that virtually any drug or chemical product is an endocrine disruptor, causes birth defects and cancer at unimaginably low exposures, contributes to global warming, and causes people to vote Republican.

Restricting primary sources causes a lot of hassle, but from my POV loosening the restrictions would inevitably turn Wikipedia into a soapbox for every fringe group on the planet. You can't imagine how many fringe viewpoint edits supported only by outlier primary references I've deleted in the last few months. I can't imagine how much effort it would take to keep these articles reflective of mainstream opinion if I had to debate the merits of each individual primary source.Formerly 98 (talk) 16:14, 3 April 2014 (UTC)

Agree with 98. One of the primary reasons for the strong preferance for review articles / secondary sources is it helps us to determine what weight to give a specific topic. Otherwise you just end up saying study from 2011 states X study from 2012 disagrees and says Y and this could gone on for dozens of pages as we list dozens of primary sources. It is than to easy for people to cherry pick the ones they like. We stop becoming an encyclopedia that is summarizing conclusions and simply become a long list of abstract summarizes selected by Wikipedians. Doc James (talk · contribs · email) (if I write on your page reply on mine) 07:22, 17 April 2014 (UTC)

break

To take a concrete example. I'm looking at the article Polysaccharide-K which mainly details how extracts from this mushroom may be used for cancer treatment, but is very heavy on primary sources. This may be regarded as a edge case because some stronger sources (e.g. American Cancer Society) are equivocal, and because it's apparently used as mainsteam medicine in some countries. Or is the use of primary sources here undue? Alexbrn ^{talk|contribs|COI} 06:52, 3 April 2014 (UTC)

Yes, that's serious over-reliance on primary sources. (I'm sure I could tell you, without looking, who wrote it, too.) However, the important point is what should be done: It should not be blanked. Instead, it ought to be re-written using PMID 22204346 and PMID 22139128 (both reviews, second one free). WhatamIdoing (talk) 15:15, 3 April 2014 (UTC)

Yes, there is a solution in sight there, so it's not as hard as case as I hoped. How about Calorie restriction then? This is an article which seems to make primary studies its centrepiece. Alexbrn ^{talk|contribs|COI} 06:56, 4 April 2014 (UTC)

I don't even need to look at the article. It's the same song, second verse. Go to PubMed, put in "calorie restriction". Limit to Reviews, Free full text, Last 5 years, Humans. Look at the list of 76 very handy sources that will (in nearly all cases) be more appropriate than whatever primary sources are cited. Additionally, for that rather broad and popular topic, you should look at textbooks, which may be even better. WhatamIdoing (talk) 16:22, 4 April 2014 (UTC)

Yes agree with WAID. Finding secondary sources is easy. We have put a template on nearly every talk page to help people find these sources. Doc James (talk · contribs · email) (if I write on your page reply on mine) 07:15, 17 April 2014 (UTC)

That PSK article is a victim of the same misguided process which created Medicinal mushrooms. Sometimes coaching editors just leads to worsened behaviour. I've abandoned hope of solving that one.LeadSongDog come howl! 16:35, 17 April 2014 (UTC)

There's Medicinal molds too. Yes, they're among the articles I too have mentally assigned to the "beyond help" pile. Alexbrn ^{talk|contribs|COI} 22:20, 17 April 2014 (UTC)

"Primary sources should generally not be used for medical content"

I propose we add the following line to WP:MEDRS to clarify things for people. Doc James (talk · contribs · email) (if I write on your page reply on mine) 19:27, 27 February 2014 (UTC)

SUPPORT::I think it would cause a few problems (including exacerbating the ones in the section above to some extent), but would solve more than it caused.

I am a little new here to be weighing in on policy, but my impressions as a somewhat new guy and former industry guy are as follows:

• I see a lot of Admins and senior editors telling people that a secondary source is required for medical information, but when I read the MEDRS text as it currently stands, it only seems to recommend secondary sources, not require them. It would be good for the Admins and the guidance to be very clearly saying the same thing. At a minimum, pick one or the other but speak with one voice.

• Requiring secondary references as the source of all medical information would be a very powerful way of keeping a cap on those who want to use Wikipedia to WP:RGW or to "enlighten" the mis-informed mainstream experts. It might aggravate some of the issues I refer to in the section above, but I think it would be an overall improvement.

• On the other hand, as I noted above, I'm worried that the more or less blanket prohibition on results from animal studies and in vitro testing is not reflective of how toxicology is practiced in industry or in the regulatory world. I won't repeat what I wrote above, but I'd like to see the restrictions on toxicology results loosened in a few specific situations. The most important of these would be carcinogenicity and reproductive toxicology testing. Most of what we know about the carcinogenicity and repro tox of most substances is derived from animal studies and in vitro genotox studies. I'd like to see these data allowed provided they are taken from secondary references. Since case reports are frequently the basis for regulatory action in the case of rare but severe side effects, I think there should be some point at which they are allowed in, though the threshold should be high. Perhaps it is arbitrary, but the threshold I would suggest is that if a nationally competent regulatory authority takes action on the basis of AE case reports, the agency memo documenting that regulatory action and the reasoning behind it should be considered a MEDRS compliant source.

I've suggested to Jytdog that we contact the Society for Toxicology to find an expert interested in helping us set guidelines for toxicology reliable sources. I'd be interested to hear what people think of that. Formerly 98 (talk) 21:10, 27 February 2014 (UTC)

Positions of national bodies is deemed to be a high quality source. So are major medical textbooks. If these comment on the animal studies and invitro testing than so should we. (keeping in mind due weight of course) Doc James (talk · contribs · email) (if I write on your page reply on mine) 22:41, 27 February 2014 (UTC)

It would be nice to get this language clarified in the MEDRS document, as it currently seems to read "generally omit in vitro and animal data" or at least "don't imply that it is relevant in peoople". Overall the language appears non-clearcut, which may simply represent the fact that it is a compromise document with many fathers/mothers. For drugs, this information will usually be available from the FDA approval summary, available through the drugs@fda website. Formerly 98 (talk) 01:30, 28 February 2014 (UTC)

Yes agree that bit needs fixing too. If this data is emphasized in high quality secondary sources we should have it too. Much tox research is based on the precautionary principle and extrapolation from animals as the trials is people would be unethical. Doc James (talk · contribs · email) (if I write on your page reply on mine) 02:39, 28 February 2014 (UTC)

I think that the guideline is already sufficiently clear, in its Definitions, Basic advice, Use up-to-date evidence and Choosing sources sections, that primary sources generally should not be used for medical content, though it does note exceptions. I don't think that Jmh649 (Doc James)'s proposal is going to have the type of benefit that Formerly 98 is suggesting, but we can certainly try it. Flyer22 (talk) 21:56, 27 February 2014 (UTC)

SUPPORT. a lot. Jytdog (talk) 21:57, 27 February 2014 (UTC)

Jytdog, I think the reason some of us see a disconnect is that of the 4500 words of the MEDRS article, nearly 20% (and mainly in the first third of the guideline) describe when and how it is OK to use primary references. At a minimum its confusing. Maybe its just 'undue weight". : > ) Formerly 98 (talk) 22:26, 27 February 2014 (UTC)

i think you are right about that. there have been loads of loads of discussions on this Talk page about primary sources. and it will be hard to get consensus to ban them outright. Your comment about WEIGHT is awesome; I agree it should be toned down a lot. But there is no explicit sentence like the one that DocJames proposed and it would be great to get it in. It can be a separate proposal to reduce the weight on primary sources... maybe it would make sense to have a new section called "on primary sources" and consolidate and rationalize it. btw, with respect to what you wrote above, about tox... especially for tox we want only secondary sources. otherwise we open the door for every stupid paper where some basic researcher dumps a bunch of chemical right on cells and kills and them and goes "look look X is toxic!!" argh. so i deadly oppose (!) opening the door to primary sources for tox. i want it closed more tightly. the regulatory action itself becomes the source we cite, btw! Jytdog (talk) 22:36, 27 February 2014 (UTC)

Oppose - This was closed too fast. What about primary sources on human trials? Oppose as is. - Sidelight12 ^Talk 04:44, 5 March 2014 (UTC) *rationale 05:03, 5 March 2014 (UTC)

Yes primary source on human trials should not be used. Many primary human trials go unpublished as the results are negative. The positive ones are published. A proper review article will take into account both the published and unpublished results. Doc James (talk · contribs · email) (if I write on your page reply on mine) 04:51, 5 March 2014 (UTC)

Also the category of human trials can span an enormous range of trial design and execution quality and therefore result quality and statistical power. Zad68 05:09, 5 March 2014 (UTC)

This is a drastic proposed change, I think we should get more opinions. Certainly, abstract human primary studies, especially where there is open-ended interpretation, can be prohibited. I'm uncertain if everything falls under that category. - Sidelight12 ^Talk 05:57, 5 March 2014 (UTC)

It is not a drastic change. This is how this guideline is currently interpreted. This is simply clarifying how most people interpret the guideline right now. Doc James (talk · contribs · email) (if I write on your page reply on mine) 06:10, 5 March 2014 (UTC)

Hey User:Sidelight12 - the spirit of MEDRS (and wikipedia in general) is that we provide content that is reliable. As MEDRS already says, in the health arena publications get released and the press goes gaga quite often; we need to rely on secondary sources to tell us how big of a deal the work and conclusions in the primary source are - we don't decide ourselves nor do we follow popular media. We take the long view guided by reviews. So, so important and fundamental. I generally find that when people are pushing content based on primary sources into articles, they are grinding an ax and/or treating wikipedia like it is "science news" instead of an encyclopedia; neither are good. Jytdog (talk) 15:01, 5 March 2014 (UTC)

It is not drastic, it's a plain-language restatement of the current consensus regarding the application of WP:MEDRS. Sidelight removed the sentence, the sentence should be restored to the guideline. Zad68 15:13, 5 March 2014 (UTC)

Support this and even stronger wording if support can be developed for it. Zad68 05:07, 5 March 2014 (UTC)

Support as this is basically codifying what we already do with regards to use of primary sources Yobol (talk) 15:40, 5 March 2014 (UTC)

It has been open a week with substantial support. Thus added the text in question again. Doc James (talk · contribs · email) (if I write on your page reply on mine) 18:40, 5 March 2014 (UTC)

• Three more supports here: [9] Doc James (talk · contribs · email) (if I write on your page reply on mine) 23:24, 5 March 2014 (UTC)

• note: It's been less than a week, and quantification of support needs to be expressed here. Take into consideration RexxS's [10] and Formerly 98's comments, and human tests that aren't based on abstract measurements. Until these are addressed, I oppose it. - Sidelight12 ^Talk 04:52, 6 March 2014 (UTC)

• Support. This looks like a straightforward clarification. Axl ¤ [Talk] 00:27, 6 March 2014 (UTC)
• Support. This is a reformulation of what WP:RS already says. It has very broad support from current medical editors, and is an absolutely vital defense in the battle against poor quality hyped-up content. JFW | T@lk 12:58, 7 March 2014 (UTC)

• Support. But I think we should italicise, "generally". Like, for instance, a number of natural supplements are not sufficiently well-studied for secondary sources to even exist and hence I think primary sources should be preferred over no sources in these cases. Fuse809 (talk) 13:43, 7 March 2014 (UTC)

sorry but i don't agree with what you write here; if there are no reliable sources as defined in MEDRS, Wikipedia should say nothing. Primary sources are not reliable for health content - that is the whole point of this. Jytdog (talk) 13:47, 7 March 2014 (UTC)

I would say "less likely to be reliable" rather than "not reliable". There are certainly many primary sources that are reliable. The converse implication, "secondary sources are reliable", does not hold up to scrutiny either. Actually my main concern about primary sources is misuse to support minority/fringe viewpoints, especially by POV/COI pushers. Axl ¤ [Talk] 16:11, 7 March 2014 (UTC)

You're right! I have a good example of how a review article can be plain out wrong, even in its title! It's one I was trusting blindly, almost like the Germans of Nazi Germany. The article is: PMID: 15361747. I would much rather have some info about a herb than none. Like if all we have on a page about the therapeutic uses of a herb is just what's supported by secondary sources it's going to be VERY short for most herbs. Fuse809 (talk) 06:31, 17 April 2014 (UTC)

... which would be an excellent outcome for Wikipedia, in the absence of high-quality sources. (BTW, that article you mention has a title that's a question, so I'm not sure how it could be "wrong".) Alexbrn ^{talk|contribs|COI} 06:42, 17 April 2014 (UTC)

Jytdog, I think what Fuse809 means by "a number of natural supplements are not sufficiently well-studied for secondary sources to even exist" is what the Use up-to-date evidence section currently states: "These instructions are appropriate for actively researched areas with many primary sources and several reviews and may need to be relaxed in areas where little progress is being made or few reviews are being published." After all, WP:MEDRS does not ban primary sources; it makes exceptions (like I noted earlier in this discussion), which is why the word generally is used in this case. Flyer22 (talk) 06:49, 17 April 2014 (UTC)

Hi Flyer22, I am not sure what point you are making. Wikipedia policies say over and over that we rely primarily on secondary sources, and like most good policies, many good things flow from that, and many bad things are prevented. This is even more so with health related topics, since what we actually know about those things is so foggy; the consensus as expressed in 2ndary sources is our best compass to identify what is reliable and what is not. If there are no reviews on a given topic, but just a smattering of papers (X kills cancer cells; Y kills frog embryos), those sources should not be used to generate content -- all we would be doing is leading readers down a garden path; we would not be providing them with reliable information, which is the mission of Wikipedia. Jytdog (talk) 11:09, 17 April 2014 (UTC)

Jytdog, I'm not sure what else to state on the primary sources matter other than what I stated above. The Use up-to-date evidence section does not state "no reviews"; it states "areas where little progress is being made or few reviews are being published." That matter is certainly the case for some medical issues, including mental health issues (such as some paraphilias). As another example, I've also found it to be the case with the persistent genital arousal disorder topic; or maybe there is a lot more research and review articles on that topic these days? Flyer22 (talk) 20:01, 17 April 2014 (UTC)

well you just made me laugh for picking something so outre!  :) We seem to have different stances on this, so i guess we'd have to work that out if we ever bumped into one another on a given piece of content. I really do hear you that if there are no reviews, you would be Ok with citing a primary source. I say, if there are no reviews, we should say nothing. To me, the importance of providing reliable information is the key thing. I hear you, that would rather say something. The age old Deletionist vs inclusionist, I guess. Jytdog (talk) 20:37, 17 April 2014 (UTC)

Well, I generally agree with you. I feel that if there are only primary sources, we likely (not necessarily) should not be reporting the matter. I'm simply pointing out that like, WP:MEDRS does, there are areas of medicine where there are not an abundance of reviews and it's far more of a primary source matter. Not "only a primary source" matter.

As for "outre!," LOL. On that note, what do you make of the postorgasmic illness syndrome topic with regard to primary sources and non-primary sources? I and fellow WP:MED editors Jmh649 (Doc James) and Lesion recently came across that article; I haven't edited it yet (except for this revert that I don't remember: I revert vandalism and other non-constructive edits a lot), but I have commented at its talk page. Flyer22 (talk) 21:55, 17 April 2014 (UTC)

Jytdog, regarding this recent edit you made, I rather like the "Reliable primary sources may occasionally be used with care as an adjunct to the secondary literature" piece because it clearly indicates that citing a primary source alongside the secondary source is fine. Sometimes I like to cite the original source (primary source) along with the secondary source so that readers have quick/full access to the original source. I know that you made that edit to emphasize the "unreliablility of primary sources and resolving contradictions)," but did/do you feel that this line was contradictory to "Primary sources should generally not be used for medical content."? Flyer22 (talk) 20:23, 18 April 2014 (UTC)

for what it is worth, i think that if anyone cites a seminal primary source as you describe - in a way that is truly adjunct to a secondary source upon which the content relies - nobody would disagree with that. Do you think anybody would? I do think that in the text as it stood, people completely blew past "adjunct", and all the other warnings and subtle boundaries we have tried to build in. Jytdog (talk) 20:33, 18 April 2014 (UTC)

Well, I know that Doc James would rather that a primary source not be used even in that case. Maybe you wouldn't either? The adjunct wording wouldn't change his (or your) view on that, but I think it helps clarify why the primary source is there next to the secondary source. Thanks for adding the adjunct aspect back, but I feel that you made it a bit strict; for example, it's not always a historically important paper (depending on what is meant by that), or often that a secondary source will call it such even if it is, and there is also the fact that a secondary source may not mention the original source but supports the same material as the original source. But on that latter point, if a primary study is used to source a matter, and the secondary source supports the same content without mentioning that primary study, I guess it's best to simply cite the secondary source and not the primary source that also happens to support the same conclusion. Flyer22 (talk) 22:57, 18 April 2014 (UTC)

Depending on the subject, you might be able to list a seminal paper as WP:FURTHERREADING. WhatamIdoing (talk) 00:31, 19 April 2014 (UTC)

Jytdog, any proposal so that we are both satisfied with your adjunct wording? As it stands, your "if it [is] described as such by the secondary source" wording is unrealistic to me because it is not often that a secondary source describes the original source as a "historically important paper" (even if you mean wording similar to "historically important paper" as well). Your wording is also strict because it is putting a limit on simply citing a primary source alongside a secondary source. Flyer22 (talk) 00:28, 20 April 2014 (UTC)

that language, about "if it [is] described as such by the secondary source" has been in MEDRS a long time and others have complained about it too. so I just took it out; i meant to do that in my bold initial edit and failed to. i've read your comment a couple of times and not sure what else you would like to see changed... can you maybe propose something concrete? i am abit uncomfortable being so bold to change this important guideline but the changes seem to be sticking so far... but let's keep trying! thx Jytdog (talk) 00:49, 20 April 2014 (UTC)

I'm not sure what you mean by "has been in MEDRS a long time." The "if it [is] described as such by the secondary source" wording with regard to the "adjunct to a secondary source" bit was not there until you added it there yesterday. And I'm certain that the reason it was not there is for the reasons I stated above. How often do you see a secondary source describe the original source as a "historically important paper"? I certainly don't see that with regard to the vast majority of secondary medical sources. Which is perhaps why you added it? You want it to be so rare that a primary source can be used for health information, even in the case of simply providing the primary source "along with the secondary source so that readers have quick/full access to the original source," something you agreed with by stating "for what it is worth, i think that if anyone cites a seminal primary source as you describe - in a way that is truly adjunct to a secondary source upon which the content relies - nobody would disagree with that."? Flyer22 (talk) 01:05, 20 April 2014 (UTC)

Flyer, sorry I was got my wires crossed; the language I added was nodding to the language that has been in MEDRS a long time, namely: "Primary sources should not be cited in support of a conclusion that is not clearly made by the authors or by reliable secondary sources, as defined above (see: Wikipedia:No original research)." I now see what you meant and fixed it. Sorry for the misunderstanding. Jytdog (talk) 04:05, 20 April 2014 (UTC)

No problem. And thanks for this. If you feel that we should explain why someone would want to cite a primary source alongside a secondary source, we could add your "to provide readers with easy access" part back, but alter it so that it isn't as strict as the previous wording and instead states "to provide readers with easy or full access to the original source"; I was going to suggest "for example" be in that line, so that it reads as: "to, for example, provide readers with easy or full access to the original source." But I can't see any other reason than those two for why a person would want to cite the primary source alongside the secondary source. And if you're worried about overuse of this option, I think that WP:MEDRS is now clear that primary sources generally should not be used for health content; so there generally should be nothing to worry about there. If you are fine with the adjunct text as it is, just know that I am as well. Flyer22 (talk) 04:33, 20 April 2014 (UTC)

ok! :) let's see if these changes stand... so far so good. Jytdog (talk) 10:33, 20 April 2014 (UTC)

Requesting guidance on the use of primary source

Richf52 posted the following edit request to Talk:Cannabis (drug), asking that mention of a new study in The Journal of Neuroscience showing that marijuana use is associated with certain brain abnormalities be added to Cannabis (drug)#Neurological.

Please add this in the existing Neurological effects section. The news of this study just came out today and there are 100's of news links on the web about his study. It is a valid source and complies with WP:RS:

A study published in The Journal of Neuroscience says even casual marijuana smokers showed significant abnormalities in two vital brain regions important in motivation and emotion. According to co-author Hans Breiter, "Some of these people only used marijuana to get high once or twice a week." This is the first study to analyze the effects of light marijuana use.

The reference is: (#2 here)

I made the edit, which Jmh649 reverted with the edit comment, "Please read WP:MEDRS." On my talk page, Jmh649 wrote, "Please read WP:MEDRS. We do not use the popular press as a medical source."

I agreed that was a valid objection, so I rewrote Richf52's requested edit as follows to cite the actual study and describe it using language lifted almost verbatim from the abstract, hoping to avoid complaints that I had provided my own interpretation.

A collaborative study between Northwestern Medicine and Massachusetts General Hospital/Harvard Medical School of young adult recreational marijuana users published in The Journal of Neuroscience showed significant abnormalities in emotion and motivation and that the degree of abnormalities in these regions is directly related to the number of joints a person smoked per week. Researchers collected high-resolution MRI scans on young adult recreational marijuana users and nonusing controls and conducted three independent analyses of morphometry in these structures: (1) gray matter density, (2) volume (total brain and regional volumes), and (3) shape (surface morphometry). Gray matter density analyses revealed greater gray matter density in marijuana users than in control participants in the left nucleus accumbens extending to subcallosal cortex, hypothalamus, sublenticular extended amygdala, and left amygdala, even after controlling for age, sex, alcohol use, and cigarette smoking. Trend-level effects were observed for a volume increase in the left nucleus accumbens only. Significant shape differences were detected in the left nucleus accumbens and right amygdala. The left nucleus accumbens showed salient exposure-dependent alterations across all three measures and an altered multimodal relationship across measures in the marijuana group. These data suggest that marijuana exposure, even in young recreational users, is associated with exposure-dependent alterations of the neural matrix of core reward structures. This is the first study to show casual use of marijuana is related to major brain changes. According to co-author Hans Breiter, "This study raises a strong challenge to the idea that casual marijuana use isn't associated with bad consequences."

Jmh649 reverted it with the edit comment, "Please use "secondary" rather than "primary sources" secondary sources are "review articles" and major textbooks." I reverted him with the edit comment, "According to WP:MEDRS, "Reliable primary sources may occasionally be used with care"." Alexbrn reverted me with the edit comment, "rm content sourced to primaries, used with insufficient care". This is where the article now stands.

In a long, fruitless discussion at Talk:Cannabis (drug)#Semi-protected edit request on 16 April 2014, I have been attempting to understand what Alexbrn thought was insufficient about my care, promising that if the shortcoming could be identified, I would do whatever he wanted to fix it. Alexbrn has answered, ""Sufficient care" would have been shown by taking account of the context: the amount of active research in this area, the existence (or not) of corroborating sources, etc." But this is neither testable nor actionable nor even, so far as I can tell, supported by the guidelines.

It appears to me that the revised edit complies with WP:MEDRS but realistically, I'm not a doc and I don't even play one on TV. This is outside my expertise. It the revised edit doesn't comply and there's a way to fix it, I would like to do that, if only someone is willing to tell me what must be done. If indeed it's just not usable at all, I would appreciate help understanding why so that I don't make the same mistake again. Respectfully, I am requesting guidance. Msnicki (talk) 07:22, 18 April 2014 (UTC)

It is unclear to me, what you are confused about - you are clearly committed to including this content, and are ignoring what people are telling you on the Talk page. I will say the same thing again, that they have been saying to you: all of Wikipedia's sourcing policies overwhelmingly call us to rely on secondary sources - there are many good things that flow from that and many bad things that are avoided. Sure, primary sources ~may~ be used, but you need damn good reasons for doing so. I reviewed the talk page and at no point did you even engage with this. So - What is exceptionally important about this study? (the fact that it is new, is not relevant - WP is not a newspaper) Jytdog (talk) 11:19, 18 April 2014 (UTC)

We allow the use of primary sources all the time. Per WP:GNG, we do require reliable independent secondary sources to establish notability of a topic. But once notability is established, reliable primary sources are generally acceptable. From WP:PRIMARY, "primary sources that have been reliably published may be used in Wikipedia; but only with care, because it is easy to misuse them." The misuse concern is explained as "Exceptional claims require exceptional sources" (linked to WP:REDFLAG) and that "All interpretive claims, analyses, or synthetic claims about primary sources must be referenced to a secondary source, rather than to an original analysis of the primary-source material by Wikipedia editors." WP:REDFLAG speaks to reliability of the source, e.g., fringe theories, self-published sources with an apparent COI, etc. But I haven't offered my own interpretation of source and I find it inconceivable that most people would regard a refereed paper a major medical journal on a collaborative study by researchers at Northwestern, Harvard and Mass General as fringe theory, an SPS with apparent COI or otherwise unreliable.

Your claim that I have been "ignoring" what others were telling me is simply revisionist and untrue and overlooks the fact the debate is split 2 and 2. Perhaps they are ignoring us. I know I have not been ignoring them. Alexbrn reverted with an edit comment claiming that I had used the source without "sufficient care". I've merely been trying to understand, what does that mean? What would constitute sufficient care? How could anyone tell the difference? I suspect the phrase is meaningless, that there simply is no level of care that Alexbrn would accept as sufficient to allow use of a primary source in a medical journal and it sounds like that's your view as well. But if that's the case, it would be helpful if you and Alexbrn could merely state that as your position, not make me guess. More to the point, if the guidelines said you can't use a primary source from a medical journal without a secondary source also from a medical journal, I would follow it, no question. But it just doesn't say that.

As I understand the lede paragraphs in WP:MEDRS, the concern is that many people may look to WP for medical information and that they will make decisions based on that. That by itself isn't special. People look to WP for all kinds of information and make all kinds of decisions based on what they find here. Implied is that there's a difference when it's medical information, that there is a heightened risk that harm could result from decisions based on medical misinformation. But what is the risk here and how much care is required? This study suggests smoking dope might not be a good idea for young people. Does anyone seriously believe there is a risk of harm that someone might stop using marijuana when they really didn't need to? What would that harm be, that they had missed out on so many delightful hours they could have spent high?

Except for the fact that I invested some time in attempting to help on a requested edit, I don't actually care so much whether the edit goes in or not. This wasn't even my edit, it was merely a requested edit I made because I thought I was being helpful and which I tried to fix when objections were raised. Whether they're known or not, it seems to me that only an idiot would not realize that recreational drugs that get you high probably carry risks, even if presently unknown, that they could do something to your brain that might not be helpful. My frustration is that I'm having difficulty understanding the guidelines, which I would like to follow not just in this case, but any new ones I may later encounter. Telling me I should just defer to Alexbrn and Jmh649 is unhelpful as guidance for the future. Msnicki (talk) 18:34, 18 April 2014 (UTC)

You came here asking for more input which you again ignored and simply repeated the same arguments you have been making. Should not have bothered responding. Phooey. All you ask are rhetorical questions. Rhetorical questions suck. I don't believe you came here asking anything but just came here to forum shop. prove me wrong and ask a real question. Jytdog (talk) 19:01, 18 April 2014 (UTC)

Once again, absolutely zero helpful content addressing my questions, only a lot of gratuitous disrespect and questioning of my good faith. Phooey on you, too. Msnicki (talk) 19:08, 18 April 2014 (UTC)

Msnicki, I realize that this is an odd area for sourcing. Normally, if you had a hundred newspapers running an article about how Joe Filmstar was arrested for drunk driving, you'd be able to get that information past BLP. So why isn't the same level of media attention about a scientific study good enough to merit inclusion of information about the study? Mostly it's because MEDRS was written with the WP:DEADLINE firmly in mind, and a keen awareness that the study reported on Tuesday may be contradicted by another on Thursday, or that it may be the only one, out of dozens, that came to this conclusion, and the only reason that this one made it into the popular press is because the researcher has a more aggressive publicity department.

On the basis of this study, I think you'd be justified in removing any contradictory statements. For example, given the existence of this study, we shouldn't have any statements that say "There have never been any studies showing that occasional use affects the brain". But until it's been confirmed or reported in reviews, then its inclusion may not be really justified. I suggest that you leave information about it on relevant talk pages, and then come back in a couple of months. Studies that make a big media splash usually turn up in an official review article only a few months later, and then you can cite the review. WhatamIdoing (talk) 16:54, 18 April 2014 (UTC)

This is a far, far more helpful than Jytdog's near trouting of me for not realizing that if Jmh649 and Alexbrn don't want something in WP, it's out, period. Still, I would appreciate if you could flesh out more clearly how one could decide, based on WP:MEDRS, when primary sources are acceptable and what constitutes acceptable care in their use, especially in a case like this, where the source is an article in a major peer-reviewed journal, the authors are all qualified researchers at institutions with impeccable reputations, the claims don't appear to be exceptional or fringe and there seems to be essentially zero risk of harm that someone might stop their recreational use of marijuana when they didn't have to. Msnicki (talk) 18:34, 18 April 2014 (UTC)

I am nobody's follower. What I was trying to explain to you, is that WP overwhelmingly favors 2ndary sourcing - the burden is on you to say why a primary source should come in. As WAID hints at, in the biomedical/health related space, primary sources are even more unreliable than they are in other fields. For a popular example -- one day carbs are great for a diet, the next day they are terrible -- primary sources say this then that, the popular press follows them around, and the public feels yanked around. Caffeine is great for your brain, caffeine is horrible for your brain, on and on. We don't do that in health related articles in WP. We wait to see what the consensus is, as per high quality review articles. Again, to bring a primary source for something health-related, you need to make a case that this is exceptionally important.Jytdog (talk) 19:06, 18 April 2014 (UTC)

On reflection, WhatamIdoing, I've decided you've given me exactly the answer I need. I like the very pragmatic advice you've given, pointing out that because it's in the news, it will likely get reviewed soon. (I'm an engineer and it doesn't work that way for us, so I don't come here knowing this.) I also like your point that one acceptable use right now would be to remove contradictory material and that this is a nice bright-line distinction in care. I'm not being coy when I say I don't really care if it goes in, I wanted to understand the guidelines. Once I rewrote it with the gibberish from the abstract rather than the plain English used in the Washington Post, I really didn't like it that much anyway and felt more like a good soldier who'd promised to do a favor only to find out there was more to it than I'd expected. I'm totally satisfied that it's out for now but will probably be a good candidate for inclusion in a few months.

But also, you've given me a pragmatic insight that answers the hypothetical, suppose there was a similarly wonderful article by similarly wonderful researchers at similarly wonderful institutions that wasn't in the news and thus, wasn't perhaps so likely to be reviewed soon. It would still be useful for challenging other sources and, more to the point, I draw from your comments that that's a case doesn't happen because those wonderful institutions have PR departments. To an engineer, that's a very appealing problem simplification. Thank you very much for your very kind and very helpful answer. Have a great weekend. Msnicki (talk) 19:50, 18 April 2014 (UTC)

I am glad you found WAID's explanation useful - sugar does go farther than vinegar. Quick note on your last paragraph - MEDRS says "Individual primary sources should not be cited or juxtaposed so as to "debunk" or contradict the conclusions of reliable secondary sources" - so no, a primary source would not "be useful for challenging other sources" in the biomedical/health space. Jytdog (talk) 20:04, 18 April 2014 (UTC)

Here's what I suggest, Jytdog. Flesh out some clear guidance in WP:MEDRS that reasonably smart but otherwise completely ordinary lay people can understand delineating appropriate versus inappropriate use of primary sources and explaining what "care" means in the special context of medical content. It needs to be testable and actionable. You need examples. It's just not there right now in any form the rest of us can understand. Compare the helpful answer I got from WhatamIdoning with the unhelpful content in this guideline page. It's not about bolding stuff, like a tourist who speaks louder hoping to overcome a language barrier, and it should definitely not be about questioning other editors' motives. It should be about helping the rest of us understand how to reason through the question of whether and how a medical source can used. WP:AGF that if someone tells you they don't understand your guideline, they're telling you the truth, they don't understand it and the problem really could be that the guideline isn't well-written, not that that individual is stupid, ill-intentioned or just won't listen. Msnicki (talk) 20:38, 18 April 2014 (UTC)

Primary sources should not normally be used to "debunk" a secondary source, but it would be unfortunate for a Wikipedia article to continue saying "Nobody's ever done this (as of 2010)" when we know perfectly well that someone did it in 2014. It is often best to simply remove the demonstrably outdated information than to "debunk" or "contradict" the older secondary source in the text, but we shouldn't keep information that anyone can see is outdated. WhatamIdoing (talk) 01:09, 19 April 2014 (UTC)

MEDRS fails again?

As an aside, this is yet another example, relating to the section above, of a fairly experienced editor having read WP:MEDRS and failing to "get" that primary sources should generally not be used. If that's the message WP:MEDRS is trying to convey, it's signally failing to be good at doing it. The problem I think are provisions couched in the language of permission rather than prohibition. MEDRS says "Reliable primary sources may occasionally be used ..." but it would be better to say "Primary sources should not be used other than in exceptional circumstances, for example when ... ". Alexbrn ^{talk|contribs|COI} 11:41, 18 April 2014 (UTC)

What we need is a FAQ attached to this page that lays out common scenarios when primary sources generally should be used, and when they should not. This comes up waaaaaay to many times to be spending this much effort on this when we should be able to direct someone to a page discussing it. Yobol (talk) 15:05, 18 April 2014 (UTC)

The FAQ will not help if the guideline continues to give contradictory advice. Right now, I certainly get the impression that primary sources are okay as long as one uses them with care. In particular, "primary sources should not be cited in support of a conclusion that is not clearly made by the authors or by reliable secondary sources". It's actually not very clear to me what this sentence is trying to achieve. So... you mean that unlike secondary sources, which I may freely use to back up my own arbitrary conclusions, we should, in the very special case of primary sources, actually stick to the source? Anyway, it would also help if you replaced every instance of "secondary source" in this guideline with "high-quality systematic review". Newspapers are reliable secondary sources in normal wiki-jargon! Vesal (talk) 17:12, 18 April 2014 (UTC)

Yes! Thank you. This is exactly why I'm having trouble. If indeed the answer is that primary medical sources should never be used, period, it would be helpful if the guideline just came out and said that. Right now, it doesn't. Holding out the possibility they could be used with care if in fact there is no amount of care that would be considered acceptable wastes everyone's time. Msnicki (talk) 18:34, 18 April 2014 (UTC)

Agree that the current wording needs adjustment to match the current practice of avoiding primary sources. We do not have a blanket prohibition of primary sources because there are rare circumstances where they are permitted. This is why I think a FAQ would be useful in providing that information for when they are permitted. Yobol (talk) 19:36, 18 April 2014 (UTC)

Guidance has been updated and will hopefully decrease confusion. Doc James (talk · contribs · email) (if I write on your page reply on mine) 20:55, 18 April 2014 (UTC)

Respectfully, you are not done. I'm okay for now but please take another look at my suggestions. I don't think they've all been done yet. Msnicki (talk) 23:07, 18 April 2014 (UTC)

Let's see what we can come up with. Naturally, in each case, you're going to have to use judgment and consider the alternatives. Speaking very generally, though, I think there's pretty widespread support for these cases:

1. Historically important papers, like a century-old, first-ever case report for a disease.
2. Major reports on notable experimental drugs. For example, you might cite results from a Phase II study for a potential future drug, if there's nothing better available. (We try to avoid having articles about these at all, but if the article exists, then primary research from a peer-reviewed journal is better than a WP:PRIMARYNEWS source.)
3. What I'll call "minor details" that have nothing directly to do with human health, if no secondary source exists. For example, you might cite a rodent study to support an LD50 claim for a small molecule drug.
4. Results from a WP:Notable study, if it seems relevant enough to call the trial out by name in the article. (Here I'm thinking of things like the Million Women Study or the Framingham Heart Study.)
5. Just about anything for really rare diseases. My favorite example here is ODDD, which affects about a hundred people in the entire world. For some rare conditions, you really have to choose whether you want "up to date" (meaning anything in the last five or ten years) and "secondary", because you often cannot have both.

What else would other people add? WhatamIdoing (talk) 01:04, 19 April 2014 (UTC)

1. Historically important papers are reliable sources for the date of discovery and the identification of the discoverers, but if the discovery was important, the medical subject matter can be sourced from secondary sources.
2. I think you'd be surprised at how well developmental drugs are covered in review articles. It may take a few months after the trial results are announced, but I don't see a big problem with that. I might make an exception for pivotal phase 3 trials as temporary bridges unit the reviews come out, as these are difficult to "cherry pick" as there are so few of them.
3. Most, but perhaps not all of these "minor details" can be found within the official prescribing information or the FDA approval summaries. The latter are available for most drugs approved within the last decade or so at the drugs@fda website.
4. Notable studies tend to be extremely well covered in reviews. There are 56 articles with "Women's Health Initiative" in the title alone in pubmed.
5. I'm more inclined to agree with you on the rare diseases than on some of the other points.

Thanks for raising some interesting issues. Formerly 98 (talk) 02:45, 19 April 2014 (UTC)

1. If you check the archives, you'll find that I opposed citing "historically important papers" (these occasionally appear in the ==History== section), but pretty much everyone else likes citing them there. Obviously, they're not "up to date", so they aren't used for biomedical information.
2. Some editors might be surprised, but I wouldn't. In fact, what would surprise me is if the larger pharma companies haven't looked over MEDRS and adjusted their publication strategy to suit. But having said that, some drugs are well covered in reviews, and some are not. Some are only covered in reviews behind paywalls. I just checked a drug that I've heard about in trials for years—I believe it's been in clinical trials for about 20 years now, and is currently Phase 2 or 3—and there is, in fact, a review article about it. Specifically, there is exactly one review article, written by the PI, and it mentions this drug in just eight sentences. It may not be ideal, but people do seem to accept primary sources for experimental drugs when nothing better is available.
3. Official prescribing information is not an independent source and is sometimes also considered a primary source.
4. Notable studies tend to be well covered in reviews, but not as soon as major events are announced. When a major trial gets stopped early, as the Women's Health Initiative did over breast cancer, we should consider including that fact while we wait for the review cycle to catch up. And really, as MEDRS says, you want two or three reviews, not just one, because the first review is often written by someone with a conflict of interest or other biases. Using WHI as an example, the HRT trial stopped at the beginning of July 2002. The first review came out the next month, written by the WHI staff. The first independent review appeared in October 2003. It would have been silly not to include the (non-biomedical) fact that the trial was stopped for over a year, including the (biomedical) reason given for choosing to stop it, while we waited for an independent review. WhatamIdoing (talk) 18:26, 21 April 2014 (UTC)

WhatamIdoing - I am surprised by your #3; can you please say more about how you see labels as not "independent" in they way we use that term in WP? And why they would be considered primary by anybody? (as you know they are strongly governed by law and regulation and cannot see the light of day without approval from regulators... a harsher peer review than any journal would give an article, imo) — Preceding unsigned comment added by Jytdog (talk • contribs) 19:13, 21 April 2014 (UTC)

Hi Jytdog,

Prescribing information is written by the manufacturer. It is based on the information that they submit and it is the product of their negotiations. It is therefore not truly independent, although, due to regulatory requirements, better than some other non-independent sources (e.g., the website of anyone selling whatever herbal supplement is popular this week).

The main quality of a secondary source is that it provides an analysis that is based on other publications. Prescribing information isn't exactly known for analysis. It's a pretty bare list: Here are the only indications that we have legal approval for. Here is a laundry list of side effects that our lawyers negotiated with the regulatory agency. Because of that, some people prefer to use it as a primary source. Others don't. Personally, I do both. The line between primary and secondary can be fuzzy, and even true experts will disagree in some cases. It seems likely to me that this will be one of those permanently unclear cases. WhatamIdoing (talk) 03:45, 22 April 2014 (UTC)

Thanks for responding. I do know that labels are negotiated by drug sponsors and regulators; i was interested in how you would apply the "independent" standard. i think you are maybe more cynical than me about corporate science and the good faith efforts of regulators, and i am maybe more cynical than you about academics... but i hear you and i appreciate your flexible approach. thanks again. Jytdog (talk) 04:10, 22 April 2014 (UTC)

So basically we'll have to wait 8 years until some other "review" that can qualify as a secondary source mentions this study. Until then 1000's and millions of readers will not be informed about this ground breaking study even though it has passed all other requirements for a WP:RS (which says secondary sources are preferred but not the only ones that can be used). To me this doesn't make any sense and making readers miss out on important information (that could prevent them from being brain damaged) is a tragedy. There are serious consequences of preventing important information from reaching the masses.

I dont see why WP:MEDRS should have some exclusive policies that are different from WP:RS which is a core guideline and it allows primary sources. The best option is to allow primary sources but in a section of an article called "Primary Sources" (etc) otherwise there's the 8 years/millions of readers issue I mentioned but it looks like people dont care about that and are not willing to consider other options and I'm not going to pursue this much longer. You guys can do what you want. Thanks to Msnicki for trying to pass a perfectly valid source with 100's of mentions in the news but according to others, it doesn't qualify as a source. --Richf52 (talk) 03:22, 20 April 2014 (UTC)

You seem to be assuming both that this study is valid and that competent medical authorities won't recognize this and incorporate the material into reviews in a timely way. This reasoning is exactly what MEDRS is here to protect against. If its valid, and it truly is "groundbreaking", the mention in secondary reviews will come within weeks or at most months. If that doesn't happen, its probably a sign that experts don't agree with your interpretation. Formerly 98 (talk) 12:59, 20 April 2014 (UTC)

Richf52 The turnaround time in medical publishing is now so rapid that a major discovery will be mentioned in a good secondary source within weeks or months. The policy creates exceptions for situations where a study or result is so important as to warrant inclusion. On the whole, this is unusual, but all you need is WP:V and WP:CONS. You will have noticed that "mentions in the news" is absolutely no guarantee that a result is of any long-term relevance. The story of neutrinos exceeding the speed of light got loads of headlines, and turned out to be based on an erroneous result.[11] As an encyclopedia we certainly have a duty to be up-to-date, but reliability is probably more of a core value. JFW | T@lk 15:16, 20 April 2014 (UTC)

Richf52 you are assuming that the study is valid and will stand the test of time. If you read MEDRS, you'll see that it describes how only ~25% of even high-profile papers turn out to be replicable. Just because the 24-hour news cycle and universities' need for donors leads to all kinds of hype, it doesn't mean that any given paper is important or even that its conclusions are valid. Let's see, should I drink coffee or not? Maybe I will live longer and drive safer, and hey if I am woman maybe i will be less depressed, but oh no! it alters my estrogen levels and maybe it will screw up my baby. Every one of those links is from the New York Times, is just from the past couple of years. We don't jerk readers around like this - we are an encyclopedia, not a newspaper; we provide reliable information to readers about health related matters, and for reliable information, we as editors rely on reviews and statements by major medical and scientific bodies. Jytdog (talk) 16:09, 20 April 2014 (UTC)

I'm also confident that this particular study will be included in review studies as soon as the publishers can manage to process it. In fact, I'd be surprised if that review wasn't already written. Researchers know what their colleagues are working on, and pre-press papers often make their way into the hands of someone who's working on an upcoming review. WhatamIdoing (talk) 18:34, 21 April 2014 (UTC)

So according to the people here who have wanted the removal of that reference at Cannabis (drug): if a reference is only using the cite journal template (and no review or secondary sources), it is a primary source and therefore should be deleted. Correct? There are 350,000 pages using the template and I'm sure many of the references are only journal references with no reviews or secondary sources. A bot should be set up to remove all such references from all of Wikipedia, and new insertions of Cite journal should be flagged from review. Correct? I will be removing references to journals whenever I come across them. Here's one that I just removed (this is not WP:POINT, I had to read the article to look up something so I came across it). That journal has 7 other references which I havent removed yet but before I do that I want to see what others think. I did the right thing, right?

So are people going to be fair and apply these standards to all articles or are these rules just for Cannabis (drug)? --Richf52 (talk) 02:46, 22 April 2014 (UTC)

What? The Cite Journal template is commonly used to cite secondary sources. Alexbrn ^{talk|contribs|COI} 03:26, 22 April 2014 (UTC)

(edit conflict)

Hi Richf52,

We use {{cite journal}} for all kinds of things, including citing review articles. It works for magazines and other periodicals, too, by the way; {{cite magazine}} is exactly the same thing. The method of typing out the citation is irrelevant. We want most biomedical information to be sourced to a secondary source. That could be a review article, a good textbook, or several other kinds of sources. Note, too, that review articles are published in medical journals. What you removed is PMID 23032936, which is a review article (a secondary source). If you click on the PMID link I just gave you, and then on the "Publication Types, MeSH Terms" item, then you will see that it is labeled as being a review. If your reason for removing it was that you believed it to be a primary source, then you should revert your mistake right away. Also, based on your edit summary, you need to read WP:PAYWALL. Requiring registration or payment to see the source is perfectly acceptable.

You might find it helpful to read Wikipedia:Identifying and using primary and secondary sources. WhatamIdoing (talk) 03:31, 22 April 2014 (UTC)

So how do we know whether a certain Cite journal reference is a primary source or a review article/secondary source? The person who has to insert Cite journal has the duty of proving that its not a primary source. Correct?

The number you gave "23032936" does not match what is here although the one I removed does have the "review" tag.

What about this one? [12], [13], [14], [15] and [16] and others? They dont have any "review" types there so they should all be removed from Onychomycosis, correct? --Richf52 (talk) 03:38, 22 April 2014 (UTC)

The other source that you removed, and what I linked with PMID (a WP:Magic word; just type PMID followed by the number) is https://en.wikipedia.org/wiki/Onychomycosis#cite_note-23 which is a review.

The way you know whether something is a review—with certainty—is to know how to identify reviews by reading them. But the 99.44%-reliable shortcut is to go look them up on PubMed and see how (and whether) the paper is categorized by them. WhatamIdoing (talk) 04:02, 22 April 2014 (UTC)

Yes, those are primary sources. (I'm just going to assume that there are no good reasons to have primary sources in that article: common condition, lots of reviews and textbooks out there, etc.) However, immediate removal is not always the ideal solution. Ideally, anyone who is concerned about it would first see whether it's possible to replace the inappropriate primary sources with decent secondary ones. It's often possible (especially for common conditions). If you don't want to bother with that (it is a bit of work, and we can't assume that everyone has hours and hours of time to clean up sourcing), and always assuming that the material doesn't seem dubious to you, then we often add {{primary-inline}} to tag them for a while (weeks or months). That helps direct attention to the need, in the hope that someone who cares about the article will take the time to replace the sources. WhatamIdoing (talk) 04:08, 22 April 2014 (UTC)

Urine therapy and Urophagia

Just reverted an editor who added, amongst other material that "This age old natural therapy can prevent and cure cancer, AIDS, Renal failure, gall bladder stones, cerebral palsy" with some non-RS sources. Can someone here more familiar with the subject counsel him? Thanks. Dougweller (talk) 17:56, 26 April 2014 (UTC)

Drug labels

There was some discussion above about the package inserts (labels) and whether they are independent, secondary references. Being somewhat familiar with the regulatory process, I'd like to mention that the drug label ("package insert", "Prescribing information") is not simply something that the manufacturer writes up with a cursory review by the FDA, but is in fact the core legal document that outlines the specific indications that a drug may be marketed for along with the approved doses and health claims. The approval of any new drug is in fact the FDA's determination that there is "substantial evidence that the drug will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling or proposed labeling thereof." (21 USC Section 352 paragraph d) This is in fact where the term "off-label marketing" that we've all become so painfully familiar with in recent years comes from.

I've provided a link to the drugs@FDA website summary of the regulatory history of linezolid here. Linezolid was approved for marketing in April of 2000, in a letter from the FDA to Pharmacia and Upjohn's head of Regulatory Affiars. The letter states:

" We have completed the review of these applications, as amended, and have concluded that adequate information has been presented to demonstrate that the drug products are safe and effective for use as recommended in the agreed upon enclosed labeling text. Accordingly, the applications are approved effective on the date of this letter.

The final printed labeling (FPL) must be identical to the enclosed labeling (text for the package insert) and submitted draft labeling (immediate container and carton labels submitted August 11, 1999). Marketing the product with FPL that is not identical to the approved labeling text may render the products misbranded and unapproved new drugs."

The webpage includes a series of changes to the label over time, each of which is associated with a letter from the FDA stating that the company's "supplemental New Drug Application" has been approved. Any change to the label is a supplemental NDA that requires FDA approval. As such, these documents reflect the entire FDA review and approval process, and seem to me to be at least as rigorous a secondary source as any review article written by an associate professor and reviewed by two harried referees who have grant applications due next month. Formerly 98 (talk) 08:41, 22 April 2014 (UTC)

WP:Secondary does not mean independent. "Written by the manufacturer" is practically the definition of "not independent".

It's worth noting here, too, that "secondary" is not a fancy spelling for "good". Some primary sources and some non-independent sources are excellent sources (depending, of course, on what statements you're trying to support). WhatamIdoing (talk) 22:54, 26 April 2014 (UTC)

Well, I think if you read the material I've posted above, characterizing the label as "written by the manufacturer" is a bit of an oversimplification. The label is in fact the endpoint of the entire regulatory process - according to the strict language of the statute, the FDA technically does not approve drugs, it approves labels. And one thing it doesn't do at the end of a 10 month review process is allow the manufacturer to just fill in the blanks with its own conclusions about the drug's safety and efficacy.

On the drugs@FDA website, the Agency publishes an Approval Summary for each new drug, containing a detailed analysis of all the key clinical and preclinical data, and the Agency's internal analysis of the data. Its really all the same to me whether we use the FDA Approval Summary or the drug label as our source, because it 15 years of looking at these, I've never found any discrepancy in tone or content between the two. Nor, given the legal framework, would one ever expect to. Formerly 98 (talk) 11:06, 27 April 2014 (UTC)

Trust your doctor, not Wikipedia, say scientists

http://www.bbc.com/news/health-27586356

Count Iblis (talk) 18:40, 27 May 2014 (UTC)

Sorry count - removed copy/paste of article, which is a big WP:copyvio - that policy applies even to Talk pages. The link is sufficient. Jytdog (talk) 19:09, 27 May 2014 (UTC)

This matter has been discussed in several threads at WikiProject Medicine, most recently here. Axl ¤ [Talk] 19:27, 27 May 2014 (UTC)