Amateur experts

I ran into an interesting article today, about how juries suddenly think they're experts on how evidence should be presented at a trial having watched CSI. Take this example from the article:

A disappointed jury can be a dangerous thing. Just ask Jodi Hoos. Prosecuting a gang member in Peoria, Ill., for raping a teenager in a local park last year, Hoos told the jury, "You've all seen CSI. Well, this is your CSI moment. We have DNA." Specifically, investigators had matched saliva on the victim's breast to the defendant, who had denied touching her. The jury also had gripping testimony from the victim, an emergency-room nurse, and the responding officers. When the jury came back, however, the verdict was not guilty. Why? Unmoved by the DNA evidence, jurors felt police should have tested "debris" found in the victim to see if it matched soil from the park. "They said they knew from CSI that police could test for that sort of thing," Hoos said. "We had his DNA. We had his denial. It's ridiculous."

It struck me that this type of problem is hardly limited to crime scene investigation. Almost all of the cranks you see are people with next to no knowledge in a subject area who have heard snippets about it and think they've come up with some insightful breakthrough - never having actually gotten an education in it. And then there are creationists who think they know enough about biology to disprove evolution, or enough about astrophysics to prove the universe couldn't be 13.7 billion years old, or enough about geology to... Well, you get the picture.

But by far the worst offenders in this area are conspiracy theorists. These people are not only experts in crime and how to keep a conspiracy secret (from everyone but themselves, naturally), but they'll make arguments based on their "expertise" of almost any area of science and engineering, from the "speed of gravity" to how a flag waves in a vacuum.

The problem is, for any of these areas there are actually numerous people who are unquestionably experts looking at the same data. If there were a problem, why wouldn't the experts see it? Ah yes, they must be in on the conspiracy, too. Only the average people (who coincidentally have a worse understanding of the principles involved) will dare to point out the flaws.

The Idiot Box

But what is it that makes so many people falsely believe they understand these situations as well as experts? Well, like in the CSI example, a big cause of it is likely television. People these days watch it a ton, and they expect it to be true. There's some innate expectation on people that they wouldn't be allowed to say (or show) something on television if it weren't true (or scientifically accurate or at least plausible in the case of fiction). Possibly this is due to a simple trust for authority (which might explain why this prediliction isn't present in everyone), but I can't say for sure - I'm no expert on this matter.

So, people watch a lot of TV and expect what they see there to match with the real world. Unfortunately, a lot of things get in the way of this. First of all, the writers of shows almost never have a good grasp of the science involved (that's why Bad Astronomy exists: to set them straight). Sometimes they'll consult experts, but not often. The second problem is that the writers will often actively choose to go against what's accurate in order to increase drama or humor.

The result of this is that television is filled with ridiculous notions such as exploding cars, sound in space, lack of inertia in space, gunshots sounding like explosions (they're more like firecrackers in general), etc. People get so used to the TV version of science that the real-world version ends up seeming less real.

Lies to Children

Another problem that leads to false expertise is that many of these people simply don't realise their knowledge is incomplete. Almost every adult has taken various science classes in high school, and many of them roughly remember the lessons they learned. However, high school science is vastly simplified compared to expert-level science, but this isn't always made clear if they don't go on to learn more. For instance, students are generally taught that both mass and energy are conserved, but it's less often they're taught that nuclear reactions can convert between the two, and that thus all mass is energy, and this combined quantity is what gets conserved. For that matter, even that's a bit of a simplification as there are a couple subtle complications to it, such as conservation looking the other way for a moment for quantum tunneling to occur, or in cosmological redshift where energy is drained from light as it travels across space for a long time (this is sometimes handwaved away as a form of potential energy).

It requires a vast amount of knowledge to be an expert in any academic subject, and most people simply don't have it. In general, to become an expert you should expect to graduate from high school, go through around 8 years at university, and then spend many more years in postdoctoral studies. On top of that, it's expected that you get published in a reputable journal multiple times and/or gain significant praise and/or awards from peers in the field.

Starting from a bit more than Scratch

But what if you disagree strongly with a major tenet of a certain a field, and still wish to become an expert in it? For instance, let's take the classic example of evolution in the field of biology. The first question that needs to be answered is whether you started to disagree with evolution because you studied biology for 10 years and it just didn't add up, or whether you disagree with evolution for other reasons. In the first case, you're probably alright. You've already done all the studying, and can hopefully show that you understand the material and maybe convince other professionals of the flaws in the model. If you can do so reasonably, you can get to be regarded as an expert.

In the second case, however, you're making a fundamental error. You've come to a conclusion on a subject when you haven't studied it extensively for yourself. Now, if you were accepting the word of experts on this subject and trusting that they've probably come to the right conclusion, this isn't bad at all, but that's not the case we're talking about here. What's happening here is that you either disbelieve in the theory either due to your own faulty knowledge or due to the authority of a non-expert. If you want to become an expert, you have to accept where the evidence and greater understanding leads you. You can never become an expert by starting with an assumption and then trying to find all the evidence that justifies it.

In short, my advice to anyone who wishes to become an expert in a field but holds a differing belief from the bulk is this: Drop that belief, try to get rid of any emotional investment you may have in it (try to revert back and start at zero, before it was ingrained in you), and go where the evidence leads you. Start your investigation with questions, not answers. In fact, this is probably good advice even if your beliefs coincide with those of experts; you'll learn how it was all figured out from scratch, will ask all the right questions, and if the theory turns out to be wrong you just might discover this.

But of course, I'm no expert on expertise, so you don't have to take my word for it.

We may not torture, but we play Metallica

In the US news, it seems the best stories always slip through the cracks - sometimes even the Daily Show misses them. Here's one from a few years ago which I just found which seems to have only showed up in the British papers, and no, it is not a joke: Metallica is latest interrogation tactic.

This makes me wonder: Why do they use good music (for the heavy metal part, at least)? Why not just pull out the absolute worst? I guess maybe, since they're already skirting the line with Metallica, anything worse would cross over into definitely being torture.

Victory is ours!

The Tripoli Six are now free. (Found via PZ and Wikipedia)

I feel left out

Recently, it seems the trolls have been infesting all the skeptical blogs around here, but there's one big notable exception: Me (oh yeah, and JanieBelle too, though the adult content on her blog could explain it). So I'm asking you: What am I doing wrong (or right, depending on your perspective)? Do I not post frequently enough? Am I too nice? Do they not get my Terry Pratchett references?

Pseudodebate

Continuing on the theme of "pseudoskepticism" over the past few weeks (see here and here), I've come upon some of Marcello Truzzi's original comments on the subject and have decided to give some specific responses. You can read the article, originally published in his Skeptical Inquirer, here.

After a brief introduction, Truzzi starts out with discussing the burden of proof of evidence. Most of his article follows on this theme, so I'll just quote this to give the summary. (All italics is his original emphasis.)

In science, the burden of proof falls upon the claimant; and the more extraordinary a claim, the heavier is the burden of proof demanded. The true skeptic takes an agnostic position, one that says the claim is not proved rather than disproved. He asserts that the claimant has not borne the burden of proof and that science must continue to build its cognitive map of reality without incorporating the extraordinary claim as a new "fact." Since the true skeptic does not assert a claim, he has no burden to prove anything. He just goes on using the established theories of "conventional science" as usual. But if a critic asserts that there is evidence for disproof, that he has a negative hypothesis --saying, for instance, that a seeming psi result was actually due to an artifact--he is making a claim and therefore also has to bear a burden of proof.

So here's the situation: Some experiment somewhere did a test for the presence of psi. They came back with results and claimed that these constituted evidence for psi. What is a skeptic to do? Truzzi's strawman of a skeptic would immediately say that the result must have been an artifact or due to some explanation other than psi, and then give no justification for this. But just because no justification is given doesn't mean no justification exists.

Let me illustrate this, once again with the example of "slood," a postulated substance with ill-defined properties. In our world, we can explain almost everything without appealing to slood. Any events that people have attempted to explain with slood are just as easily - if not more easily - explained by known phenomena. There are many models of the underworkings of the universe, and not one of them predicts the existence of slood. In fact, fitting it in to any of these models would require a drastic reworking. Simply put, there's absolutely no reason to believe that slood exists.

Now, let's say one group of never-say-die scientists designs an experiment to test for the presence of slood. If slood exists, this experiment would be expected to give Result A (which is generally a statistically significant deviation from what we would expect in a sloodless universe). The experiment is performed, and Result A is found. The scientists then claim this is evidence of slood.

So what's a skeptic to do here? Accepting slood would require a drastic reworking of known science, and up to now there's been no reason to believe it exists. On the other hand, there are many other possibilities to explain why the experiment resulted in the way it did:

1. A statistical fluke - Generally, statistical significance requires 95% certainty. This means that there's a 5% chance this result was obtained purely as a matter of chance. Of course, some results can be more extreme, and the chance of them happening randomly is less (though never zero).
2. A known mechanism - The experiment might have failed to control for (or properly control for) the possibility that some known physical phenomenon influenced the result. In this case, the result could just mean that this phenomenon was coming into play.
3. An unknown mechanism - Maybe it wasn't slood that caused the result of the experiment. Maybe it was dools, a substance which no one has even thought of yet (but which doesn't require a drastic reworking of science to fit in). Or maybe our universe is all a giant simulation and the designer decided to toy with us by changing the results.
4. Fraud - Many "scientists" make a living promoting the existence of slood, and finding evidence that it exists would be in their benefit. Some such scientists might have been involved in this experiment and had a chance to cheat. Also, it's known from past experiments to test for slood that fraud has indeed occurred.

Adding these four possibilities in, we've got a total of five ways this result could have occurred, only one of which implies the existence of slood. Now, the number of ways it could have occurred alone doesn't give us the chances that it's any one way. To get this, we have to look at the likelihood. From the beginning, we know that accepting slood would require a drastic reworking of science and we've seen very little prior reason to believe it exists. This makes this possibility very unlikely, so it's much more likely that instead one of the other possibilities came into play.

So even despite this experiment's results, it's still most likely that slood doesn't exist. Many amateur skeptics will stop there and simply say the evidence isn't convincing enough. However, some skeptics will then go a step further and analyze the experimental protocols to judge the plausibility of alternatives 2 and 4. Some scientists might even see if they can come up with a theory for a new phenomenon to meet alternative 3.

Generally, the result of these will be the skeptics coming up with ways the experimental controls can be tightened and the scientists with ways to change it to rule out alternative theories. If the original experimenters are serious, they'll then take these changes to heart and perform another, better-designed experiment with a few checks against fraud. If they aren't serious, they'll whine about "pseudoskeptics" and make no progress towards a better experiment.

Sometimes, in the real world, investigators do indeed take recommendations to heart. For instance, this happened in the case of the Ganzfeld experiments. What's happened in every one of these cases I've come across is that the results take a sudden plunge back towards the expected results. Sometimes the significance fades away completely, while other times it remains. In either case, this gives evidence that the skeptics were right in that there were some problems with the experiment.

The next step of inquiry will generally have skeptics and scientists looking over the experimental controls again to see if there are any remaining flaws. If some are found, the process is repeated. If none are, slood proponents will then dance on the rooftops claiming slood is proved to exist, but one important step is still left. This step is replication. What is done here is that a completely independent group should recreate the experiment from scratch, and see if they get the same results. Preferably, no one involved with the first experiment should be involved in the replications, and if possible, those involved in the replications should be neutral on the matter or even negatively biased.

The point of replication is to provide another level of checking against design flaws and fraud. The replicators can start off with all the recommendations over how to keep it tight, and shouldn't have any bias which would lead them to consciously or subconsciously influence the results. It also provides another check against statistical flukes, as it's less likely the same fluke would occur twice. Only once the experiment has been independently replicated do we truly have a reason to believe that slood exists - though we can never rule out alternative explanations. Further experiments should likely be performed to test other proposed properties of slood and to see if alternatives can be ruled out. After a long time has gone on with many tests of slood, none of which it fails, it will eventually become accepted science.

A single experimental result is never enough to change scientific consensus, and considering it most likely that an unusual result is due to an artifact, poor experimental design, or fraud is simply the default position. It takes work to overturn the current scientific consensus, and paranormal enthusiasts rarely want to put in this work. When they actually do, it tends not to work out for them. Skeptics are there to make sure they follow proper procedure and don't try to short-circuit the system, not to bar the way to new findings. If some claim has truth behind it, it can make its way in through the standard system. Accusations of skeptics being pseudoskeptics really just amount to giving up and fighting the wrong the battle.

You know what else really grinds my gears?

Having my comments not approved to be posted in the first place. A moderation queue is generally intended to block off spammers and trolls from ever getting their comments onto your blog. When you use it to block off people who say stuff you just don't like, you've gone too far.

What makes it particularly frustrating is that this incident didn't occur on some woo's blog; it was on one of the blogs at ScienceBlogs. I won't mention which one in order to be more tactful that is probably deserved, but I'll note that it wasn't one of the big controversial blogs. All my comment really was was some helpful advice, but apparently some people can't take it.

Disclaimer: I have heard rumors that there have been some weird interactions of ScienceBlogs' e-mail notifications and Gmail's spam filter. I have seen other comments appear on this blog since I sent mine, but the interaction is apparently a bit capricious. This exists as an alternative explanation, though the cynic in me doesn't believe it.

Skeptic's Circle #64

The latest Skeptic's Circle is now up at The Skeptical Alchemist. Read. Learn. Pass it on.

Open thread as usual, but mentioning how this is so late it's kind of pointless as you already knew about this Skeptic's Circle is FORBIDDEN.

Litcraft: Even the story about a story gets a Prologue

Given a ratio of one for to none against my idea of talking about writing a fictional skeptical story (or as I like to call it, an infinite approval ratio), I've decided to go ahead and try it out. If it doesn't go so well, I can always delete every post talking about it and pretend it never happened, just like I did with... er... Oh, nothing.

So let me first summarize what I'm going to be doing here. The goal of my writing is to create an engaging fictional story which postively represents Scientific Skepticism. Some other goals to go along with this:

• The main character, who learns the benefits of skepticism throughout the book, will not be made to appear foolish or tragic for his/her beliefs at the beginning.
• The story will attempt not to be overly preachy, and should not turn away fence-sitters or mostly-reasonable people (there's just no helping a few lunatics).
• The story will apply the mantra of "Show, don't tell," to its goal. It will show the benefits of a skeptical approach rather than just claim them.
• The universe must be one with preset rules. As this is a fundamental assumption about our universe when working with the scientific method, throwing it away will make this venture utterly pointless.
• The story will not be solely focused on the goal of promoting skepticism. It will contain many other elements in an attempt to make it a good book in addition to any message it brings across.
• One sub-theme will be promoting having a sense of wonder and asking questions.
• I will try to avoid many big narrative pitfalls that can ruin a story, such as a deus ex machina to solve a big problem.

At this point, I'm going to give you guys a chance to chime in. Is there anything I missed here that you think I should add as a goal? Or, is there any goal here you think I shouldn't have? Or would you prefer I delete this post and never mention the subject again?

Pardon me...

If you've been watching the news lately, you may have heard that Scooter Libby recently had his prison sentence commuted by President Bush (though given that the Daily Show and Colbert Report are on vacation this week, I can forgive you for not knowing). Bush's rationale for this, taken from Wikipedia:

Mr. Libby was sentenced to thirty months of prison, two years of probation, and a $250,000 fine. In making the sentencing decision, the district court rejected the advice of the probation office, which recommended a lesser sentence and the consideration of factors that could have led to a sentence of home confinement or probation.

I respect the jury's verdict. But I have concluded that the prison sentence given to Mr. Libby is excessive. Therefore, I am commuting the portion of Mr. Libby's sentence that required him to spend thirty months in prison.

My decision to commute his prison sentence leaves in place a harsh punishment for Mr. Libby. The reputation he gained through his years of public service and professional work in the legal community is forever damaged. His wife and young children have also suffered immensely. He will remain on probation. The significant fines imposed by the judge will remain in effect. The consequences of his felony conviction on his former life as a lawyer, public servant, and private citizen will be long-lasting.

I can't find the exact quote at the moment (help me out here if you can), but I'll point out that the unmasking of an agent such as Valerie Plame was specifically described by Bush as being "treason." So, for someone committing treason, why would he think that 30 days in jail was too harsh? Doesn't he usually do much worse things to people who haven't even been convicted?

Okay, let's drop the veil of neutrality. Anyone disagree that he just did this because Libby was acting under orders from higher up? Of course, that's not an excuse for breaking the law, but when the people ordering to break it have the ability to protect you from any punishment for doing so, there's a little conflict of interest.

We really need to reform the pardoning system. As it is, it gives unlimited power to the executive branch to immunize anyone they want for breaking any law they want (unless the legislature goes through an impeachment proceeding, but this only works on elected officials). This has been abused in the past, and it's being abused now. However, there are a few arguments for it, such as it being a check on judicial decisions. In response, I think what we need is another check. For instance, make it possible for Congress (or some subsection) to overturn a pardon by popular vote. Any thoughts on this idea?

No place left for God

From a story via Bad Astronomy, it looks like there's an interesting new theory coming out which might explain what happened prior to the Big Bang. Essentially, it turns into another universe contracting down to almost nothing, then undergoing a Big Bounce to come back out again.

Anyone else thing that if this theory gains ground, it's going to drive religious people crazy? If the universe had no beginning, the concept of a God creating it goes right out the window. Sure, some of them might come up with ideas like a secondary temporal axis, but this isn't going to be tractable to the public. What they're going to have to do is deny, deny, deny. If this theory becomes commonly accepted, expect it to see just as much controversy as evolution.

Strange searches, June edition

I recently installed Google Analytics into this blog, so I'm now able to see some of the strange searches that lead people here without resorting to checking the sitemeter every three days. Unfortunately, I don't tend to get as many completely sick ones, but I do get my own subsection of weird. Anyways, onto the show!

reverse mmorpg english - How do you reverse an MMORPG?
quiddle - I can't even make up words properly anymore. At least I'm on the front page for this one, so it's not too common.
religion "double slit" - No, saying God decides which way the particle goes doesn't actually explain anything (and it doesn't even work).
what good chiropractors do - Become massage therapists.
ridiculous commandments - All of them (Okay, maybe thou shalt not kill and thou shalt not steal have some merit. Maybe).
past tense of relief - Nouns have no past tense; they exist only in the present. If you buy into Zeno's crap, this means they can't exist at all.
answer 43 ultimate - Subtract 1.
reasons why not to believe in christianity - Have you read the Bible? It gives a ton of good ones.
191f4e7bcac950618399a24eb37bbcbda4af1b8609995234 - This number does not appear once in my blog, and search engines I've tried with it return zero results. The strange part is how it showed up as a keyword for me.
when 2 neutral charges are put together - The Third Doctor comes in and tries to reverse the polarity of their flow.
is ghost hunting antichristian - Probably. These days, your average pen is antichristian.
satirist are - I can't imagine how far down the search results they must have gone to get to me.
intelligent"sound" - I think Douglas Adams came up with an intelligent shade of blue at one point, but that's the closest I can get.
key combination "shift key stuck" - Next time you search for that, do a favor and type in all caps. It's just funnier, 'kay?
ask god - Again, how far down in the search results are they looking to find me?
how to use somehow in a sentence - Somehow, it doesn't surprise me there are people out there who can't figure this out.
a glass of water has one ice cube floating in it. the ice melts. disregarding evaporation, the water level: - Do your homework yourself.
zealot render - Now, this one sounds like a good name for a weapon skeptics can use against religiosos. Might just have to keep it in mind in case they try to start a holy war.
measures to prevent melting of icecaps - Sorry, quantum observation effects don't work on a scale large enough that measuring icecaps will stop them from melting.

Not on the strange side, but I also got a ton of searches for "Quantum Mechanics for Dummies" and variants on it, plus some miscellaneous other physical stuff for dummies, so I know my posts on these subjects are hitting some people. Of course, the problem is that you just can't dumb down Quantum Mechanics past a certain level, so there's no way I'll be able to get everyone to understand it. Nevertheless, maybe it's helping some people.

You know what's also strange? Despite my liberal seasonings of Terry Pratchett references, I didn't get a single search for any of them. With that in mind, let's see what we can do for next month:

Tiffany Aching
Agatean Empire
Ankh-Morpork Assassins' Guild
Ankh-Morpork City Watch
Ankh-Morpork Post Office
Carrot Ironfoundersson
Dark Desert
Death's Domain
Detritus
Discworld
Djelibeybi
Cohen the Barbarian
Cut-Me-Own-Throat Dibbler
Dungeon Dimensions
Ephebe
Gaspode
Genua
Granny Weatherwax
Greebo
Hex
History Monks
Igor
Bloody Stupid Johnson
Klatch
Lancre
Leonard of Quirm
The Librarian
L-Space
Lu-Tze
The Luggage
Magpyr family
Mort and Ysabell
Nanny Ogg
C.W. St J. "Nobby" Nobbs
Moist von Lipwig
Mr. Pin and Mr. Tulip
Mustrum Ridcully
Nac Mac Feegle
The Great God Om
Parasite universe
Pseudopolis
Rincewind
Roundworld
Sto Lat
Susan Sto Helit
Ponder Stibbons
General Tacticus
Twoflower
Überwald
Unseen University
Verence II of Lancre
Havelock Vetinari
Samuel Vimes
Lady Sybil Vimes/Lady Sybil Ramkin

There we go, that should do it.