Not all science is literally experimental [View all]

[cthulu2016]

Last edited Mon Dec 10, 2012, 05:13 PM - Edit history (9)

It has often come up here, in discussions or economics, psychology, sociology and other "soft" sciences that real science is experimental.

This is not quite right. A tremendous amount of very good science is not literally experimental. All science does, however, use the same rational framework implicit in the experimental model... thinking of the world in the same way one thinks of the elements of a laboratory experiment.

Science is about observation, logic and hypothesis/theory. Experiment is a sub-set of observation. Experiments create a very controlled reality to observe, and thus produce a highly trust-able sort of observation.

And anyone would, all other things being equal, prefer rigorous experimental data. It would be more scientific to sexually abuse children in a laboratory than to interview children who had been abused in the real world, but it would be evil, so we don't do that.

And it would be interesting to have a full scale version of our universe where gravity was 0.01% stronger than it is in our universe, but we cannot do that.

There is a hell of a lot that cannot be done experimentally, but much, and perhaps even most of science has been done with "natural experiments"... thinking of the world in experimental terms.

People who grow up drinking water with a very high level of natural fluoride have 'mottled' teeth... patches of brownish enamel. Dentists noticed that people with mottled teeth had less tooth decay. That's counter-intuitive. A seeming flaw (mottling) isn't what you'd expect from the strongest teeth.

So dentists looked at where people had mottled teeth and studied the water. The best candidate for a meaningful difference--what was most distinctly unusual about their water--was fluoride levels. So people started studying the matter backward -- measuring fluoride levels and looking at tooth decay rates. There was a high correlation of high fluoride and low tooth decay. And it turned out that it made a big difference when people moved to a town with a lot of flouride in the water... most benefit appeared to come from being there when your adult teeth were still growing inside your gums.

Correlation is not causation. We did not know that fluoride makes teeth stronger but we had a lot of evidence for the proposition. The next step was to give developing kids fluoride, and later to introduce flouride in local water supplies, and observe the tooth decay rates. That was the experiment. And from that experiment we proved that fluoride, and specifically fluoride, ingested in childhood makes teeth stronger.

Now, what if for some reason it had been impossible to do the experiment? Would we have known nothing? Of course not! We didn't start putting chemicals in people's water until we were already fairly confident of the effect. We had a lot of evidence from which a reasonable person would think it very likely that fluoride makes teeth stronger. It was of sub-experimental quality but our studies of local fluoride levels and tooth decay rates were not unscientific.


No lab experiment tells us whether global warming is happening... we don't have another Earth in a lab somewhere where the internal combustion engine was never invented. But we do have tree rings and calcium carbonate strata and such in the real world that predate automobiles and we can look at them. Most climate data is measurement (observation) of our real climate. We are very limited in our ability to run experiments on the climate. Fortunately, most of what we know about physics and chemistry, the stuff climate is made out of, has been established through first rate controlled experiment.

No experiment proves that a dinosaur species can end up as a chicken over millions of years of evolution because we haven't had millions of years to run such an experiment. We have experiments with fruit flies showing what sort of generational genetic change is possible. We have observations of fossils of dinosaurs with breastbones we can compare to those of modern chickens. We have theories that try to account for things in the world that can be tested against both genetic experiment and fossil observation.

It's a big, complex, multi-disciplinary rational approach to the world and it is all science.



It is fashionable to dismiss economics as quackery because it is unable to predict things with precision in the real world. But it predicts all sorts of things very reliably. If I go try to sell my car for a dollar I will succeed. If I try to sell it for a milion dollars I will fail. How did I know that???

Granted, I didn't have to get an economics degree to know that, but I didn't need a physics degree to know to not jump off a tall building.

In both cases there is some reality... gravity, and a general tendency for people to want to get more for less. And reality can be studied.

Economics is unique in that all predictions are dismissed as obvious until we reach the point where economics cannot predict well. This is because People Like Money.

The stuff people desperately want economics to tell them is whatever it is economics cannot tell them because value comes from knowing what other people don't know. (And it is hard to get people to put money down on both sides of a bet if the outcome is known to everyone.)

Physics can predict to a hundred decimal places how long it will take a bowling ball to drop ten feet. Why can't economics predict what the stock market will do tomorrow?

Well, consider the alternative. RESOLVED: An economic theory succeeds in predicting what the stock market will do.

What happens? Everyone knows what the stock market will do and acts accordingly. You know the stock market will br flat today and will go up 100 points tomorrow so you buy today. And so does everyone else, and now the stock market is 100 points higher than it would have been if everyone hadn't known it would go up 100 tomorrow and, obviously, the theory that predicted a 100 point gain tomorrow was NOT predicting a 100 point gain today so that theory turned out to be wrong. The theory undid itself by being so good that people accepted it as reliable.

The acceleration of a falling bowling ball does not depend on observers betting on how fast it will fall.

So, given that it is impossible for a theory of the stock market to be both accurate and well-known, does that mean that economics is junk?

Hardly. A sharp modern stock trader could take the market of the 1920s apart. I could kick Henry Ford's ass running a factory. Almost any of us could depose the great banks of Venice or Florence.

Modern economic thinking is vastly more sophisticated and effective than older economic thinking, which suggests that there is real valid stuff going on.

But every participant in an economic system is seeking to game the system. The more we learn the better we can game.

Any modern chess master would demolish Paul Morphy (the first great American chess champion). We know a lot more about chess strategy today. But that does not mean that we always win chess games today.

It is still a game with a winner and a loser. (Excluding draws.) And all that we have learned about chess has not changed the fact that half the players lose.

And our learning about chess has created a world where nobody can casually kick-ass on the whole world like Paul Morphy did.

No chess player today is as superior as Paul Morphy was, in the context of his times.

Does that mean that we are worse at chess... that what we have learned about chess is invalid? No, it means that competition exists. Bobby Fisher was fond of King Pawn openings even though it was believed that Queen Pawn openings were superior. But 1960s-1970s tournament players had spent most of their time thinking about Queen Pawn openings and didn't know KP openings as well, so the supposedly 'inferior' openings were often more confusing to opponents.

In a dynamic system the "right answer" will tend to shift.

The rules of chess, and all the strategic implications of those rules are static. There is probably an optimal first move "out there"... and all other things being equal best first move. The competitive system of people studying chess and playing tournaments, however, is dynamic.

I don't think economics can ever perfectly predict... that it is actually impossible. I am, however, certain that the CBO can 'score' the dynamic effects of hypothetical legislation much better than a random number generator can.

And I am sure that things we learn about economics are, like what we learn about chess, real advances in understanding--but advances in understanding that actually make the game harder, not easier, because a lot of people know a lot more about chess than we used to.