W i l l T h e s e P i l l s M a ke Me Smar ter? S o u r c e : G o l d , P. E . , C a h i l l , L . , & We n k , G. L . ( 2 0 0 2 ) . G i n k g o b i l o b a : A cognitive enhancer? Psychological Science in the Public Interest, 3, 2-11.
Goals: Te a c h t h e i m p o r t a n c e o f e va l u a t i n g c l a i m s. Te a c h t h e i m p o r t a n c e of evaluating sources of information. Te a c h a b o u t p o t e n t i a l b i a s o f s o u r c e s. Te a c h t h e g e n e r a l procedure to conduct a double-blind experiment.
Basic Idea: Many companies advertise “miracle drugs” that claim to m a k e p e o p l e s m a r t e r. This lesson discusses the scientific method used to test the truth of these claims for one o f t h e s e d r u g s.
C I RC
Gain Attention/Interest: Vignette To n y a h a s a b i g m a t h t e s t t o m o r r o w a n d s h e n e e d s t o s t u d y t o n i g h t . W hile at the grocer y store she sees a Ginkgo Biloba advertisement that c l a i m s i t w i l l i m p r o v e h e r m e m o r y. B e c a u s e s h e w a n t s t o d o w e l l o n her test, she buys the Ginkgo Biloba, goes home, and reviews the same m a t h s h e h a s s t u d i e d f o r t h e l a s t t h r e e w e e k s f o r t w o h o u r s e v e r yd a y i n s t e a d o f g o i n g o u t w i t h h e r f r i e n d s. T h e n e x t d a y, To n y a d o e s v e r y w e l l o n h e r t e s t , wh i l e h e r f r i e n d s d o n o t d o a s w e l l . To n ya t e l l s a l l h e r friends she bought Ginkgo Biloba and used it to raise her score. Should her friends go out and buy Ginkgo Biloba? Will you? Ginkgo Biloba is an herb that comes from the Ginkgo tree. W h e n i t i s s o l d i n s t o r e s i t i s u s u a l l y i n t h e f o r m o f a n e x t ra c t from the leaves of the tree that is made into a pill. Companies that sell Ginkgo Biloba often make many claims a b o u t w ha t i t c a n d o. T h e s e c l a i m s c a n i n c l u d e :
· li omnpgr-ot ve remd smh eomr to- rayn d · i m p r o v e d a l e r t n e s s · faster reaction time · energ y boost · better mental clarity · decreased anxiety T h e s e a r e s o m e p r e t t y i m p r e s s i ve c l a i m s. H ow e ver, a r e t h e y tr u e ? B e f or e p u r c h a s i n g s u ch a p r o d u c t , wo u l d n’t yo u w a n t t o know whether or not it actually worked, or even if it might hurt you? How can you check?
www.circ.cornell.edu C o p y r i g h t © 2 0 0 5 M a t t h e w C . M a k e l & We n d y M . W i l l i a m s. P r i n c i p a l I n v e s t i g a t o r : We n d y M . W i l l i a m s . C o n t e n t : M a t t h e w C . M a k e l & We n d y M . W i l l i a m s . L a y o u t & D e s i g n : P a u l B. P a p i e r n o.
T h i n k & Wr i t e # 1 Wo u l d yo u b u y g i n k g o b i l o b a ? Why or why not? Some students may need a more specific question, such as, would you want to use it to prepare for finals? How would you feel if everyone in your class but you was taking it?
Ask: What is Science? “ C h e c k i n g � i s e x a c t l y w h a t s c i e n t i s t s d o. I t i s t h e i r j o b to eliminate all other possible answers and figure out t h e m o s t l i k e l y a n s w e r. A n o n s c i e n t i s t , l i k e To n y a , m i g h t n o t i c e h e r p e r f o r m a n c e i m p r o ve a f t e r t a k i n g Ginkgo Biloba, and attribute the success to having taken Ginkgo Biloba.
1 Ask: What is Science?
Should we believe Ginkgo can do all these great things j u s t b e c a u s e t h e b o t t l e s a y s s o ? N o. S c i e n t i s t s f o r m u l a t e hy p o t h e s e s , o r p r e d i c t i o n s, a b o u t w h a t t h e y think is the case, and then for mulate ways to test their g u e s s e s. T h i s m a y s o u n d s t r a n g e, b u t n o n s c i e n t i s t s t r y t o p r o v e t h e m s e l v e s r i g h t , w h i l e s c i e n t i s t s t r y t o p r o ve t h e m s e l v e s w r o n g. S c i e n t i s t s b e l i e v e t h e y m a y h a v e t h e HYPOTHESIS right answer only if they can find no way to show that their A hypothesis is a hy p o t h e s i s i s w r o n g. T h i s i s c a l l e d t h e m e t h o d o f p r o o f b y prediction about disproof. Scientists gather proof for their hypotheses by how and/or why things will happen. d i s p r o v i n g a l l o t h e r p o s s i b l e a n s w e r s.
2 Define the Problem: See Many Sides.
The lab for this lesson approximates the procedures scientists have used to test whether or not Ginkgo B i l o b a wo r k s.
Define the Problem: See Many Sides What is our problem here? S o l i c i t r e s p o n s e s. D o e s G i n k g o w o r k ? What does it do? As a potential consumer who might buy Ginkgo at the store, you may look at the problem from a different perspective than other people. What are the different groups of people who might look at the situation differently?
P e o p l e wh o m i g h t l o o k a t t h e s i t u a t i o n d i f f e r e n t l y i n l c u d e :
· store owners · mg iannkugfoa c t u r e r s · doctors
of
· the gover nment · mg ianrkkgeot e r s o f · memor y
· insurance companies · pmraonduufcat cs t u( er .egr.s, dorfu g o t h e r companies)
researchers
Manufacturers and marketers of Ginkgo want to sell as much Ginkgo as p o s s i b l e , w h i l e i t ’s t h e g ov e r n m e n t ’s j o b t o p r o t e c t c o n s u m e r s. S c i e n t i s t s l o o k f o r p r o b l e m s, o r t h i n g s w e d o n’t k n o w, a n d t r y t o s o l v e t h e m . I m a g i n e w e a r e To n y a ’s f r i e n d s and she has just told us of the fantastic powers of Ginkgo Biloba. This is an important part of science: clearly defining the problem with which you are w o r k i n g. CONSIDER THE SOURCE
An independent scientist does not usually have an investment in how an experiment turns out. Scientists are usually t r u s t wo r t h y s o u r c e s.
CONSIDER THE SOURCE
Some people or groups of people may have different motivation than others, so knowing the source of infor mation is important when evaluating how you should interpret it.
With so many interested parties all looking at the s i t u a t i o n f r o m d i f f e r e n t p e r s p e c t i ve s, t h e r e i s a l o t o f i n f o r m a t i o n o u t t h e r e. M u c h o f t h i s infor mation contradicts what some other groups claim. What kind of infor mation is best?
Distinguish Fact From Opinion: Learn What Constitutes Scientific Evidence If we want to find out whether Ginkgo really wo r k s, w e n e e d t o s e p a r a t e f a c t s f r o m o p i n i o n s. T h i s i s e x a c t l y w h a t s c i e n t i s t s d o. What is the difference between a fact and an o p i n i o n ? ( S o l i c i t r e s p o n s e s. ) A n o p i n i o n m i g h t be what a person thinks about a certain topic; d i f f e r e n t p e o p l e c a n h a v e a v a r i e t y o f o p i n i o n s. On the other hand, a fact is not something that changes; it is the same for all people. Scientists want to know the facts about whether o r n o t G i n k g o B i l o b a r e a l l y w o r k s.
3
Fact Versus Opinion: What Constitutes Evidence?
Activity Have the class listen to a string of numbers read aloud, then have them w r i t e d o w n t h e n u m b e r s, i n o r d e r, a s b e s t t h e y c a n r e m e m b e r t h e m . Example: instr uctor, “T his is a method similar to the one scientists use to test whether taking Ginkg o improves perfor mance. I am g oing to r e a d a l o u d 1 2 d i g i t s. W h e n I a m f i n i s h e d s p e a k i n g w r i t e d o w n a s m a n y of the numbers as you can in the order in which they were read. 98074713568” (students write). B e s u r e d i g i t s a r e r e a d e ve n l y a n d w i t h n o e m p h a s i s. A l t e r n a t e l y, y o u may consider writing the digits down on an overhead and showing them to the class for 1 second each.
T h i n k & Wr i t e # 2 Hypothesis for mation Fo r m a h y p o t h e s i s a b o u t G i n k g o B i l o b a t h a t scientists could test using a procedure similar to the one the class just perfor med. N e x t , s p l i t t h e c l a s s i n t o t wo g r o u p s. T h e l e f t h a l f o f t h e c l a s s r e c e i v e s “Ginkgo” (orange tic tacs) and the right side of the room gets a “fake pill” (white tic tacs). Do not tell the class the significance of orange or white u n t i l t h e e n d o f t h e a c t i v i t y. A f t e r e a c h h a l f t a k e s t h e i r “ p i l l ” , r e p e a t t h e a c t i v i t y w i t h a n e w s t r i n g o f n u m b e r s : 8 0 6 4 5 2 6 5 9 2 1 . To s h o w a n e x a m p l e o f b i a s, s e l e c t a s t u d e n t w h o t o o k a “ f a k e p i l l ” a n d b e v e r y g e n e r o u s i n giving him/her credit for a correct response, i.e., if she had 8046 as the f i r s t f o u r n u m b e r s, s a y s h e g o t i t c l o s e e n o u g h . T h e n , s e l e c t a s t u d e n t w h o took the “Ginkgo” and be more strict with his score, i.e., having 5912 at t h e e n d w o u l d b e i n c o r r e c t . I n t h i s c a s e , t h e b i a s i s a g a i n s t G i n k g o. U s i n g s i m i l a r m e t h o d s, t h e b i a s c o u l d b e i n f a v o r o f G i n k g o. [ A s a n a l t e r n a t i ve t o u s i n g t i c t a c s, y o u c o u l d h a v e h a l f t h e c l a s s s t a n d u p a n d d o j u m p i n g j a c k s o r s o m e o t h e r p h y s i c a l a c t i v i t y. ] Methods Used by Scientists To c o m b a t b i a s, s c i e n t i s t s f r e q u e n t l y u s e s o m e t h i n g c a l l e d a d o u b l e - b l i n d test. A double-blind test does not mean that both people involved are blind. Double-blind means that neither the person taking the medicine nor the experimenter giving the tests knows if the participant is taking the a c t u a l p i l l ( i n t h i s c a s e G i n k g o B i l o b a ) o r a f a k e p i l l c a l l e d a p l a c eb o . A placebo contains no active ingredients (like taking a breathmint to heal an
e a r i n f e c t i o n ) . Pe r f o r m i n g d o u b l e - b l i n d t e s t s p r e v e n t s b o t h p a r t i c i p a n t s and scientists from being biased in how they act or record data. Fo r e x a m p l e , a s c i e n t i s t w h o r e a l l y w a n t s t o s h o w t h a t G i n k g o d o e s n o t work could be more likely to look for mistakes made by a participant taking G i n k g o. S i m i l a r l y, i f a p a r t i c i p a n t k n e w s h e h a d t a k e n g i n k g o, s h e m i g h t expect to do better, and maybe not tr y as hard. Knowing what outcome is expected can inf luence your behavior; this is called response bias. Biases such as this are not possible in a double-blind test. Remembering a list of n u m b e r s i s j u s t a n e x a m p l e o f h o w s c i e n t i s t s c a n t e s t m e m o r y. T h e y c a n a l s o u s e l i s t s o f wo r d s, h a v e p a r t i c i p a n t s l i s t e n t o s t o r i e s a n d t h e n w r i t e t h e m d o w n , o r a s k t h e m t o r e m e m b e r l i s t s o f l e t t e r s. I f t h e g r o u p t h a t took Ginkgo performs better than the group that did not, then it is b e l i e v e d t h a t G i n k g o i m p r o ve s m e m o r y. F r e q u e n t l y, s t u d i e s l i k e t h i s g i v e d i f f e r e n t a m o u n t s o f t h e d r u g t o d i f f e r e n t people, predicting that people who receive more of the dr ug will show g r e a t e r e f f e c t s. I n t h i s c a s e , p e o p l e r e c e i v i n g m o r e G i n k g o w o u l d b e e x p e c t e d t o s h o w g r e a t e r m e n t a l a b i l i t y. S o m e s t u d i e s u s e o n l y a s i n g l e d o s e w h i l e s t u d y i n g G i n k g o. T h e y c o m p a r e people who either have just taken Ginkgo to those who have not. They can a l s o v a r y t h e a m o u n t o f G i n k g o t h e y g i v e p a r t i c i p a n t s, g i v i n g a l i t t l e t o s o m e a n d m o r e t o o t h e r s. Ye t a n o t h e r m e t h o d o f t e s t i n g G i n k g o i s t o h a v e p e o p l e t a k e G i n k g o o v e r a p e r i o d o f t i m e , s a y 6 m o n t h s, a n d t h e n c o m p a r e t h e m t o a n o t h e r g r o u p o f p e o p l e w h o h a v e n o t b e e n t a k i n g G i n k g o, t o s e e if there is a long-ter m effect.
Example: Long-ter m Effects I f y o u e a t t h r e e d o u b l e c h e e s e b u r g e r s i n o n e d a y, i t w o n’t d r a s t i c a l l y alter your weight. However, if you eat three double cheeseburgers e v e r y d a y f o r s i x m o n t h s, y o u wo u l d p r o b a b l y s e e a b i g g e r d i f f e r e n c e . T h e r e a r e a l o t o f d i f f e r e n t w ay s t o t e s t G i n k g o ’s e f f e c t s. H e r e i s a q u i c k review of how scientists check to see if Ginkgo actually improves perfor mance.
· Give Ginkgo to half the group and a placebo to the other half. · G i v e m o r e G i n k g o t o s o m e p e o p l e t h a n o t h e r s. · Gt oi vpee oGpilnek gnoo tt og i vpeeno pGl ei nokvgeo.r a l o n g p e r i o d o f t i m e a n d c o m p a r e t h e m
Weigh Evidence and Make Decisions T he results are in: Ginkgo does improve perfor mance on memor y tests! S o, g o o u t a n d b u y a l i f e t i m e s u p p l y a n d y o u ’ l l r e m e m b e r e v e r y t h i n g y o u nutti fwiaci t E mp rSo veel emcetmi oorny c o uE l dxpaom s s ipbll ye :wS a nct i. e B : hvoiwd e mnu cche daone sd GRi nakng d o oi m perfor mance? It tur ns out, not ver y much at all. In fact, some studies have found that Ginkgo improves memor y perfor mance about as much as drinking something with sug ar in it, which is not ver y much. N o o n e i s t r y i n g t o s e l l s u g a r a s a w ay t o m a k e o u r b r a i n s wo r k b e t t e r.
4
T h i n k & Wr i t e # 3 How about now? Have students write about what they think about Ginkgo Biloba now that they know what scientists have found.
Move From Science to Society
Weigh Evidence and Make Decisions.
Imagine if one of your friends started taking Ginkgo and then made the honor roll for the first time in her/ his life. Or think about seeing a commercial on TV showing testimonials from people who have tried Ginkgo and have experienced miraculous changes in t h e i r l i v e s. W i t h a l l t h e s e p e o p l e s a y i n g i t wo r k s f o r them, how can they be wrong? A variety of factors c o u l d e x p l a i n t h e d i f f e r e n c e . Fo r e x a m p l e , g o i n g o u t and buying Ginkgo is a sign that they were looking for w a y s t o i m p r ov e t h e i r p e r f o r m a n c e , s o m a y b e t h e y s t a r t e d wo r k i n g h a r d e r. O r p e r h a p s t a k i n g G i n k g o raised their confidence in themselves and so they s t a r t e d p e r f o r m i n g b e t t e r. W h a t d o e s t h i s m e a n f o r u s ? We l l , i t m e a n s t h a t c o m p a n i e s h a v e s o m e r e s e a r c h t h a t they can tr y to use to sell us a product for $10 a bottle. N o s t u d i e s h a v e s h o w n G i n k g o w i l l h u r t u s, s o s o m e people believe that there should be no restrictions on its sale. If people want to pay for it, and get what little b e n e f i t i t h a s t o o f f e r, t h e y s h o u l d h a v e t h a t o p p o r t u n i t y. H o w e ve r , w h e n c o m p a n i e s a d v e r t i s e G i n k g o, t h e y d o n’t u s u a l l y s a y t h a t i t i m p r o v e s perfor mance as much as drinking a glass of fr uit punch. Can you imagine a f r u i t p u n c h l a b e l c l a i m i n g t o i m p r ov e y o u r m e m o r y ?
5
Move From Science To Society.
As you can see from this discussion, whether or not Ginkgo Biloba works c a n i n f l u e n c e a l o t o f p e o p l e i n a l o t o f d i f f e r e n t a r e a s a n d c a r e e r s. To f o s t e r p o t e n t i a l i n t e r e s t i n s u c h c a r e e r s, t h e e d u c a t i o n a l r e q u i r e m e n t s o f each field are also included. A d v e r t i s i n g : A p e r s o n w o r k i n g i n a d ve r t i s i n g n e e d s t o u n d e r s t a n d t h e results of scientific research so he/she can accurately market the product. Advertisers typically attend four-year colleges and work for large a d v e r t i s i n g f i r m s. Dr ug Company Repr esentative: Dr ug companies spend billions of dollars r e s e a r c h i n g h o w p o t e n t i a l n e w m e d i c i n e s h e l p p e o p l e . Pe o p l e w h o w o r k for dr ug companies and make their products need to know which dr ugs are backed up by scientific evidence. Te s t P r e p I n s t r u c t o r : A p e r s o n w h o p r e p a r e s s t u d e n t s f o r t a k i n g standardized tests wants to give her/his students as much of an advantage a s p o s s i b l e . T h u s, k n o w i n g w h e t h e r o r n o t G i n k g o B i l o b a i m p r o ve s t e s t p e r f o r m a n c e i s i m p o r t a n t . Te s t p r e p i n s t r u c t o r s a t t e n d f o u r - y e a r c o l l e g e s. Fr u i t P u n c h S a l e s p e r s o n : I f c o m p a n i e s c a n m a r k e t G i n k g o a s i m p r o v i n g perfor mance when a glass of fr uit punch helps just as much, a fr uit punch s a l e s p e r s o n n e e d s t o k n o w t h i s.
Revisit, Review, Reflect, and Re-evaluate E v e n t h o u g h m a n y c o m p a n i e s a d v e r t i s e i t s m a g i c a l c a p a b i l i t i e s, s c i e n t i f i c investigation shows us that Ginkgo is not the wonder pill many companies would like us to believe. There are still relatively few scientific studies that investigate Ginkgo Biloba. Scientists continue to revisit and review how Ginkgo inf luences perfor mance, and ref lect on how previous research could be improved. By re-evaluating what they have already lear ned, scientists can for mulate new ways of testing Ginkgo and new ways to improve perfor mance.
6
Revisit, Review, Reflect, and Re-evaluate.
T h i n k & Wr i t e # 4 What's next? Now that you know what scientists have found, where do you think research regarding Ginkgo Biloba and perfor mance enhancement should go?
Discussion Questions 1. What can scientists do differently to test Ginkgo Biloba/perfor mance enhancement? 2 . Wo u l d i t b e f a i r i f t h e r e w a s a p i l l p e o p l e c o u l d t a k e t o m a k e t h e m p e r f o r m b e t t e r, b u t o n l y s o m e p e o p l e t o o k i t ? H o w d o e s t h i s c o m p a r e to some students taking test prep classes? 3. Should the government be stricter on companies that mislead the public?
H o m e wo r k Q u e s t i o n s 1. Are there other products similar to Ginkg o Biloba that tr y to market themselves as miracle products? What exactly do they claim and how do they support these claims? 2. Should the government pass a law regulating how companies can market products like Ginkgo Biloba? How might such a law influence t h e f o l l o w i n g g r o u p s o f p e o p l e : t h e e l d e r l y, t e e n a g e r s, a n d a d ve r t i s i n g firms?
Cor nell Institute for Research on Children
January 3, 2005
Quiz Questions Ve r s i o n A
1. What is an alternative explanation for why taking Ginkgo improves performance on memory tests? a. It does so about as much as drinking something with sugar in it, such as fr uit punch. b. Ta k i n g G i n k g o r a i s e s p e o p l e ’s c o n f i d e n c e , s o t h e y r e m e m b e r more. c. Buying Ginkgo indicates that people are looking for ways to i m p r o v e t h e i r p e r f o r m a n c e , s o m a y b e t h e y a l s o s t a r t e d wo r k i n g h a r d e r. d. All of the above. 2. What is the difference between a fact and an opinion? a . A n o p i n i o n i s s o m e t h i n g s o m e o n e e l s e b e l i e v e s, a f a c t i s something I believe. b. A f a c t i s s o m e t h i n g s o m e o n e e l s e b e l i e ve s, a n o p i n i o n i s something I believe. c. Facts are things scientists use to sound intelligent, and an opinion is something non-scientists have. d. Facts remain the same for all people, opinions can var y from person to person. 3 . I m a g i n e y o u a r e h e l p i n g y o u r f r i e n d p r e p a r e f o r a b i g t e s t . Yo u r f r i e n d sees an advertisement for extra strength Ginkgo Biloba. Should she buy it?
Quiz Questions Ve r s i o n B
1 . D o e s t a k i n g G i n k g o i m p r o ve p e r f o r m a n c e o n m e m o r y t e s t s ? E x p l a i n . What are some of the limitations of this evidence?
2. What is the difference between a fact and an opinion?
3. If you were a governmental regulator, would you change the ways that Ginkgo is marketed or advertised? Explain.
4. If you were going to design a study to test the effectiveness of G i n k g o, w h a t a r e t h e 2 - 3 m o s t i m p o r t a n t h y p o t h e s e s t o t e s t ?
Quiz Questions Ve r s i o n C
1. If you were going to design a study to test the effectiveness of G i n k g o, g i ve t wo h y p o t h e s e s t h a t w e r e n o t d i s c u s s e d i n c l a s s. 1
2
2. Choose one of the above hypotheses and explain a procedure a scientist could follow to test it.
3. How could someone check to see if an opinion is also a fact?
4. What are some potential ways a scientist could “cheat� on a doubleblind experiment?