TIMMI Progress Report for the Heritage Middle School (02/24/15 - 03/17/15) by Adam Chang

The Heritage Middle School Livingston, New Jersey February 24th to March 17th, 2015 Taught by TIMMI Partners from the Livingston High School

timm-i.org The Teaching Investment and Money Management Initiative

Table of Contents I.

Introduction ................................................................................................ 2 Introduction to TIMMI About the Teaching Investment and Money Management Initiative Teaching Details Community Details School Details Collection of Study Data Assessment Details Scoring Details

II.

Results ........................................................................................................ 11 Growth Between Pre-Tests and Post-Tests A Better Metric – Change Index Comparing the Assessment Versions Performance Differences Between Genders Individual Question Analysis Question 1 – Budgeting and Decision Making Question 2 – Credit and Debt Question 3 – Investing Question 4 – General Ideas of Money Management One Last Thing About Absences Feedback

III. Conclusions ............................................................................................... 28 Successes Study Improvements Teaching Improvements Notes on the PARCC In Summary

IV. Appendix: Original Data ........................................................................... 30 V.

Acknowledgements ....................................................................................51

â&#x20AC;&#x153;The number one problem in today's generation and economy is the lack of financial literacy.â&#x20AC;? Alan Greenspan

This report was written by Adam Chang The assessments were transcribed and formatted by Paul Yang, Vamshi Eppanapally, Justin Patel, and Jay Bhanushali Correspondence: research@timm-i.org TIMMI c/o Michelle Winter 30 Robert H. Harp Drive Livingston, NJ 07039 United States of America

Introduction Introduction to TIMMI TIMMI stands for the Teaching Investment and Money Management Initiative. Our organization works to comprehensively teach students in 6th-8th grade money management education, through the teaching and sharing of a central curriculum for financial literacy. In essence, we teach in our home community of Northern New Jersey/New York City and recruit/train/support global partners who teach in their own communities around the world. We are run entirely by students and neither charge for nor profit from our mission.

About the Teaching Investment and Money Management Initiative Our youth seriously lack education in financial planning and investing. Subjects such as FCS (cooking and sewing) and keyboarding are taught early, but money management remains largely under-taught if taught at all. This lack of financial understanding results in economic problems that remain and compound for decades. Taking student loans at expensive private colleges, poor budgeting, and making incompetent economic decisions (such as in investments) often extensively cripple one’s financial independence and stability. TIMMI’s goal is to change that. The financial crises of the past—the dot-com bubble, the housing bubble, and the Great Recession—as well as today’s general problems in consumerism and indebtedness are products of a lack of education. They can be averted through initiatives like TIMMI; our past cannot be our future. And the premises of financial security and prosperity, largely damaged by these past and present failures, must become our society’s realities so that the American Dream no longer has to be a dream. More information is available on http://timm-i.org. About the TIMMI Curriculum TIMMI’s curriculum consists of a series of four modules in a money management “boot camp” format, each session lasting around an hour. TIMMI is designed for middle-school students between sixth and eighth grade, and can be taught over one week up to one month depending on classroom availability. Principally, our curriculum seeks to be interactive and fun. Group discussion and dynamic/responsive teaching promote learning. So do case-scenario activities and humor.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Our curriculum can be broken down as follows: •

Day 1 begins with a pre-test and focuses on general foundations of money management and its significance, including budgeting and making decisions.

•

Day 2 focuses primarily on credit and debt.

•

Day 3 is centered on investing.

•

Day 4’s core is a fun and educational activity that incorporates material taught in the prior lessons, and ends with a post-test.

Teaching Details This report studies a teaching series held at the Heritage Middle School in Livingston, New Jersey (http://www.livingston.org/heritage). The series consisted of four sessions on 02/24/2015, 03/03/2015, 03/10/2015, and 03/17/2015. Teachers at the first and second sessions were Adam Chang, Christina Cuppari, and Kuran Malhotra. There were 21 students at the first session (12 females and 9 males), and 20 at the second session (12 males and 8 females). Teachers at the third session were A.C. and Justin Patel, where there were 25 students (12 females and 13 males). Teachers at the fourth session were Jay Bhanushali, A.C., K.M., and Paul Yang. There were 24 students at the fourth session, with 12 students of each gender present. Present at all four sessions was Mrs. Michelle Winter, advisor and teacher in the Livingston Public Schools. As a whole, thirty-four unique students attended the TIMMI sessions. This session was taught from the TIMMI curriculum, with adjustments. Some parts of Day 1 and Day 2 were not covered due to logistical issues, but the main concepts were thoroughly taught. The Kahoots were skipped on Days 1, 2, and 3, and played on Day 4 in place of the game. The administrative contact at the Heritage Middle School was: Mr. Robert Grosso rgrosso@livingston.org (973) 535 8000 x8524 The sessions were advertised with flyers placed around the school, through word-of-mouth, and through an email sent to parents of 8th grade students. TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Community Details Livingston is a suburban township in New Jersey, located around an hour west of New York City. Its population estimated by the 2013 5-Year American Community Survey is 29,420 people 1. Of adults in Livingston aged 25 and older, around 68.1% have a Bachelor’s degree, 31.7% have a graduate or professional degree, and 96.4% have a high school diploma 2. There is a strong agreement in this town about the importance of quality education. Livingston’s poverty rate is recognized as 2.5% for the general population, and 2.0% for children 3. However, United Way of Northern New Jersey found that 14% of households struggle with basic necessities like housing, food, and healthcare 4. We believe that this proportion of “Asset Limited, Income Constrained, Employed” households represents a truer examination of poverty than the federal poverty line, which doesn’t account for the high cost of living in this area or reflect current economic trends. From this information, it’s clear that Livingston is a highly-educated, (upper) middle-class town. Even with this higher income, however, the augmented finances make money management only more important. For example, there is a very high matriculation rate into college: 97% of students in the Class of 2014 at Livingston High School attended college 5, with many popular out-of-state and private schools (that accordingly come with high costs nearing $60,000 a year) represented. Another example: minors in Livingston often begin using credit and debit cards early on, due to having higher income families. Ultimately, it is essential that people of all incomes and backgrounds be financially literate, and Livingston has some special characteristics that even further prioritize financial literacy.

School Details Heritage Middle School has an enrollment of around 920 students, serving seventh and eighth grades, and is ranked at the 94th percentile for academic achievement in New Jersey 6. 94% of its students scored at least “Proficient” on the Language Arts Literacy section of the NJASK (standardized test), and 89% scored the same on the Mathematics section. Heritage is ranked at the 76th percentile for college and career readiness, and 84th percentile for student growth statewide. Fifteen percent of students are classified as “with disability,” and 1.5% of students are classified as economically disadvantaged5. However, in light of the statistics presented by the United Way study, we estimate that the actual percentage of economically disadvantaged students is much higher. U.S. Census Bureau, 2009-2013 5-Year American Community Survey, Livingston NJ, Table DP05 ACS Table S1501 3 ACS Table S1701 4 United Way of Northern New Jersey, ALICE Study of Financial Hardship 5 Livingston High School, School and Community Profile 2014-2015 6 NJ School Performance Report, Heritage Middle School, 2013-2014 1 2

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Collection of Study Data The pre/post assessment 7 was administered to students at the beginning of their first session of TIMMI, and at the conclusion of the last session of TIMMI. Students who did not attend the last session of TIMMI (10 of 34) did not take the post-test. Students were given the following instructions when taking the pre-test/post-test: •

This is a pre-test/post-test that is not graded and does not get associated with you in any way; the only purpose of this test is to see how well we teach you

•

Ignore the instructions at the very bottom today (pre-test only)

•

The purpose of this test is to see how well we taught you (post-test only)

•

Please try your best to complete each answer, though you might not know some answers

•

On the back, please write one thing you liked about TIMMI, one thing you didn’t like about TIMMI, and one sentence about how TIMMI will help you in the future (post-test only)

•

We cannot help you with this test, but you’ll learn everything on there in TIMMI

Students were given around five to ten minutes to complete the assessment. No questions related to the assessments were answered, but students were encouraged to try their best. Assessments were similarly given to a control group of ninth graders in the same school district (attending the Livingston High School). These students have essentially equivalent backgrounds (in the areas covered by TIMMI) as the middle school students in this study. Because of practical limitations, the post-test was administered immediately following the pre-test.

Assessment Details There were three versions of a pre/post assessment that were randomly administered. The assessments (found in the following pages) consisted of four questions, assessing knowledge in budgeting/decision making (Question 1), credit and debt (Question 2), investing (Question 3), and the general idea of money management (Question 4). During the post-test, students were asked to provide feedback about the quality of the sessions, as well as the motivation for financial literacy they received from the sessions. This qualitative feedback was not graded but will be analyzed in this report. The rectangle at the bottom of the page contains metrics for each student. NPX signified absence on Day X, either “Pre” or “Post” was circled for the correct assessment, the Cluster Record for this teaching series was “Her1,” and the ID of each student was a random unique number.

TIMMI, An Investment in Knowledge Curriculum

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Assessment A

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Assessment B

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Assessment C

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Scoring Details Each objective question was scored out of a 4-point scale. The tests, then, were scored out of 16 points in total. The guidelines for scoring are as follows: (Note that students could have responded in a way that would gain them more than 4 points, but we capped each questionâ&#x20AC;&#x2122;s score at 4 points. So a 6-point answer and a 4-point answer would both receive 4 points.) Assessment A 1) Think about a recent decision you made. Give & explain what the opportunity cost was. a. 2 points each for identifying the decision, identifying opportunity cost, and explaining the opportunity cost b. 1 point for very comprehensive details on decision, like alternatives available 2) Name one main factor involved in getting credit and explain how you can improve it. a. 3 points for identifying a factor thatâ&#x20AC;&#x2122;s clearly one of the three Cs, or 2 points for identifying a factor that can be somewhat linked to one of the three Cs b. 2 points for identifying the improvement of this factor in a clear and correct manner, or 1 point for partially doing so 3) How are risk and returns related? a. 4 points for identifying that greater risks can yield greater returns b. Partial credit (3/2/1) for an inconsistent answer Assessment B 1) Why is creating a budget important? What does it allow you to do? a. 4 points for identifying that budgets help balance income and expenses, or for identifying that budgets allow for the allocation of savings 2) How do you benefit from having good (or great) credit? a. 4 points for identifying that it makes it easier/possible to get loans or for identifying that it grants a lower rate on debt 3) Explain the difference between blue-chip and penny stocks. Which do you prefer? Why? a. 3 points for identifying that blue-chip stocks are safer than penny stocks (or similar) b. 3 points for identifying that blue-chip stocks are from large companies and penny stocks are from smaller companies (or similar) c. 1 point for identifying that blue-chip stocks are almost certainly safe, and that penny stocks are almost always very risky d. 1 point for explaining preference with desire to take risk or stay safe (or similar)

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Assessment C 1) What do you think is the most important step of the decision-making process, and why? a. 3 points for identification of a step in the decision making process (Identify the Problem, Brainstorm Alternatives, Evaluate Alternatives, Decide, and Reflect), or 2 points for identifying a step that somewhat relates to the decision making process b. 2 points for strong explanation of why, or 1 point for inconsistent explanation 2) Who does the system of credit benefit, and how? a. 2 points for identifying borrowers b. 2 points for identifying lenders c. 2 points for explaining that borrowers can gain convenience, or the power to make large purchases they wouldn’t be able to immediately afford d. 2 points for explaining that lenders profit from interest 3) What’s the golden rule for choosing a stock to invest in? a. 4 points for identifying “if you don’t know what the company does, don’t invest in the stock!” or 3/2/1 points based on partially identifying this rule b. If the wrong rule was identified, 3/2/1 points for identifying a similarly important “golden rule” for stocks All Assessments 4) Do you currently (plan to) practice smart money management? If so, how and why? a. 1 point for saying “yes” b. 2 points for consistent explanation of how that makes sense and is realistic (such as a strategy taught in TIMMI) c. 2 points for consistent explanation of “why” d. 1 point for very strong explanation overall

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Results Growth Between Pre-Tests and Post-Tests The following graphs show the overall progression between the pre-assessments and postassessments.

The figure on the left examines the average score on the pre-test and the average score on the posttest for students who took both tests (in essence, students who were there on Day 4). There was an average 6.583 point increase between the assessments, which was statistically significant (p < .0001). The figure on the right examines the average score on the pre-test and the average score on the post-test for all students (including 10 who did not take the post-test). There was an average 7.078 point increase that was statistically significant (p < .0001) between these averages. While this reveals that the average score on the pre-test was decreased by the inclusion of the students who did not take the post-test, we do not find this concerning. Of the ten absentees, we found an even split between students who lost interest (or were never interested), and students who were simply absent on Day 4.

The average pre-test score of students who took the post-test was 1.683 points greater than the average pre-test score of students who did not take the post-test, but this difference was not significant.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

In the control group of students, we found no change between the average score on the pretest and the average score on the post-test. This, of course, gives us confidence that the increase was not due to any experience involved with taking the assessment.

Converting the pre-test and post-test differences to percent values, we find that students increased an average of 41.1% (when including only students who also took the post-assessment) or an average of 44.2% (when including all students).

Another way to examine the change is to identify the respective proportions of students who scored better, worse, or with no change on their post-assessments compared to their pre-assessments.

87.5% of our students who took both assessments did better on the post-test than the pre-test. In the control group, 30% did better, whereas 26.7% saw no change and 43.3% did worse (this is close to the even split between better and worse we would expect if there is no difference).

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

A Better Metric – Change Index Because some students attended fewer than the four sessions offered, we must standardize the difference with the number of days students attended. In other words, it would be very reasonable to expect that a student who attended all four sessions would learn more than a student who only attended two sessions. The following graph reflects the post-test and pre-test differences for students, grouped by the number of TIMMI sessions they attended. Curiously enough, it seems that students who attended TIMMI for only three sessions learned more than students who attended TIMMI for four sessions. By ANOVA analysis excluding the one student who attended only one session, this difference was significant (p < .05).

However, we will hypothesize that attending TIMMI for an extra session does not make students learn less. Instead, upon review of the original data, we found that most (6) of the students who missed only one session were absent from the first session. On that day, February 24th, there was a popular (relatively speaking) math competition after-school, the “New Jersey Math League.” As students who seek out math competitions for fun are typically smarter, we accordingly believe that there was a non-random difference in intelligence/learning ability here that explains the discrepancy. Going back to the initial idea of standardization, we chose to divide the student’s difference in score by the number of sessions the student attended to create the change index. The average change index was 2.219, and the 25th and 75th percentiles were 1.250 and 3.917 respectively. This change index is very clearly significantly different from a “no difference” or 0 change index (p < .0001).

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

In the control group, the average difference was 0 (first quartile was -2 and third quartile was 2). This makes it very convenient to say that the average change index was also 0, since there is no regard to the number of sessions attended (we swear this was a complete coincidence).

Below we highlight the distribution of differences in the students from Heritage and the control group, as well as the change indices of the students from Heritage Middle School.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

After converting the change index, what we perceive as a stronger metric, to percentages, we found an average 55.5% change in score between the pre-assessment and post-assessment. This suggests that the previously found 41.1%/44.2% increase in average score may not be the best measurement of change, as it does not take into account absences of students (which we can infer to affect their learning). Thus, we will keep this 55.5% change as our best measurement of student progress. The histogram of change indexes converted to percentages is shown below.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Comparing the Assessment Versions The three versions of the assessment all contained different questions for Q1, Q2, and Q3. While the different questions corresponded to the same subject area (Budgeting/Decision Making, Credit/Debt, or Investing), they may not necessarily have been at the same difficulty level. Here, we evaluate the difficulty of each version of the test by comparing its average score relative to those of the other versions.

The results from Heritage are on the left; the control group is on the right. The pre-assessments are on top, and the post-assessments are seen on the bottom. There was no consistent trend of difficulty between the assessments. For example, C seems the hardest for control group students taking the post-assessment, but seems easiest for Heritage students taking the post-assessment. When the average scores were compared with ANOVA, the difference was not found to be significant anywhere except for Heritage students taking the postassessments. From a qualitative standpoint, the assessments seemed to be equally difficult. The questions used were all around equally covered/reinforced in the curriculum. The quantitative assessment seems to differ when taken at any single assessment point (e.g., one group of students for the post-test), but holistically reviewed as a whole reveals no underlying consistencies in difficulty.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Performance Differences Between Genders The short version is that while females outperformed males on both the pre and post assessments, both genders had the same average change index. Females had an average score of 4.750 on the pre-test, and males had an average score of 2.556 on the pre-test. Females scored around 2.194 points higher, and this difference was close to being statistically significant (p = .0579). On the post-assessment, females had an average score of 12.83, 4.333 points greater than the male average score of 8.500 (statistically significant, p < .05).

The average change index for males was 2.146, while the average change index for females was 2.292 (no significant difference). This was somewhat unexpected by our student teachers â&#x20AC;&#x201C; almost all of the â&#x20AC;&#x153;rowdyâ&#x20AC;? students were boys, but it seems like they were still learning (or their peers were compensating for them)!

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Performance on the assessments was relatively uniform across genders in the control group. And as we would expect, the pre-test/post-test differences between the genders were both close to 0. Females had an average score of 5.444 on the pre-assessment, .8889 points lower than the male average score of 6.333. This difference was not significant, and neither was the difference on the post-assessmentâ&#x20AC;&#x201D;females scored .3810 higher at 6.286 points and 5.905 points respectively.

Females had an average .8889 point increase in score, while males had an average .3810 point decrease in score. Both are evidently very close to 0 (the graph shows pancakes), and the difference between males and females was not statistically significant.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Individual Question Analysis In this section we will attempt to individually analyze each question and its pre/post difference. Each question, across the three versions of the assessment, targeted one specific area/module/day covered in TIMMI: budgeting/decision making, credit/debt, investing, and the general concept of money management (this last â&#x20AC;&#x153;areaâ&#x20AC;? was not specifically taught, but was included in all of TIMMI). The graphs of scores on the individual questions are shown below, for students who took both assessments as well as all students in general. The graph for the control group is on the next page.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

For the pre-test, when all of the questions are analyzed together, it seems that Questions 2/3/4 were similar in difficulty and progress, while Question 1 was comparatively easier initially. Two versions of Question 1 asked students about making decisions, so it is likely that those concepts were taught in classes outside of something like TIMMI (whereas the three Cs of credit or blue-chip stocks would probably not be). The statistical evidence supports this similar difficulty between Questions 2, 3, and 4 in the pre-tests: ANOVA finds their average scores to not be significantly different. The relative easiness of Question 1 in the pre-test is also supported by ANOVA through being statistically significant (p < .0001 for Heritage samples, p < .001 for control group). In the post-tests, there seems to be a similar trend. Question 1 has a statistically significant higher score in both the Heritage and control group samples (p < .05 for Heritage, p < .01 for control), so it seems to retain the quality of being the easiest on the assessment. The rest of the questions have average scores that are not significantly different from each other, by way of ANOVA. Analyzing the per-question differences, we see the trend that Question 1 had the least improvement, likely coming from its relative easiness. We see the trend that Question 3 saw the greatest progress, and that Questions 2 and 4 saw less improvement, but these differences are not statistically significant.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

In the control group, we do not see any difference (the graph is pancakes!) between the pre/post assessments on an individual question basis as well. We will now analyze the individual questions.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Question 1 – Budgeting and Decision Making The three versions of the first question were: •

Think about a recent decision you made. Give & explain what the opportunity cost was.

•

Why is creating a budget important? What does it allow you to do?

•

What do you think is the most important step of the decision-making process, and why?

These questions assessed proficiency in making decisions, and understanding the concept of budgeting. They tie in to Day 1 of TIMMI, which focused on budgeting and decision making.

As the above figures reflect, there was a statistically significant (p < .01) improvement on this question. The answer increased on average 1.042 points (26.1%), and the overall average for all students increased 1.368 points (34.2%). In the control sample we saw a mean decrease of .2667 points (6.67%) but this was not statistically significant.

The improvement on this question from the Heritage students was the smallest, but we refer again to the hypothesis that questions on decision making were relatively easier initially (because of exposure outside of a money management class like TIMMI).

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Question 2 – Credit and Debt The three versions of this question were: •

Name one main factor involved in getting credit and explain how you can improve it.

•

How do you benefit from having good (or great) credit?

•

Who does the system of credit benefit, and how?

These questions assessed knowledge in credit and debt, and tie in to Day 2 of TIMMI, which focused on topics like obtaining and using credit.

Here, we see strong evidence of a gain in knowledge on credit. While in the control sample, the average score increased .2333 points or 5.83% and the difference was not at all significant, there was an average score increase in the Heritage students of 1.708 points or 42.7% (and if we consider all students, an average increase of 1.770 points or 44.3%). This gain was statistically significant (p < .001 for students who took both assessments, p < .0001 for all students).

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Question 3 – Investing The three questions on investing were: •

How are risk and returns related?

•

Explain the difference between blue-chip and penny stocks. Which do you prefer? Why?

•

What’s the golden rule for choosing a stock to invest in?

Topics directly addressed by these questions include risk, types of stocks, and investing in stocks, which were all parts of the lesson on Day 3.

We see the strongest improvement of all questions here: 2.167 points and 54.2% within the students who took both the pre and post assessments, and 2.221 (55.5%) within all students. Both gains were statistically significant at p < .0001. In the control group, the mean gain was .1 points or 2.5%, and was evidently not significant.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Question 4 – General Ideas of Money Management The same question was posed to all students: •

Do you currently (plan to) practice smart money management? If so, how and why?

This question addresses three important concepts covered consistently throughout TIMMI: the idea of managing money, the practices behind managing money, and the motivation for managing money. The material from any day could be tied in here – saving in bank accounts and creating budgets were both equally valid ways to practice money management.

We see solid progress here. The control group saw a gain of .0333 points, or 0.83% that was not statistically significant. On the other hand, we found that Heritage students scored on average 1.667 points higher on an individual basis, or 41.7% better, and 1.721 points (43.0%) better as a whole. Both of these differences were significant (p < .0001).

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

One Last Thing About Absences These graphs demonstrate the pre-test to post-test improvement for each question, but divided into two groups. The left two bars represent those who were present for the corresponding day (so Day 2 for Question 2, etc.), and the right two bars represent students who were absent during the lesson that addressed the question. Upon inspection of the graph for Q3 performance, we clearly see that students who attended Day 3 had a significant (p < .0001) improvement between their pre and posttests, while those who were absent did not have statistically significant improvement. This is what we would expectâ&#x20AC;&#x201D;if a student is not there for the lesson, he/she will not learn about that topic. However, looking at the graphs for Q1 and Q2 performance, it seems like being present at the lesson did not change learning in that area. What does this suggest about TIMMI? Well, it might lead to the erroneous conclusion that TIMMI was of no benefit (we have shown this is false). But what we really see from this is that TIMMI teaches financial literacy holistically and comprehensively â&#x20AC;&#x201C; the material for one session might be repeated or expanded on in other sessions. (This also helps us explain why being absent from Day 3 limited Q3 performance, as there would only be one session after Day 3 to review/reexamine the material of Day 3.)

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Feedback In general, students overwhelmingly appreciated TIMMI for its “fun” and engaging programming. Many mentioned the interactive examples, activities, and demonstrations that both made the learning more exciting (one student “felt like [she] actually enjoyed learning) and connected it to the real world. The Kahoots were incredibly popular and well-liked as well. One student said she did not like “how [few] demonstrations there were” – we agree that a reasonable amount more of demonstrations would have made the curriculum even more engaging. Beyond TIMMI’s programming and style, many liked its content that would be important for their futures. Many specifically mentioned that they enjoyed learning about stocks, which seems to come from the real world application in being able to make money. One student mentioned budgeting, which we guess can be exciting. Aside from the core programming, students also mostly seemed to appreciate the food, candy, and other motivators (like $5 gift cards to Starbucks for Kahoot winners); one student called these “question encouragement.” One student, however, disliked the prizes, perhaps because of the inherent increase in competitiveness. Another student disliked the food, citing that “most students only came here for it”… well, whatever gets our students in the door, right? Some things that students reported wanting to change were explanations that “were confusing with big words,” the lack of paper to take notes on, and the fact that TIMMI only lasted for four sessions. We will certainly offer paper in the future for students to take notes on, and will consistently review our curriculum to make sure it is taught at an appropriate level (most students evidently found TIMMI understandable, however). One student asked that we “emphasizing more in stocks,” which can be a possibility. However, we shy away from trying to provide specific tips or advice for investing stocks – which are skills that take years of experience to learn. We retain index funds as our best investment advice. Logistically, we received “dislikes” for starting too late, and not being very organized. On average, we started 5-10 minutes after the scheduled time (which was 5 minutes after school ended) – our bottleneck was traffic in the commute from Livingston High School to Heritage, as well as delays with suppliers. We take heed of this advice and will work towards organization and punctuality. Overall, students had fun attending the “great lessons” (a student’s words, not ours) of TIMMI. They appreciated learning about a highly practical topic that would be important in their futures, liked that they were taught money management at an early age, and were entertained by our highly responsive teaching (as well as the high school student teachers, one attendee noticed).

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Conclusions Successes It is clear that the Teaching Investment and Money Management Initiative successfully taught and shared money management at the Heritage Middle School in this series. This success is evidenced by significant quantitative results—an increase in financial literacy in 87.5% of students, and an average assessment score gain of 55.5%—as well as consistently positive qualitative feedback. Our largest gain, however, does not come from improved answers on a specific question or subject area. Rather, we see the recognition of the importance of financial literacy as the single most important concept we can share. The American Dream did not die because someone did not know the decision making process, or because someone did not know what blue-chip stocks were, or even because someone did not know what a credit score was. It’s gone because there was the general attitude in our society that money management did not matter. We believe that we greatly succeeded, more than any numbers could show, in this regard. Our thirtyfour students saw high-schoolers come excited to spend an hour teaching them. They experienced an entertaining yet educational atmosphere centered on something not covered in school. They may have wondered why a local pizzeria would donate four large and delicious pies every week, for what noble cause. And these students’ parents and friends may have asked what “TIMMI” was all about. And those living in the greater Livingston area may have looked into “TIMMI” after reading about this in the local news. So while we did teach about topics like budgeting, credit, and investing, our most important legacy from this is that we showed our society how important we think financial literacy is. In doing so, we find hope that the American Dream may one day reemerge as a reality.

Study Improvements Conducting the control study on a group of similar middle school students from the Heritage Middle School, over the same four week period, would have been theoretically most ideal. However, practical limitations here (a significant one being willingness to participate) suggest that our control study on the group of ninth graders was an acceptable alternative. Being able to standardize the different assessment versions would also be very helpful in comparing performance and differences. We will strive to do so as we collect more results, but there exists large obstacles in not being able to administer test versions the same way a drug approval study would administer medication (we don’t have hundreds of students at equal intelligence/background/etc.). Besides, the various assessment versions were shown to already be moderately similar earlier.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

One very practical question we can try to answer is why female students scored consistently higher than their male counterparts on the pre/post assessments, even though they exhibited no greater growth. This discrepancy was not present in the control group. Was it because female students put in greater effort to the assessments? Because they, for some reason, started out with greater knowledge in this area? Or because of any number of other reasons?

Teaching Improvements As this series of TIMMI started the Tuesday after a week-long vacation, it was hard to coordinate when and how to let Heritage students know about this exciting opportunity. In the future, we look to advertise and promote TIMMI sessions significantly more. We will take practical, logistical measures to improve our teaching, including building in more time for transiting to the school. We will also take a look at and revise our central curriculum, to make sure it is as interactive and inclusive as it can be.

Notes on the PARCC On the day of the final session of TIMMI, Heritage Middle School students took the infamous Partnership for Assessment of Readiness for College and Careers (PARCC), a new standardized exam in New Jersey. Testing tension was high with exams done on computers for the first time, and numerous complaints that the tests were bad in some way, shape, or form—the Livingston school district actually made the news for having an unprecedented 1,000+ students who refused to take the assessment. One teacher at Heritage said it was “surprising” how well the students learned and behaved during Day Four of TIMMI, compared to how they were in classes during school that day. We believe that taking PARCC caused students to learn less during TIMMI than what they would have without having taken PARCC.

In Summary We have found that students at the Heritage Middle School who participated in the financial literacy series offered by the Teaching Investment and Money Management Initiative tremendously gained money management skills and knowledge. Our results demonstrated that TIMMI’s students became significantly better equipped to understand today’s financial world, manage and grow money, and work towards achieving the American Dream.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Appendix: Original Data The original student response matrices, only identified by random IDs, are presented here. Below each studentâ&#x20AC;&#x2122;s ID is the studentâ&#x20AC;&#x2122;s gender, as well as the days the student did not participate in TIMMI (NP3 would indicate the student missed Day 3). To the right of the original student answers is the grade assigned to that answer, and the bottommost row sums the individual scores. The Heritage Middle School students are identified by IDs 1 through 34, and the control group students are identified by IDs C1 through C31 (excluding C30). The original tests are viewable at http://timm-i.org/progress/research/her1.

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

The following are the original student response matrices of the students from Heritage.

Pre–C

NP2 NP3

1. How will it affect me? Is this important? 2. (Blank) 3. (Blank) 4. No

1 0 0 0

Post Assessment Not Taken

Pre-Test Total: 1

Post-Test Total: NA

Change Index: NA

1. Identify the problem 2. The spender F 3. Find the least expensive one 4. No 1. My recent decision was whether to go to the movie or study. In the end, I chose to study. The opportunity cost is that I didn't get to watch the movie and hangout with my friends. 2. One factor involved in credit is you 3. Risks and returns are related because the bigger the risk, the bigger the reward and the smaller the risk, the smaller the reward. 4. I current plan to practice smart money management by putting any money I get in my bank account Feedback: One thing I liked is that you taught money management at a young and early age. Also, it involved examples and activities to help you understand. One thing I didn't like is that some things you explained were confusing with big words. Pre-Test Total: 6 Post-Test Total: 11 Change Index: 1.25

3 2 1 0

0 0 0 0

Pre–C

Post–A

Pre–C

NP2 NP3 NP4

Pre-Test Total: 0

1. (Blank) 2. (Blank) 3. (Blank) 4. No

4 0 4 3

Post Assessment Not Taken

Post-Test Total: NA

Change Index: NA

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

1. I could make more money if I invest wisely 0 2. (Blank) 0 M 3. (Blank) 0 4. No 0 NP2 1. I watched TV instead of studying. I might not get a good grade. Not studying. 4 2. One main factor about getting involved in credit is that you pay money at the 3 month for everything you buy. You can improve by paying every month. 3. The bigger the risk the higher the return. 4 4. Yes, I want to put money in stocks because, even though it would be risky, I 3 can make more money Feedback: I liked that it was interactive and fun. I disliked the prizes. TIMMI can make me a great investor. Pre-Test Total: 0 Post-Test Total: 14 Change Index: 4.67 Post-A

Pre–A

Pre–B

NP2 NP3 NP4

1. (Blank) 2. (Blank) 3. You invest and when you get in return 4. No, not really.

0 0 2 0

Post Assessment Not Taken

Pre-Test Total: 2

Change Index: NA

1. Choosing the right decision and reflecting on it. 2. Everyone? F 3. Seeing which stock does the best 4. Not really, because I don't know what it is. If I get exposed, I would. 1. It helps you manage your $. You need to know how much you spend and make sure that you have money saved up. You need to know how much you spend in what. 2. The bank can trust you more. They think you're more reliable and that you have good character. They can trust you you could have $ in times of need. The bank can help you. 3. Blue chips are big companies such as Apple and Samsung. They are much riskier but you get more $ when you invest. Blue-Chips are small companies. They are not big and they are very risky. May not set $ back. I prefer penny because I could get lots of $ and blue chips are lots of $ to invest in 4. Yes, I would like to manage save money. Maybe invest in stocks to help may w/ college Feedback: Like - Learn about stocks. Don't - No papers to take notes on. Pre-Test Total: 8 Post-Test Total: 15 Change Index: 1.5 Post–B

Pre–C

Post-Test Total: NA

4 2 2 0 4 4

3 3

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

1. Creating a budget is important because it allows you to have a successful business and invest money. M 2. You can benefit from having a good credit if you know most facts. 3. (Blank) 4. No. However I might want to start practicing money management 1. Creating a budget is important because it allows you to show your savings. This will allow you to have a great business. 2. You benefit from having good credit when you pay your debt daily. 3. Blue chip stocks are not that risky than penny stocks. Blue chips are better so you won't lose too much money. 4. I currently do not at this time, but would really like to do so in the future. Feedback: TIMMI helped me learn a lot. This club showed me how important investing is. One thing I liked learning about was investment. One thing that could improve could be emphasizing more in blue chips and stocks. This will lead me to a really successful life. Pre-Test Total: 1 Post-Test Total: 8 Change Index: 1.75 Post–B

Pre–B

0 0 0 2 2 4 0

1. (Blank) 2. (Blank) M 3. (Blank) 4. I currently plan to practice smart money management because I wish to use these skills for the future. Only buy what is necessary, don't spend money on wants. 1. Creating a budget is important because 2. (Blank) 3. The difference between blue-chip and penny stocks is that penny stocks are extremely risky while blue-chips are owned by large companies 4. (Blank) Feedback: I wish there wouldn't be food. Most students only came here for it. Great lessons. Pre-Test Total: 2 Post-Test Total: 4 Change Index: .5

0 0 0

4 0 4 1

Post–B

Pre–A

Post–C

Pre–A

1. I made the decision to sleep later. The cost was I was tired in the morning 2. Making mad money M 3. Risk for returns 4. Yes. About 2.15 NP2 1. You have to identify the problem, because without that, there would be NP3 nothing else. 2. It benefits the banks because they earn interest 3. To make money 4. Yes, because then I'll make it rain Feedback: Liked - Food. Disliked - Not anything really, had few disagreements. Future - Be rich. Pre-Test Total: 9 Post-Test Total: 12 Change Index: 1.5

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

2 0 0 4 0

4 4 3 1

Pre–B

NP4

1. Know how much $ spent. Create savings. 2. No extra payments. Less interest costs 3. (Blank) 4. Yes, I am a boyscout and achieved the personal management merit badge

4 4 0 1

Post Assessment Not Taken

Pre-Test Total: 9

Post-Test Total: NA

Change Index: NA

1. Creating a budget is important so you will not go in debt and you would easily 3 manage your money F 2. (Blank) 0 3. (Blank) 0 4. I have a bank account and try to save and invest my money for later in my life 2 1. I think the most important step of the decision-making process is reflecting off the results because after you reflect off the results you will realize/know if 4 your decision was successful so you will learn and know for next time. 2. The credit benefit 0 3. The golden rule is to pay your debt as early/fast as you can. Also, blue chips is 3 the best way to invest in. 4. Yes, because this will greatly affect me in the future, and affects our life in a 3 great and big way. Feedback: I liked... how I was able to learn a variety of things that will help me in the future. I disliked how Pre-Test Total: 5 Post-Test Total: 10 Change Index: 1.25 Post–C

Pre–B

1. Being able to create the steps. 0 2. (Answer) 0 F 3. (Answer) 0 4. (Answer) 0 1. I think the best step is analyzing your other options. Doing this will help you 4 choose the best decision. 2. Credit benefits the person by allowing him/her to pay payments at a more 4 flexible time. If someone can't pay something @ a certain deadline, he/she can 3. The golden rule is 0 4. Yes, I can save my money. & spend it wisely. 2 Feedback: I like the games/food & examples. I do not like how it only lasted 4 days. It will be important to my future by teaching me how to manage my money. Pre-Test Total: 0 Post-Test Total: 10 Change Index: 3.33 Post–C

Pre–C

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

1. Creating a budget is important because it gives you a point in which you have to manage the amount of cash is being spent. A budget allows you to keep track 4 of your money. F 2. Later on in life if it is ever necessary you can get a loan because the bank 4 trusts you. 3. (Blank) 0 4. (Blank) 0 1. To analyze and evaluate your brainstorms. This is the most important step 4 because you are checking if your ideas will be successful or not. 2. Credit benefits the people involved with the credit. This is because if you have good credit and you need to buy a car or something the bank will know your 3 trustworthy because you always give your payments on time 3. Make sure you are aware of what the company produces before you invest in 4 it. 4. I do. This is because I am investment in a back to have emergency money. 4 Feedback: I liked that there was question encouragement, (each time you answered a question you got candy). I did not like how little person demonstrations there were. It will teach me about stock investments and how to choose what stocks to invest in. Pre-Test Total: 8 Post-Test Total: 15 Change Index: 1.75 Post–C

Pre–B

Pre–C

NP2 NP3 NP4

1. (Blank) 2. (Blank) 3. (Blank) 4. (Blank)

0 0 0 0

Post Assessment Not Taken

Pre-Test Total: 0

Post-Test Total: NA

Change Index: NA

1. The most important step of the decision making process is the step where you evaluate your options, because this is where you see which is better, and the F consequences of each. 2. (Blank) 3. (Blank) 4. Yes, by investing money and saving. 1. Evaluating your other decisions. This is because if you dont look at all your different choices you won't know which to choose or if there is another path you can take. 2. The system of credit benefits you because credit cards are easy to use and carry, also you can get miles on an airplane/points. No interests if you pay the bill on time. 3. The golden rule for choosing a stock to invest in is invest in a blue chip stock. Only buy what you can afford to loose. 4. Yes by making a buget for spending, putting my money in a bank account, and saving. I will do this so I can earn more money on intrest and saving. Feedback: I liked learning about the stocks. Pre-Test Total: 6 Post-Test Total: 15 Change Index: 2.25 Post–C

Pre–C

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

4 0 0 2 4 4 3 4

1. The most important step is making the decision, because this determines the outcome. F 2. The system of credit benefits the bank because they earn interest. 3. Find a stock that's increasing. 4. Yes I do, I manage my money with my debit card to prepare for the future. 1. I made the decision of going to school or not after my doctor's appointment. By not going to schol the oppurtunity cost was to not be able to take my quiz and miss work. 2. Character is a main factor involved. You can improve character by signing with your parents card to make it look like you have good character when getting a credit card. 3. Risk and returns are related because a bigger risk means less return. If you invest in a penny stock you might not earn as much and might loose money. 4. Yes, I plan to practice smart money management with my debit card so I learn for the future. Feedback: I liked the encouragement with the candy. Pre-Test Total: 12 Post-Test Total: 10 Change Index: -.5 Post–A

Pre–C

NP3 NP4

1. (Blank) 2. (Blank) 3. (Blank) 4. (Blank)

4 4 3 1 4 4 1 1

0 0 0 0

Post Assessment Not Taken

Pre-Test Total: 0

Pre–A

NP4

Post-Test Total: NA

Change Index: NA

1. My decision was 2. (Blank) 3. (Blank) 4. No

0 0 0 0

Post Assessment Not Taken

Pre-Test Total: 0 36

Post-Test Total: NA

Change Index: NA

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

1. (Blank) 2. (Blank) F 3. (Blank) 4. I planned to practice smart money management because when I get older I NP2 am able to use my money wisely 1. Once I had to decide whether to hangout with my friends or study for a social studies test. I stayed home a studied and got a good grade. 2. (Blank) 3. (Blank) 4. Yes, so that in case of an emergency, I would have money to pay for things that have gotten damaged. Feedback: TIMMI is good because they help with budgeting. One dislike was the class would start kinda late. Pre-Test Total: 3 Post-Test Total: 7 Change Index: 1.33 Post–A

Pre–A

1. So you know how much money to spend and won't overkill it. Save money for the future. M 2. (Blank) 3. (Blank) 4. (Blank) 1. Making sure you have someone that you can trust and a balence buget. 2. It benifits you because it makes you have less money in your pocket. 3. Pays fairly 4. I will by talking with local bank and learn more. Because Feedback: The best thing was the prizes and contest to make thing fun. NO DISLIKES Pre-Test Total: 4 Post-Test Total: 8 Change Index: 1 Post–C

Pre–B

Pre–A

NP2 NP3 NP4

1. I went to Delias because they were going out of business and they were having a sale of 70-90% off 2. (Blank) 3. (Blank) 4. (Blank)

0 0 0 3 4 0 0 3

4 0 0 0 0 4 2 2

2 0 0 0

Post Assessment Not Taken

Pre-Test Total: 2

Post-Test Total: NA

Change Index: NA

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

M NP1 NP4

Pre–A

1. I thought about coming to this class. The cost of coming here was learning knowledge. 2. (Blank) 3. When you take a risk you will get what you did returned back to you. 4. When I get money I save and do not blow it all in the same week. I always have to have $100 in my wallet.

2 0 2 2

Post Assessment Not Taken

M NP1

Post–A

Pre–A

Pre-Test Total: 6

Feedback: Pre-Test Total: 0

Change Index: NA

1. (Blank) 2. (Blank) 3. (Blank) 4. (Blank) 1. Decision: Studying or doing an activity. Opportunity cost: studying. 2. (Blank) 3. (Blank) 4. (Blank) Post-Test Total: 4

Post–C

0 0 0 0 4 0 0 0

Change Index: 1.33

1. (Blank) 2. (Blank) M 3. (Blank) 4. (Blank) NP1 1. Finding the problem because it starts the process and helps you complete the other processes. 2. It benefits the bank because if you don't pay the money, the bank collects more money from you, the longer you don't pay. 3. The golden rule is to know the stock that you are investing in. 4. Yes, I plan to practice smart money management. I will save money, pay bills on time, and invest in good stocks. Feedback: I love TIMMI Club and will be happy to come back next year. Pre-Test Total: 0 Post-Test Total: 15 Change Index: 5 Pre–C

Post-Test Total: NA

0 0 0 0 4 4 4 3

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

1. It helps you save money for more expensive things such as a college fund. If 4 you save enough, it will allow you to afford more expensive things. F 2. (Blank) 0 3. (Blank) 0 NP1 4. (Blank) 0 1. The most important step is to identify the problem. If one doesn't identify the 4 problem, he will not realize that he is not saving $ efficiently. 2. The system of credit benefits the consumer because they don't have a limit as 4 opposed to debit. 3. Choose blue chip stocks because they are not as risky as penny stocks. 4 4. Yes. I should start investing in blue chip stocks, which will help me gain 4 money. Feedback: I really liked how TIMMI incorporated examples in real life. The only thing I disliked is that the lesson wasn't long enough. I want it longer! Pre-Test Total: 4 Post-Test Total: 16 Change Index: 4 Post–C

Pre–B

1. A budget is important because it allows you to see where your money is coming and going. It also helps you not overspend. F 2. It helps you because then you are allowed to take loans for things like a car & a house. NP1 3. (Blank) 4. (Blank) 1. Creating a budget is important because it allows you see where you should spend your money. It is important to help you manage your money. 2. You can benefit from good credit because it allows you to take large loans if you need, because the bank trusts you. 3. Blue chip stocks are stable stocks from well known companies, & penny stocks are from unknown companies & are unstable. I prefer blue chip stocks because they are bit more reliable. 4. I plan to practice smart money management by creating a budget to see where I should spend my money, & invest in blue chip stocks. Feedback: I liked learned about stocks. Pre-Test Total: 8 Post-Test Total: 15 Change Index: 2.33 Post-B

Pre–B

Pre–A

Post–C

NP1

Feedback: (Blank) Pre-Test Total: 3

1. I went rollerskating with friends and I had to use my own money. I spent more money than I was supposed to. 2. (Blank) 3. (Blank) 4. (Blank) 1. Evaluating your decision because you then know if you made the right decision. 2. It lets you pay all the money you spent in that month. 3. If you don't know it, don't invest in it. 4. Yes, I have an alowence and I plane to start saving up. Post-Test Total: 14

4 4 0 0 4 4 4 3

3 0 0 0 4 3 4 3

Change Index: 3.67

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

1. I played basketball yesterday, and I got a good excercise. 2. (Blank) M 3. (Blank) 4. (Blank) NP1 1. I chose to opt out of the PARCC. I chose to maybe loose some knowledge. 2. Paying interest on time. 3. If you take a risk, you can get a good return. 4. Yes, I plan to save money because I want to have a happy and healthy life. Feedback: TIMMI was fun. No dislike. Pre-Test Total: 2 Post-Test Total: 15 Change Index: 4.33 Post–A

Pre–A

Pre–B

Post–A

NP1 NP3

Feedback: (Blank) Pre-Test Total: 4

M NP1 NP2 NP4

Pre–B

1. Creating a budget is important so you may limit yourself and not spend too much. 2. I almost always benefit from good credit so I may never have trouble. 3. (Blank) 4. (Blank) 1. I was given the choice of making a saving account and to add more than what I owe, after a year I could pay them back. 2. (Blank) 3. (Blank) 4. (Blank) Post-Test Total: 1

2 0 0 0 4 3 4 4

3 1 0 0 1 0 0 0

Change Index: -1.5

1. Creaing a budget allows for you to manage your finances to safely work and budget your own money. 2. You can benefit when looking for jobs as well as earn some revenues back. 3. (Blank) 4. Yes, I currently practice as I currently earn money online for different services and I've learned to setup things like Paypal and such.

2 1 0 1

Post Assessment Not Taken

Pre-Test Total: 4

Post-Test Total: NA

Change Index: NA

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

1. Creaing a budget is important because this way, you don't go over the amount 4 of money than what you have. F 2. (Blank) 0 3. (Blank) 0 NP1 4. I save up my money and only keep it for emergencies. 2 NP2 1. The most important part of the decision-making process is identifying the 4 problem. Based on the problem, you can think about solutions and alternatives. 2. The system of credit benefits you because you can pay the full amount at the 4 end of the month. 3. Don't invest in a company that you don't know. 4 4. Yes. I plan on keeping an emergency fund. I can save up a little money every 4 week and it will add up. Feedback: Liked: It was appealing to kids like us. I felt like I actually enjoyed learning. Disliked: Nothing? Maybe you can be more organized. Future: I now know how to handle money in a responsible way. Pre-Test Total: 6 Post-Test Total: 16 Change Index: 5 Post–C

Pre–B

1. I think that the most important step is thinking about the good aspects and the bad aspects of the situation at hand because it allows the decider to see all 2 F the views before deciding. 2. It benefits the government because the government is always given the money NP1 owed in the end. It also benefits the people because they are given time to pay 0 NP2 their money. 3. Choosing a stock whose company has potential. Looking at the stock and 4 tracking it before you buy/purchase the stock. 4. I do plan to because 1 1. A budget is important because it allows you to keep track of your money and 4 lets you save money in the long run. 2. You benefit from having good credit because in the end you do not have to 4 pay interest. 3. Blue-chip stocks are safe stocks, large company names such as Apple or Microsoft. Penny stocks are risky stocks, small, growing companies. I prefer 4 blue-chip investments because they are safer and you have an increased chance of getting more money back. 4. I currently practice smart money management because I do have a budget and 3 I keep my money in a bank account without risk. Feedback: I like TIMMI because it was fun and interactive. R really there is nothing you can do better. TIMMI will teach me how to manage my money better and know both the advantages and disadvantages of investing in stocks. Pre-Test Total: 7 Post-Test Total: 15 Change Index: 4 Post–B

Pre–C

1. (Blank) 2. (Blank) M 3. (Blank) 4. (Blank) NP1 1. If I ate pizza, I would either eat it or not eat it. NP2 2. (Blank) 3. They are related because you risk and them return. 4. Yes I will by spending it on valuable. Feedback: TIMMI is important to help me someday. Pre-Test Total: 0 Post-Test Total: 7 Change Index: 3.5 Post–A

Pre–A

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

0 0 0 0 3 0 3 1

1. (Blank) 2. (Blank) M 3. (Blank) 4. (Blank) NP1 1. (Blank) NP2 2. (Blank) NP3 3. (Blank) 4. (Blank) Feedback: I like that the teachers were highschools and fun. No dislikes. Pre-Test Total: 0 Post-Test Total: 0

0 0 0 0 0 0 0 0

Post–A

Pre–A

Change Index: 0

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

The following are the original student response matrices of the control group.

Post–B

Pre–C

Post–C

Pre–B

Pre-Test Total: 8

Post–A

Pre–A

Pre-Test Total: 3

Post–C

Pre–A

Pre-Test Total: 5

Pre-Test Total: 3

1. Identifying the outcomes. Can't just frivolously make decisions. 2. (Blank) 3. (Blank) 4. Yup. Gonna take financial classes. Cause I wanna know what I'm doing with my money. 1. Allows me to have enough money left over, expenses to buy... things. Buy the things I need. 2. (Blank) 3. (Blank) 4. Yup. Financial classes. Yup Post-Test Total: 7 Change Index: -1

4 0 0

1. So that you sustain yourself and your money to benefit in the future. 2. (Blank) 3. (Blank) 4. Yeah. I'm not sure yet. 1. Creating a budget. 2. Consumers and credit card companies. 3. (Blank) 4. Yes. Not sure yet. Post-Test Total: 5 Change Index: 2

2 0 0 1 0 4 0 1

1. Studying an extra hour for a test, I did better on the test. 2. The classes you take, taking more difficult classes. 3. I don't know. 4. I do plan to practice smart money management through stocks. 1. Refer to Pre Sheet. 2. Refer to Pre Sheet. 3. Refer to Pre Sheet. 4. Refer to Pre Sheet. Post-Test Total: 5 Change Index: 0

2 0 0 3 2 0 0 3

1. (Blank) 2. (Blank) 3. Cost? Consequences? 4. Creating a budget. 1. Knowing what the problem is first. 2. Benefactor? 3. (Blank) 4. By creating a savings account. Post-Test Total: 6

0 0 0 3 3 0 0 3

Change Index: 3

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

4 4 0 0 3

Post–C

Pre–A

Post–B

Pre–C

Pre-Test Total: 4

Pre-Test Total: 3

Post–C

Pre–C

Post–A

Pre–C

Pre-Test Total: 4

Pre-Test Total: 6

1. What is opportunity cost? 2. You can get credit by playing video games and upgrading your stats. 3. Risk leads to returns 4. I don't know how. 1. The end because it's not about how you start something, it's how you finish. 2. It benefits rich people because they get richer. 3. Pick Apple and make bank. 4. I don't know what this means. Post-Test Total: 1 Change Index: -3

0 0 4 0

1. The most important is seeing what choices you have first. 2. (Blank) 3. (Blank) 4. No 1. Budget is important. It allows you to see what you can buy with the money you have. 2. (Blank) 3. (Blank) 4. No Post-Test Total: 3 Change Index: 0

3 0 0 0

1. Considering the advantages and disadvantages because it helps with actually making a good decision. 2. (Blank) 3. (Blank) 4. (Blank) 1. See answer on other sheet. 2. (Blank) 3. (Blank) 4. (Blank) Post-Test Total: 4 Change Index: 0

1. I don't know. 2. Everyone, they get $ 3. Choose one that will profit 4. Yes, by keeping my money safe and not spending. 1. Getting nachos - not being able to buy panini. 2. Get $, get from many banks. 3. Risks are the consequences for returns, is not good. 4. Yes, save money/invest Post-Test Total: 11 Change Index: 5

0 0 1 0

3 0 0 0

4 0 0 0 4 0 0 0

0 3 1 2 4 2 2 3

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Post–A

Pre–B

C10

Post–B

Pre–C

Pre-Test Total: 7

Post–A

Pre–B

C11

C12

Post–A

Pre–B

Pre-Test Total: 9

Pre-Test Total: 5

1. So that people don't overspend on stuff they might need or might not need. It allows me not to overspend. 2. So I can get stuff without being ask about my credit and I made the most of it. 3. No clue 4. Yes 1. Don't remember 2. (Blank) 3. (Blank) 4. Yes Post-Test Total: 1 Change Index: -6

1. What are the consequences of each decision? 2. What is this?? 3. What stock has a recent drop at price. 4. Yes so I have money for later spending. 1. Allows me to know how much money I spend. 2. Being able to get a loan. 3. Not a clue. 4. Yes, so I have money for later spending. Post-Test Total: 10 Change Index: 3

1. You keep track of expenses. Allows one to be sure they won't be in debt unforeseen. 2. Companies are more willing to lend money, and to approve credit cards. 3. Don't know the difference. 4. I do, however I am unaware of the methods. 1. Not doing all the homework for Alg II Honors, need to do at a later date. 2. Maintaining monthly payments on loans and cards. 3. Higher risk = Higher returns 4. Yeah, sure. Post-Test Total: 13 Change Index: 4

1. Spend $ wisely. 2. (Blank) 3. (Blank) 4. Yes, plan out how to spend money. 1. (Blank) 2. (Blank) 3. (Blank) 4. You plan out how to spend $. Post-Test Total: 2

Change Index: -3

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

4 2 0 1 0 0 0 1

3 0 1 3 3 4 0 3

4 4 0 1 4 4 4 1

2 0 0 3 0 0 0 3

Post–A

Pre–B

C13

C14

Post–B

Pre–A

Pre-Test Total: 4

Pre-Test Total: 8

Post–B

Pre–A

C15

Pre-Test Total: 4

1. (Blank) 2. Income, get a better job. 3. Take a bigger risk for a bigger return. 4. No 1. To limit money usage. 2. Easier to take a loan. 3. (Blank) 4. No Post-Test Total: 7

Change Index: -1

1. I decided to play U10 Cricket national during the summer. The opportunity cost was that I would have less free time during summer and less time to practice for high school soccer. 2. (Blank) 3. (Blank) 4. No 1. It is important because you save money and you can use it later if a problem occurs (job loss etc.) 2. I make the most of it. 3. Penny stocks, because it sounds like small money -> saving up to loads of money. 4. No Post-Test Total: 4 Change Index: 0

1. It allows you to keep track of money, it saves you money. 2. People are willing to give you loans. M 3. Penny stocks are bought in great volume, I prefer penny, you get a lot of stocks. 4. Yes, with a budget, it saves money management. 1. Keep track of money. 2. More loans are given/accepted. 3. Penny is tiny stocks, penny pref. (cheap) 4. Yes, with a budget it is good for saving $. Pre-Test Total: 15 Post-Test Total: 13 Change Index: -2 Post–B

Pre–B

C16

1. Creating a budget is important because it helps plan money management. 2. I have more choices to do things and a better reputation. 3. IDK. 4. No 1. I chose to buy chocolate. Opp. cost? 2. No idea. 3. No idea. 4. No Post-Test Total: 2 Change Index: -2

2 2 0 0 2 0 0 0 0 4 4 0 3 4 0 0

4 0 0 0 4 0 0 0 4 4 4 3 2 4 4 3

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

1. Evaluating each decision b/c that's what makes sure you don't do something stupid. F 2. Creditors (credit companies) -> they charge interest on your loans, have confusing rules so they can charge you more interest. 3. Evaluate company history (past APR and stuff), don't buy too much at first (might fail=you're in trouble) 4. Yes, I will budget. 1. I ate an orange instead of an apple. I lost the satisfaction of eating an apple (opportunity cost). 2. Credit score -> Pay bills on time, don't take out too many loans. 3. Higher risk -> More chance for great gain (for high risk=high interest) but probably won't gain 4. See pre test Pre-Test Total: 14 Post-Test Total: 15 Change Index: 1 C18

Post–B

Pre–C

Post–A

Pre–C

C17

C19

Post–B

Pre–C

Pre-Test Total: 6

Pre-Test Total: 2

Post–A

Pre–C

C20

Pre-Test Total: 9

4 4 3 3 4 4 4 3

1. Choosing the right decision. 2. (Blank) 3. (Blank) 4. Yes by spending only a certain amount each week. 1. So you don't run out of money and it allows you to spend it wisely. 2. (Blank) 3. (Blank) 4. (Blank) Post-Test Total: 3 Change Index: -3

3 0 0 3 3 0 0 0

1. I don't know to the decision making process. 2. I don't know. 3. I don't know. 4. By spending it wisely. 1. It is important to create a budget so you don't overspend any money. It allows you to spend your money more wisely. 2. The benefits to having good credit is you don't owe anyone anything. 3. I don't know. 4. I plan to do it but I don't know how. Post-Test Total: 7 Change Index: 5

0 0 0 2

1. The thinking of the decision and making sure it works. For making the right decision. 2. The students. They look better on college applications. 3. Always invest in the one with the best short quarterly gains. 4. Yes, bank investments. 1. I made a decision to eat lunch. I lost money and lost hunger. 2. Credit involves working hard at an extra activities. We can do more to get more credit. 3. What you put in can be returned or lost. 4. See pre Post-Test Total: 8 Change Index: -1

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

3 3 0 1

3 0 3 3 3 0 2 3

Post–C

Pre–B

C21

C22

Post–B

Pre–C

Pre-Test Total: 8

Pre-Test Total: 4

Post–C

Pre–B

C23

Pre-Test Total: 3

Post–A

Pre–B

C24

Pre-Test Total: 7 48

1. So you know how much money you can spend on stuff. 2. You can get better job offers. Better insurance and better reputation. 3. No idea. 4. No 1. Actually making the decision. 2. No idea. 3. No idea. 4. No Post-Test Total: 3 Change Index: -5

4 4 0 0 3 0 0 0

1. Planning 2. (Blank) 3. Research prior to deciding 4. Yes 1. (Blank) 2. You get more money :) 3. Blue chip is more risky 4. Savings Account Post-Test Total: 6

1 0 2 1 0 3 0 3

Change Index: 2

1. Being able to create a budget you will know what to spend. Allows management. 2. (Blank) 3. (Blank) 4. No 1. Process of getting materials b/c this is the base on how things will be named. 2. (Blank) 3. What makes the most money and in efficient working 4. No Post-Test Total: 2 Change Index: -1 1. It prevents you from spending far more money than you originally intended to upon starting a project and allows for easier management of finances. 2. (Blank) 3. (Blank) 4. Yes, because managing money is a task that nearly everyone performs in their lifetime, so learning how to do it early on prepares me for the future, when I'll be forced to manage my money. 1. (Blank) 2. (Blank) 3. Returns are the effect that producing & selling a product has. One return from taking a risk is that it often piques consumer interest, as it's usually a risk they're never seen taken. This can increase sales, and lower the need for advertisement, as publicity alone can promote products. 4. Answer on other sheet. Post-Test Total: 5 Change Index: -2

3 0 0 0 1 0 1 0

4 0 0 3 0 0 2 3

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Post–B

Pre–A

C25

Pre-Test Total: 4

Post–A

Pre–C

C26

C27

Post–B

Pre–A

Pre-Test Total: 7

Pre-Test Total: 9

Post–A

Pre–B

C28

Pre-Test Total: 4

1. I did track, which will make me a better runner. 2. Being prepared before you do something like taking a test. 3. They are the starting idea & an outcome. 4. No, but in personal finance I will. 1. Allows you to save money. 2. Good credit=good reputation 3. Blue Chip - Penny's cheap 4. No Post-Test Total: 10 Change Index: 6 1. Finding a solution, because without a solution, you cannot solve the problem. 2. You can get loans. 3. (Blank) 4. Yes 1. I studied instead of watching TV. The opportunity cost was that I would get a good grade. 2. You can pay off loans. 3. Yes, I plan to keep track of all the money I spend so that I will have money to spend on something I want. 4. (Blank) Post-Test Total: 12 Change Index: 5

2 0 2 0 4 3 3 0

2 4 0 1 4 4 0 4

1. I decided to not study in Spanish and I failed the test. 2. Balancing money and having a budget. 3. (Blank) 4. Not spending too much. 1. To have a stable balance of money. It allows you to have money and not spend too much. 2. You can have more savings and have more money. 3. (Blank) 4. Yes, by not spending too much money and balancing it. Post-Test Total: 7 Change Index: -2

4 2 0 3

1. To save money and set aside for later. 2. (Blank) 3. (Blank) 4. No money to manage. 1. I made a decision to try to fix my laptop by mself. 2. (Blank) 3. Sometimes the risk is high (computer breaking) and return is high as well (computer working) 4. No. I have no money to manage. Post-Test Total: 6 Change Index: 2

4 0 0 0 2 0

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

3 1 0 3

4 0

Post–A

Pre–A

C29

Pre-Test Total: 3

1. (Blank) 2. (Blank) 3. (Blank) 4. Yes, by creating a budget to manage my money. 1. (Blank) 2. (Blank) 3. (Blank) 4. Yes, by creating a budget to better mange money. Post-Test Total: 3 Change Index: 0

0 0 0 3 0 0 0 3

Student C30 was removed from the sample. The student received a pre and post-assessment that were the same version, and inadvertently only filled out the pre-assessment. We assume this was because the student assumed receiving a duplicate assessment was a mistake (the student did not ask the researcher who administered the test about this).

Post–C

Pre–B

C31

Pre-Test Total: 6

1. It is important because it allows how to be responsible with how people spend money. It allows me to know how much I should spend on something. 2. (Blank) 3. I do not know what they are. 4. I already have, I save most of my money but will sometimes spend it for gifts. 1. Being able to predict which stocks can do well, or which ones won't. 2. (Blank) 3. (Blank) 4. I do by saving most of my money, because then I can use it for important things like college. Post-Test Total: 3 Change Index: -3

4 0 0 2 0 0 0 3

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org

Acknowledgements Principally, we wish to thank Mrs. Michelle Winter for advising our initiative. Her countless hours and immeasurable support have made TIMMI possible. I, of course, must thank the AP Statistics class at Livingston High School and Mr. James Page for teaching me how to design the experiment and analyze the results. We also wish to thank Mr. Robert Grosso and Mrs. Bronawyn Oâ&#x20AC;&#x2122;Leary for helping us implement this teaching of TIMMI at the Heritage Middle School. We thank Mr. Robert Rynone for letting us use his classroom at Heritage. Finally, we thank Anthonyâ&#x20AC;&#x2122;s Coal Fired Pizza in Livingston for their generous in-kind support, which provided nourishment and encouragement for our students (and our student teachers).

TIMMI Progress Report for the Heritage Middle School | 02/24/15 - 03/17/15 | timm-i.org