Guidance Document
Appropriate Use of Technology in Physical Education
GUIDANCE DOCUMENT
Appropriate Use of Technology in Physical Education Acknowledgements SHAPE America extends its appreciation to the members of the Physical Education Council for contributing their time and effort to developing this document. Adriana Lucero (Chair), University of New Mexico Nilo Ramos (Past Chair), Coastal Carolina University Valeri Cheseldine, Eva Turner Elementary School Meg Hemstrought, Utica University Robert Holmes, Chicago State University Srecko Mavrek, KAPPA International High School Will Potter (Board of Directors Liaison), Serendipity School Terri Stewart, Palm Beach School District Joe Velardi, Connecticut Association of Schools
Copyright © 2023 by SHAPE America ● www.shapeamerica.org ● All rights reserved. PO Box 225, Annapolis Junction, MD 20701 ● 800.213.9527
Table of Contents Introduction ......................................................................................................................6 Support for Technology in Physical Education ............................................................7 Appropriate and Beneficial Uses of Technology for Physical Education ..................8 Inappropriate/Ineffective Use of Technology ............................................................. 13 Steps for Using Technology in Your School and/or District ..................................... 14 Social Media ................................................................................................................... 15 Artificial Intelligence/Chat GPT .................................................................................... 16 Conclusion ..................................................................................................................... 17 References ..................................................................................................................... 18 Appendix 1: Instructional Technology Resources for Physical Educators and Students .................................................................................................................. 20
SHAPE America
Appropriate Use of Technology in PE
5
Introduction Technology can aid teaching and learning — and help students become physically educated individuals who have the knowledge, skills, and confidence to enjoy a lifetime of physical activity. In addition to being used as a means of activity itself, technology expands opportunities for students to: ●
Learn about the benefits of physical activity;
●
Monitor their progress toward fitness;
●
Learn physical skills;
●
Acquire feedback; and
●
Plan for a healthy and active lifestyle.
Physical education teachers use technology to present lessons, provide feedback, share resources, monitor activity, communicate with students and parents, and assess progress (McVicker, 2018; Parks, 2021; Perumal & Subramani, 2020). SHAPE America believes that technology can be an effective tool for supplementing instruction when used appropriately. Therefore, the primary purpose of this document, developed by SHAPE America's Physical Education Council with input from the Adapted Physical Education Special Interest Group, is to provide guidelines for using technology to help students achieve SHAPE America’s National Physical Education Standards. This document has three goals: 1. State unequivocal support for physical education programs in their effort to acquire needed technology; 2. Present a statement of appropriate and beneficial uses of technology that enhance instruction in physical education and assist in the achievement of national standards; 3. Offer a slate of technologies that are being successfully implemented by physical education teachers.
SHAPE America
Appropriate Use of Technology in PE
6
Support for Technology in Physical Education Americans have reaped the benefits of technology to improve their health. All people can derive a benefit to their fitness, movement skills, nutrition, and their lifestyle through the use of technology (Hasset, 2016; Gao 2019; Physiopedia, 2023). Elements of technology are so integrated into students’ personalized learning experience and into our wellness that authentic learning can only occur if technology is made available to those experiences. Therefore, schools and administrators should support professional development for physical education teachers, both in-district and out of district, where teachers can learn about new technologies and observe how technology is used effectively by other professionals. In addition, teachers should be afforded the time to fully understand the capabilities of a variety of technology platforms and to practice using them. Schools and districts should support the acquisition and use of technology as a means of enhancing the students' learning in physical education to the same degree that all other content areas are supported. It is SHAPE America’s position that technology enhances teaching and learning in physical education, and that professionals in the field bear the responsibility to maximize the effectiveness of their instruction through the understanding and implementation of its appropriate and most effective usage.
SHAPE America
Appropriate Use of Technology in PE
7
Appropriate and Beneficial Uses of Technology in Physical Education Instructional technology is one of many tools that can enhance the presentation of content and convey information to students. It is, however, merely a tool, which physical educators should use only in conjunction with effective instruction. Physical education teachers should infuse technology to support pedagogical strategies in their lessons. The effective use of technology requires teachers to have both content knowledge and pedagogical skills. This involves first identifying a goal, and then selecting the technology tool that will enhance learning it. Technology is most effective when it can provide a gateway toward exposing students to new types of lifetime physical activities and can foster enjoyable and purposeful experiences. The curricular goals should be the clear focus of every lesson, including those that utilize technology (Robelee, 2014). Each guideline below has a set of reflection questions that can be used to aid in determining whether the use of technology is appropriate and relevant. When implementing the use of technology in physical education, teachers should consider the following guidelines to ensuring its proper use in relation to national standards and developmentally appropriate instruction:
Guideline 1: The use of technology in physical education is designed to provide a tool for learning content that is relevant to national standards, state standards, and local curriculum. Instructional technology is one of many tools that can enhance the presentation of content and convey information to students. It can provide a wealth of information to students, allowing them to receive instruction through a variety of devices, not just media devices. To implement them adequately, physical educators must use these resources in conjunction with careful monitoring. Providing students with instruction concerning content validity and helping them develop the skills to evaluate the accuracy of electronic information is vital. Ensuring that students understand the difference between sources that have put procedures into place for monitoring content validity and those that have not is necessary for guiding them toward obtaining correct information. Equipping students to discern how to evaluate reliable and accurate information is a core piece of learning in any content area. Appropriate practice in physical education should include activities that are aligned to student learning expectations and that fit students’ developmental levels, as well as content aligned to standards. Using technology for technology’s sake might not provide relevant instructional experiences for students, since technology is not the curriculum but rather a tool or device to supplement instruction.
SHAPE America
Appropriate Use of Technology in PE
8
The following questions will assist in reflection when using technology: 1. Will the technology enhance student motivation, engagement, and involvement? 2. Does the technology assist students in acquiring information and knowledge about topics related to physical education? 3. Does the technology support online learning in physical education, online assignments, readings, chat rooms, and examinations, coupled with face-to-face, activity-based sessions? 4. Is the technology especially valuable when learning about fitness and exercise science principles? (Can it demonstrate concepts in a more dynamic way than a traditional classroom approach?) 5. Can the technology present the lifelong physiological benefits of physical activity? 6. Does the technology provide biofeedback which can assist the individual to maintain a healthy level of physical fitness and/or activity? 7. Will the technology assist the student to build on previous learning? 8. Can the technology be used while completing projects or developing portfolios? 9. Can the technology provide an opportunity for students to diagram or draw out concepts? 10. Is the technology meaningful and accessible to all students?
Guideline 2: The use of technology in physical education should enhance instruction. Technology such as projection systems, smart boards, and wireless transmission (Wi-Fi and Bluetooth) allow for the display and transfer of information far beyond the traditional chalkboard. Teachers can enhance physical education instruction by using those tools, provided that set-up and/or implementation doesn’t reduce student engagement time. Planning and preparing effectively in advance of lesson presentation is necessary to ensure that these valuable tools become an integrated part of the lesson with minimal transition time and management. Before developing and using hybrid online courses in physical education, teachers should have adequate professional pedagogical knowledge for delivering content in the online environment, and their course design should contain interactive communication versus stand-alone content. Teachers should have time to learn how to use online technology for instruction and assessment. The following questions will assist in reflection when using technology: 1. Does the technology aid in the presentation of material? 2. Can the technology be used to create your own instructional platform (e.g., online “classrooms,” instructional videos)? 3. Can the technology be used to post or remind students of assignments?
SHAPE America
Appropriate Use of Technology in PE
9
4. Does the technology contain motion-analysis software and digital video which makes student performance evaluation easier, thereby enhancing teacher, peer and individual analysis? 5. Is the learning curve for using the technology easy enough to ensure that the technology doesn’t interfere with achieving lesson objectives? 6. Does the technology support alignment with local, state, and/or national standards? 7. Does the technology provide a platform for students to create and manage individual activity plans? 8. Does the technology allow for formal, specific feedback? 9. Does the technology enhance the learning environment (e.g., music)? 10. Does the technology display and transfer information beyond the traditional classroom materials? 11. Does the technology support lesson design that fosters interactive communication versus standalone content? 12. Does the technology provide opportunities for journaling? 13. When using the technology, can the teacher implement classroom organizational structures that maximize the use of available time and equipment? 14. Does the technology allow for instructional components that teach fitness knowledge (not just monitor fitness performance)? 15. Does the technology help with time management in the classroom?
Guideline 3: The use of instructional technology in physical education should provide opportunities for all students, versus opportunities for few. Education encompasses students from diverse economic, cultural, and demographic backgrounds. It is incumbent upon the school to provide technology that equalizes the learning opportunities for all students regardless of access to resources. Students should not be further disadvantaged educationally because they cannot afford or have access to needed technology. Physical educators must consider which types of physical activity monitoring devices are suitable for students’ developmental levels. For example, using technology to monitor children’s heart rates and comparing the data to adult ratios (or having children use equipment designed for adults) can provide invalid information. Teachers should use these tools to enhance instruction only if the data provided is accurate for the specific grade level. The following questions will assist in reflection when using technology: 1. Will all students have equal access to the technology? 2. Can the technology be used in a developmentally appropriate way? 3. Is the level of difficulty for using the technology developmentally appropriate for the student(s)? 4. Can the technology be implemented in a manner that maintains the students’ privacy? 5. How will the technology be modified so it is accessible to all students?
SHAPE America
Appropriate Use of Technology in PE
10
6. Can the technology be used to search for and evaluate community resources and local opportunities for physical activity?
Guideline 4: The use of instructional technology in physical education can prove to be an effective tool for maintaining student data related to standards-based curriculum objectives. One of the great advantages of technology is the ease with which it can be employed to store, sort and use data. The teacher can use student data to inform instruction and curriculum. The student can use it to monitor their progress, and assess their performance. Databases are helpful in storing assessment results. Students can use those results to establish personal goals. Parents as well as educators can monitor progress. To manage student data effectively, it’s important that the teacher spend time using the technology before introducing it into the learning environment provide effective instruction to students about how to use the technology. That will help ensure that the technology supports achieving lesson objectives and personal goals. The following questions will assist in reflection when using technology: 1. Does the technology monitor students’ physical activity levels and provide valuable data concerning exercise intensity and/or duration? 2. Does the technology make goal setting and progress management clearer for the student? 3. Does the technology present tools that provide objective data on activity levels and creative methods for individuals to gauge physical activity? 4. Does the technology make assessment for learning more effective? 5. Will the technology assist the teacher in maintaining student performance data? 6. Does the technology assist the teacher with data management or assist in monitoring student progress? 7. Can the technology be used to provide valuable information to the teacher, guardians, and/or students regarding student progress toward program standards and objectives? 8. Can data be transferred to other platforms easily and effectively? 9. Does the technology help physical educators determine assessment performance quickly and easily through calculation formulas, and allow them to create and customize individualized fitness and learning plans? 10. Can the technology be used for quicker assessment correction and provide feedback quicker? (Does it create efficiencies?) 11. Can the technology be adapted and used to collect data related to unique learning objectives of students with disabilities?
SHAPE America
Appropriate Use of Technology in PE
11
Guideline 5: Technology can be used to enhance communication between the teacher, parents, students, administration, and the community. There are a myriad of ways technology can be used to communicate. Physical education teachers must use these technologies to properly inform all stakeholders regarding their physical education programs. The following are examples of how various technologies can be used for effective communication: •
Websites created by PE teachers can inform parents and communities about the physical education program and advocate for it.
•
Emails to parents can inform them of what students are learning about.
•
Text chains can be sent to students as reminders of work that is due.
•
PowerPoint presentations can be used to inform administrators, school boards, and the public of the valuable lessons that are learned in physical education.
The following questions will assist in reflection when using technology: 1. Will the technology enhance communication with the students and/or parents? 2. Does the technology convey the learning goals of the physical education program to the students, parents, and/or the community? 3. Will the technology allow students to collaborate more effectively through easier communication? 4. Does the technology enhance discussions and the exchange of ideas? 5. Can the technology be used in conjunction with assistive/communication technology?
SHAPE America
Appropriate Use of Technology in PE
12
Inappropriate/Ineffective Use of Technology Physical education teachers should take the following into consideration when using technology in their health and physical education program: 1. Technology should not be used to bully or harass others. Cyberbullying policies should be communicated to the students. 2. Technology should not be abused by students. Expectations regarding the use of school property should be clearly communicated to students and parents. 3. Teachers can enhance physical education instruction by using technology tools, provided that set-up and/or implementation doesn’t reduce student engagement in activity. 4. Using technology for technology’s sake might not provide relevant instructional experiences for students, since technology is not the curriculum but rather a tool or device to supplement instruction. 5. Teachers should consider the reliability and validity of devices when selecting the technologies to use. 6. Students should be well-versed in using the devices to prevent an increase in management time and a reduction in student activity. 7. Technology should not inhibit social interaction, the exchange of ideas, the negotiation of rules, or the self-organization of play. 8. Technology should be for everyone. Technology should never exclude certain groups of students or individuals from meaningful engagement with the physical education curriculum.
SHAPE America
Appropriate Use of Technology in PE
13
Steps for Using Technology in Your School and/or District When using technology in their school and/or district, physical educators should consider the following to maximize its use: 1. Understand your district’s privacy and legal policies. 2. Comply with your district’s policies for social media. 3. Obtain any appropriate permissions. 4. Refer to IEPs and 504 plans before implementing the use of technology. 5. Confirm the process for purchasing/acquiring technology for your program. 6. Establish how the technology enhances specific aspects of your curriculum, as well as state and national standards. 7. Lesson plan for how you will introduce/use the technology during your unit. Planning and preparing effectively in advance of lesson presentation is necessary to ensure that these valuable tools become an integrated part of the lesson with minimal transition time and management. 8. Physical educators must consider which types of physical activity monitoring devices are suitable for students’ developmental levels. Teachers should use these tools to enhance instruction only if the data provided are accurate for the grade level to which they are to be applied.
SHAPE America
Appropriate Use of Technology in PE
14
Social Media An aspect of the school’s health and physical education program for mental health should include warnings to students about the possible negative effects of using social media, which may include: 1. Social media can make you feel inadequate. 2. Social media can pressure you to maintain an image. 3. Social media can reduce your contact with other human beings. 4. Social media can keep you from doing other things. 5. Social media can interrupt your sleep. 6. Social media can connect you with dangerous people and expose you to cyberbullying. 7. Social media can invade privacy. 8. Social media can spread incorrect and harmful ideas and thoughts. 9. Social media can prompt harmful activities. 10. Social media is designed to be addictive. (Lee, 2022)
SHAPE America
Appropriate Use of Technology in PE
15
Artificial Intelligence/Chat GPT Advancing technology using artificial intelligence (AI) has become a concern across all content areas in education. Current debate mostly centers around whether or not students are doing their own work. There is fluctuation regarding the limits teachers and districts are permitting for the use of AI and specifically Chat GPT. As educators move forward and AI advances, a clearer picture may emerge for the appropriate use of artificial intelligence in physical education. In deciding what limits to put on AI in their classrooms, teachers should consider the following: 1. What is the school district’s policy on AI usage? 2. Does AI enhance the students' progression toward health, activity and fitness? 3. Can AI be a valuable tool in helping to plan activity into students’ lives? 4. Can AI be used by students in class to research proper techniques for sports or other physical activities? 5. How will you communicate your expectations to students about the acceptable use of AI in your course? 6. What are your requirements for citing AI-generated content when students turn in work? 7. How should AI be sourced in student work? 8. How will you be able to tell if the work done through AI is actually being learned and applied by the students?
SHAPE America
Appropriate Use of Technology in PE
16
Conclusion Instructional technology in physical education can and should be scaffolded much like we do for all other skills. Technology in physical education can be utilized as a tiered approach. Instructional technology should be used based on best practices and for instruction, without taking away from activity time. As technology evolves, we as educators should always look for ways to remain current. In an everchanging world, professional educators should seek professional growth in staying up to date with technology.
SHAPE America
Appropriate Use of Technology in PE
17
References Hassett, L., van den Berg, M., Lindley, R. I., et al. (2016). Effect of affordable technology on physical activity levels and mobility outcomes in rehabilitation: a protocol for the Activity and MObility UsiNg Technology ( AMOUNT ) rehabilitation trial. https://doi.org/10.1136/bmjopen-2016-012074 Henriksen, A., Haugen Mikalsen, M., Woldaregay, A. Z., Muzny, M., Hartvigsen, G., Hopstock, L. A., & Grimsgaard, S. (2018). Using fitness trackers and smartwatches to measure physical activity in research: analysis of consumer wrist-worn wearables. Journal of Medical Internet Research, 20(3), e110. Gao, Zan, National Library of Medicine, Journal of Clinical Medicine, Emerging Technology in Promoting Physical Activity and Health: Challenges and Opportunities (2019). Guo, Q., & Li, B. (2021). Role of AI physical education based on application of functional sports training. Journal of Intelligent & Fuzzy Systems, 40(2), 3337-3345. Lee, B, Y. (2022). Meta, TikTok, others being sued. Here are 10 ways social media can hurt kids’ mental health. Forbes Magazine. https://www.forbes.com/sites/brucelee/2022/12/18/meta-tiktok-othersbeing-sued-here-are-10-ways-social-media-can-hurt-kids-mental-health/?sh=4c3161f856a6 Lee, H. S., & Lee, J. (2021). Applying artificial intelligence in physical education and future perspectives. Sustainability, 13(1), 351. McVicker, D. (2018). How technology changes physical education. https://learn.g2.com/technology-in-physical-education#:~:text=Monitors%20and%20trackers,Since%20not%20all&text=Apart%20from%20heart%2Drate%20monitors,in%20a%20workout%2 0or%20exercise Palička, P., Jakubec, L., & Zvoníček, J. (2016). Mobile apps that support physical activities and the potential of these applications in physical education at school. Parks, A. (2021). How physical education teachers are bringing technology into the gym. Education World. https://www.educationworld.com/teachers/how-physical-education-teachers-are-bringingtechnology-gym Perumal, S. & Subramani, A. (2020). Technology in physical education. World Academy of Informatics and Management Science. https://www.researchgate.net/publication/344403534_Technology_in_Physical_Education Physiopedia, Physical Activity and Technology, (2023). https://www.physio-pedia.com/Physical_Activity_and_Technology#cite_note-6
SHAPE America
Appropriate Use of Technology in PE
18
SHAPE America. (2014). National Standards for K-12 Physical Education. Reston, VA: Author. Suriya, P., & Arumugam, S. (2020). Technology in physical education. TECHNOLOGY, 9(4). Xiang W. (2022). Application and Prospect Analysis of Artificial Intelligence in the Field of Physical Education. Computational intelligence and neuroscience, 2022, 1042533. https://doi.org/10.1155/2022/1042533 (Retraction published Comput Intell Neurosci. 2023 Jul 12;2023:9831968). Wilde, L. J., Ward, G., Sewell, L., Müller, A. M., & Wark, P. A. (2018). Apps and wearables for monitoring physical activity and sedentary behaviour: A qualitative systematic review protocol on barriers and facilitators. Digital Health, 4, 2055207618776454. https://doi.org/10.1177/2055207618776454 Yang, D., Oh, E. S., & Wang, Y. (2020). Hybrid physical education teaching and curriculum design based on a voice interactive artificial intelligence educational robot. Sustainability, 12(19), 8000. YanRu, L. (2021). An artificial intelligence and machine vision based evaluation of physical education teaching. Journal of Intelligent & Fuzzy Systems, 40(2), 3559-3569. Zhang B, Jin H and Duan X (2022) Physical education movement and comprehensive health quality intervention under the background of artificial intelligence. Front. Public Health 10:947731. doi: 10.3389/fpubh.2022.947731.
Suggested Citation: SHAPE America – Society of Health and Physical Educators (2023). Appropriate Use of Technology in Physical Education [Guidance Document]. Annapolis Junction, MD: Author
SHAPE America
Appropriate Use of Technology in PE
19
Appendix 1: Instructional Technology Resources for Physical Educators and Students Tier 1
Tier 2
Tier 3
(Easier to use, less engagement)
(More complicated to use, greater engagement)
(Higher level tech skills and management required; high level of student engagement)
Student Learning • Pedometers • Timers/stopwatches
• •
Educator Instructional Resources • Music • Communication tools (LMS for grading/sharing grades; email, etc.) • PowerPoint • Physical activity or health GIFs • School-based apps for parent communication • YouTube videos
• •
Student Learning Google Forms/Google Classroom/Google Slides LMS tools (discussion boards, quizzes, assignment drop boxes, etc.) Exergaming Mentimeter
Educator Instructional Resources • Smartboards • QR codes • Plickers/Plagnets • Mentimeter
• • • •
Student Learning HR monitors Phone or tablet applications Video analysis & modeling Instructional video creation
Educator Instructional Resources • Microsoft Forms/Teams • Zoom • Canvas
Educator Resources Assessment Development Resources Electronic Assessments: •
Kahoot! — A gamified, multiple-choice assessment that engages students and checks for student learning and understanding.
•
Quizizz — Assessment, instruction, and practice that motivates every student to mastery.
•
Google Forms — Digital assessment within the Google platform; student data can be converted to a Google Sheet for educator analysis.
SHAPE America
Appropriate Use of Technology in PE
20
Manipulative Assessments: •
Plickers — Get real-time results of your students’ understanding based on questions you can create that align with your standards.
•
Plagnets — Plickers and magnet assessment system, with many uses within the PE classroom, including a SOLO approach to gauging student understanding and progress.
Instructional or Presentation Resources/Learning Management Systems Projector Systems: •
SMART Board
•
LCD Projector
•
Lu Interactive Playground — Each Lü system includes a high-definition laser projector, a movement detection camera, a powerful sound system, and a fully automated, multi-colored lighting system to create immersive magical environments.
•
PLT4M — A powerful, classroom-ready wellness platform to help teachers and instructors establish physical literacy, teach lifelong fitness skills, inspire student confidence, and encourage student participation.
Presentation Resources: •
Google Workspace for Education Fundamentals o
Google Docs
o
Google Slides
o
Google Sheets
•
Prezi — Dynamic, moving presentation platform that engages student learners and helps students make personal connections in the learning process.
•
YouTube o
This or That fitness videos
o
Fitness or strength training/tabata/HIIT workout videos
o
Dance/cardio drumming videos
o
Physical activity skill development videos
o
Yoga videos
Interval Timers: •
Seconds Pro — a website that allows users to create HIIT, circuit training, tabata, or custom timers.
•
Connected PE Interval Timer
•
Circle Timer (Apple App)
SHAPE America
Appropriate Use of Technology in PE
21
Learning Management System Programs: •
Seesaw — Designed with young learners in mind, Seesaw brings together instruction, assessment, and communication in one place.
•
Canvas — A web-based learning management system used by learning institutions, educators, and students to access and manage online course learning materials and communicate about skill development and learning achievement.
•
Microsoft Teams — A workspace for real-time collaboration and communication, meetings, file and app sharing. All in one place, all in the open, all accessible to everyone.
Motor Performance Feedback Resources Mobile Apps: •
Sportalyzer Bundle — Video analysis app that allows viewers to watch two videos at the same time.
•
MyDartfish Mobile — Capture, analyze, and give video feedback with just your phone.
•
BaM Video Delay — Video delay app that continuously shows what just happened for quick visual feedback.
Electronic Video Discussion and Sharing Platform: •
Flip (formerly Flipgrid) — Record and submit digital projects; supports peer and teacher feedback.
Class Management Systems/Instructional Planners Class Management System Programs: •
Class Dojo — A safe and simple communication app for teachers, parents, and students.
•
IHT (Interactive Health Technologies) — Interactive technology ecosystem helps teachers build a student wellness portfolio daily-to-yearly, delivering heart rate data, reflective journaling, and performance metrics that can correlate to academic and social and emotional growth.
Instructional Planners: •
iDoceo — Gradebook and planner for iPhone, iPad and Mac.
•
Planboard/Chalk — Free online lesson planner and gradebook for teachers.
Student Grouping Organizer: •
Team Shake — Pick random groups; organize your students into small or large groups with this digital app.
SHAPE America
Appropriate Use of Technology in PE
22
Professional Development Digital Resources: •
CBHPE — Improve student learning with Chromebooks (and other devices) in health and physical education.
•
SPARK
Student Resources Physical Activity Resources Warm Up/Brain Break/Movement Activity Websites: •
GoNoodle — Engaging movement and mindfulness-based videos for students. Great for brain breaks.
•
Cosmic Kids Yoga — Interactive yoga adventures which build strength, balance, and confidence.
•
KooKooKangaroo — Dance-A-Long KooKoo Kangaroo movement videos.
•
DJ Raphi — Follow-along dance videos.
•
PLT4M — A powerful, classroom-ready wellness platform to help teachers and instructors: establish physical literacy, teach lifelong fitness skills, inspire student confidence, encourage student participation.
Fuel Up to Play 60 Program: •
Fuel Up Classroom-Ready Resources
SHAPE America
Appropriate Use of Technology in PE
23