ChatGPT in Physics Education: A Content-Based Analysis on Newtonian Force Problems

Abstract

The integration of artificial intelligence (AI) in education has significantly transformed learning environments, particularly through the use of large language models (LLMs) such as ChatGPT. While these tools show promise in supporting science and technology education, their effectiveness in solving domain-specific problems, such as Newtonian mechanics, remains under-explored. This study aims to evaluate the capability of ChatGPT in solving essay-type physics problems involving Newton’s Laws of Motion, with a specific focus on force analysis. Using a content-based qualitative evaluation method, the research was conducted in three stages: development and validation of conceptual physics problems, submission of these problems to ChatGPT, and assessment of the AI-generated responses by expert reviewers. The problem used in this study required decomposition of forces on an inclined plane under idealized, frictionless conditions. ChatGPT's responses were evaluated across three dimensions: scientific accuracy, logical coherence, and contextual relevance. The findings indicate that while ChatGPT was able to provide structured and numerically accurate responses, it lacked depth in reasoning and failed to explicitly articulate physical assumptions and validation steps, such as analyzing counteracting gravitational forces. These limitations point to the model's partial conceptual understanding and highlight the need for human oversight. The study concludes that ChatGPT holds potential as a supplementary learning aid, particularly for reinforcing procedural knowledge. However, its use must be carefully integrated into instructional contexts that promote critical thinking and conceptual verification. Recommendations are offered for its pedagogical implementation, along with a call for further research into AI's role in physics education.