Human vs. Machine: Comparing AI-Generated and Human-Written Psychological Reports

Abstract

This study examines the effectiveness of artificial intelligence (AI) in psychological report writing by comparing reports generated by human psychologists with those produced by OpenAI’s Generative Pre-trained Transformer Version 4 (ChatGPT-4). A total of 249 licensed psychologists evaluated the reports based on overall quality, readability, writing style, organization, summary quality, recommendations, preference, and willingness to sign off on the reports. Although human-generated reports were generally rated more favorably and participants expressed greater comfort in approving them, effect sizes were typically small. Two exceptions were noted: moderate effect sizes were found in favor of human-written summaries, while AI-generated reports showed moderate effect sizes for the quality of their recommendations. These findings suggest that AI shows potential for augmenting report writing. Comprehensive guidelines are necessary for the ethical and effective integration of AI into psychological practice. Further research is needed to enhance our understanding of AI’s role and capabilities in psychological assessment and reporting.

Publication Title

Journal of Psychoeducational Assessment

Link to Full Text

Share

COinS