ChatGPT as a Jury? Multi-Modal AI Versus Human Evaluation in an Architectural Design Competition

Abstract

This study examines the capacity of ChatGPT (GPT-4o), a large language model-based AI system, to evaluate architectural design competition projects. It investigates how jury input shapes the model's assessments and how effectively it integrates multimodal data (text + visuals). The analysis focuses on eight awarded projects (three prizes and five honorable mentions) from the 2025 Kad & imath;k & ouml;y Municipality New Service Building National Architectural Design Competition. ChatGPT assessed the projects under two scenarios: with access to jury reports (jury-influenced) and without any jury data (independent evaluation). In both cases, the projects were scored on a 0 to 5 scale across six thematic categories defined in the competition brief. The findings show that when jury reports were available, ChatGPT's evaluations aligned strongly with those of the human jury (r approximate to 0.87). Without jury input, project rankings and thematic emphases shifted, indicating the model's capacity to generate alternative interpretations based on textual and visual information. This study contributes to architectural evaluation research methodologically by introducing a dual-scenario framework, theoretically by revealing how AI mirrors or diverges from human judgment, and practically by demonstrating its potential as a decision-support tool. While AI can enrich evaluation processes, human expertise remains essential for nuanced, context-specific decision-making.