| dc.description.abstract | This dissertation presents empirical and theoretical work aimed at enhancing the understanding of transformer-based Large Language Model (LLM) behaviors, with the empirical behaviors compared to established human behaviors. The dissertation introduces PopulationLM, a method employing systematic perturbations to generate model populations, facilitating the characterization of robust LLM cognitive behaviors. Using PopulationLM, the study replicates experiments on typicality and structural priming, demonstrating typicality effects in LLMs and the absence of structural priming in tested models. The dissertation examines human-like strategic behaviors in LLMs, highlighting models capable of value-based preference (VBP) and their responses in scenarios like the prisoner's (PD) and traveler's dilemmas (TD). Findings reveal that robust, VBP-capable LLMs may not exhibit certainty towards weakly dominated strategies, and align with human sensitivities to stake-size (PD) and penalty-size (TD). Moreover, the dissertation advocates for reproducible research, cautioning against reliance on closed-source models due to their lack of long-term reproducibility similar to important but privately held fossils. The theoretical contributions assert the Turing completeness of decoder-only transformers, while also identifying limitations when engaged in certain tasks, prompting exploration of alternative architectures for advancing artificial general intelligence. | |
