Bridging the Learning Gap: Exploring Human-like Language Acquisition in Large Language Models
Winnie Goh
Undergraduate Researcher
Finance Major (Kelley School of Business)
Zoran Tiganj
Faculty Mentor
Zoran Tiganj (O'Neill School of Public & Environmental Affairs)
Project Description
Large Language Models (LLMs), such as GPT-4, yield impressive results in natural language processing tasks. However, the learning mechanisms employed by these LLMs differ significantly from those of humans. While humans acquire vocabulary progressively via an environment-shaped curriculum, LLMs learn from an immense volume of text data without following a specific curriculum. The aim of this project is to delve into the implications of these distinct learning approaches. To accomplish this, we will train LLMs on data inputs that are similar to those experienced by human infants. The results of this investigation could help advance the development of human-like AI and also provide further insights into the factors that influence early language acquisition in humans.
Technology or Computational Component
The project will include training and fine-tuning existing LLMs on simple language corpora inspired by realistic human inputs. To get a sense of how training and fine-tuning could work, take a look at this Jupyter notebook that comes as a part of Hands On Machine Learning textbooks: https://github.com/ageron/handson-ml3/blob/main/16_nlp_with_rnns_and_attention.ipynb.