Since I was a kid I've dreamt of virtual worlds where the user could interact with realistic and dynamic NPCs. With the recent developments in AI this dream is now closer than ever to reality. While doing some research into work being done in this area, I came across the paper Generative Agents: Interactive Simulacra of Human behavior by Park et al.

Park's team designed an agentic architecture that allows AI to behave more similarly to humans than standard LLMs. This is done using a collection of each agent's entire memory, called the memory stream, which agents use to recall relevant information when making decisions. This enables them to have long term coherence, an area that traditional LLMs struggle with. Furthermore, the architecture allows for agents to reflect on their memories, forming higher level idea than just simple observations. For example, if a persona observes another persona frequently studying at the library, they may conclude that they are a studious person. People can generalize beyond raw observations, just like these agents now can.

Once I got the simulation running, I considered several directions: building new environments, enhancing the architecture, or designing custom agents. I ultimately decided to focus on a single custom agent — Eren Jaeger from Attack on Titan — to explore how well a fine-tuned character could integrate into the simulation. I imagined being able to insert characters at will into a virtual environment like this to see how they interact with each other. Wouldn't it be interesting to see what would happen if you put Eren Jaeger, Superman, and Einstein in a room together? However, to keep the scope realistic I chose to focus on a single character.

I'm currently collecting dialogue and behavioral data from Eren and working on aligning it with the memory-reflection framework of the simulation. I'll be sharing more next week.