A mechanistic interpretability tool to understand the internals of Turing‑LLM, a large language model. Using this tool, one can navigate the sparse autoencoder latent space of Turing‑LLM‑1.0‑254M and gain an understanding of how these large language models may function internally. Run the Turing‑LLM base model and view connections between top latents to better understand how latents work together to form thoughts.

A language model built for mechanistic interpretability research. Trained from initialization on a structured synthetic dataset. The text in the image are shortened outputs of Turing‑LLM‑1.0‑254M. ~254M Parameters, 12 Layers, SwiGLU Activation Function, Multi-Head Attention with 16 Heads Generated a synthetic dataset using outputs from Phi-3 Mini and utilised data augmentation to expand the training dataset to 2B tokens. Trained 12 TopK sparse autoencoders to obtain a more monosemantic set of Turing‑LLM latents. Each with a hidden dimension size of 40,960. Developed novel mechanistic interpretability approaches which will be presented in a publication in the near future.

Built a large web app for story writers to store and share stories. During this personal project, I gained significant skills in full-stack development. Developed many useful features including the ability to create and traverse an interactive 3D and 2D maps of a story’s universe, character relationship charts, Spotify integration, structured story content creation (for characters, plots, locations, etc), a user account system, and much more.

A web app build to help users conduct interpretability research. Chat with Google's Gemma 7B and view layer activations on each generated token. Includes an automated activation collection system accompanied by data visualizations for the gathered data.

Trained a neural network to predict the energy consumption of algorithm implementations. Build a tool to upload code, predict energy consumption of code, and search for energy efficient code.