Learning to structure the world
This project aims not only to learn representations, but also to develop generative models that use learning‑to‑structure to improve efficiency, generalisation, and the ability to express and estimate uncertainty.
Project details
Description:
Current Gen AI methods are behemoths, requiring internet-scale data, massive compute and energy resources to work. Their success derives strongly from the flexibility afforded by their size and the scale of data observed. But does this need to be the case?
Recent work in representation learning shows judicious modelling constraints can be both effective and efficient by structuring observations to enhance reuse and compositionality---small models effective in low-resource settings.
This project aims to go beyond simply learning representations to develop generative variants that leverage learning-to-structure to be more efficient and effective, better amenable to generalisation, and the expression and estimation of uncertainties.
Who’s involved?
Sid Narayanaswamy, University of Edinburgh
Expertise sought
This project is seeking collaborators
We are seeking collaborators who:
Are interested in developing methodology and efficient modelling beyond LLMs and Diffusion Models.
Have interests aligned with human-like learning and cognitive science for features of representation learning and mechanistic interpretability.
Get in touch:
If you can offer the expertise needed for this project and would like to collaborate, please email us
Contact us
