🧠 Memory Definition in LLM Applications
The ultimate goal is to generate personalized models. The actual work involves reverse engineering, using data generated by the model to infer the model itself. The model and training data are equivalent; the model is a compression of the training data. To achieve the ultimate goal, the model must be capable of on-device real-time learning, requiring a breakthrough in the current training -> inference paradigm. Training and inference must be integrated, with two core challenges - real-time updates (incremental training) and de-averaging individual data. The mid-term goal is to layer individual data (long-term memory) on top of a foundational model (pre-trained LLM).
Aug 2, 2024
