If you’re coding up a custom system, Try adding a 9b model that dose RAG searches with chromadb and outputs a memory summary that feeds into the main models context window. You can ingest old messages, web pages, character info, or whatever text. Qwen 3.6 35b is the first local model I ran that seems to be good enough to be useful if it has some infrastructure around it, and smart enough to not die on a tool call(usually).
Add in DDGS search and a web page scrape tool that adds to the RAG system, let it loop a few times and its actually quite useful, even if it takes 5-10 minutes to get an output. After that it’s all tweaking what is and isn’t in in context.
If you’re coding up a custom system, Try adding a 9b model that dose RAG searches with chromadb and outputs a memory summary that feeds into the main models context window. You can ingest old messages, web pages, character info, or whatever text. Qwen 3.6 35b is the first local model I ran that seems to be good enough to be useful if it has some infrastructure around it, and smart enough to not die on a tool call(usually).
Add in DDGS search and a web page scrape tool that adds to the RAG system, let it loop a few times and its actually quite useful, even if it takes 5-10 minutes to get an output. After that it’s all tweaking what is and isn’t in in context.
Also, try not to go too insane.