training character models that are actually fun and interesting

[PLACEHOLDER]

open questions + notes

How fast do they have to be able to talk/think/respond?

• first reaction?
• first response?
• intermediate thoughts?

How smart do they have to be?

• be helpful?

ICL >>> finetuning

solving long-context + prefix caching

tokenizers

GPTs

• LLaMA 3 series
• Mistral
• RWKV

testing out some katex

$\begin{bmatrix} 1 & 0 & 0\\ 1 & 1 & 0\\ 1 & 1 & 1\\ \end{bmatrix} \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{bmatrix} \begin{bmatrix} 1 & 1 & 1\\ 0 & 1 & 1\\ 0 & 0 & 1\\ \end{bmatrix}$