[Thinking about Local LLMs Day 45] The token table only worked for layer 0 — Replacing layers 1 by 1 with tables and measuring against 'deletion'

On day 44, I found that replacing the contribution of layer 0 with a table looked up by token ID in a small 6-layer model ...