We built a high-level native library for local LLM inference on top of llama.cpp, and solved a number of interesting problems along the way. This talk is about those problems.

This talk is going to be about:

• standardization efforts in the LLM space
• differences between the major LLM inference tools
• challenges with making assumptions about turing complete templating systems
• methods of constrainin… Read more

  ---

  Schedule:

  Not scheduled yet