A recap of what caught our attention at Kaiku this week.
Decision models versus generative LLMs
As everywhere else, Jev-style decision models were the talking point this week. A Hugging Face write-up on when to use them offers a useful rule of thumb: if you can define the answer space in advance, and the result needs to be reused, ranked, routed or blocked by code, evaluate a decision model first. If the task needs writing, explanation, creativity or open-ended reasoning, stay with a generative LLM. Early experiments with product classification and similarity-search judgement calls were fast and cheap, though they still need tuning.
We also shared several open tools and resources internally that are worth a look: OpenJev, Laya, djev-run and the Typesafe docs.
How much context is enough for decision models?
The obvious follow-up question is how much context to give a small decision model. The Typesafe guidance to include only the context relevant to the current question is sensible but vague.
Cramming the context to the limit did not work well in practice, and a whole rulebook will not fit anyway. A more workable pattern is to model the high-level rules in application code and hand the model a pre-filtered slice of the current state, such as the cards in hand and on the board rather than the full rules. Evals across different amounts of context are the next step.
Can Jev play Doom?
Yes, it can! The first test we saw was to have Jev to play Doom, and it is an impressive watch! No longer is the test "Can it run Doom?" but rather "Can it play Doom?".
We were also impressed by its speed and by it reaching the first boss in Slay the Spire 2, which (according to our resident expert) is roughly what a first-time player manages.
Fun demos aside, it will be fascinating to see where these capabilities fit into our current workflows.
Project state that lives with the repository
Epiq is a tool for keeping task and project state inside the repository itself, and it looks like a more sensible take on the idea behind Beads, with some much-needed quality-of-life features. This approach is something we have had our eye on for smaller projects and one we want to try.
Want to talk about where AI-native delivery is heading? Get in touch.