What I'm seeing work is a little more structure to both the underlying data and the structure of the prompt.
If the data is consistent and has clear lines drawn between similar, related components, it tends to be more stable. The risk here is that you miss potentially valuable inferences that are off the grid. It's a fine balance.
Also, by giving the user a little more of a structured approach to developing the prompt, you can focus the inference a little better. Same risk -- you miss out on some of the creativity.
In essence, go back to '70's weed -- the stuff that didn't send you into space quite so quickly.
Personally I don’t believe that LLMs on their own will ever achieve anything approaching sufficiently reliable/predictable/non-hallucinatory behavior. What this means for software that must harness their new potential--but must also be depended upon--is what we’re all groping with at the moment.
What I'm seeing work is a little more structure to both the underlying data and the structure of the prompt.
If the data is consistent and has clear lines drawn between similar, related components, it tends to be more stable. The risk here is that you miss potentially valuable inferences that are off the grid. It's a fine balance.
Also, by giving the user a little more of a structured approach to developing the prompt, you can focus the inference a little better. Same risk -- you miss out on some of the creativity.
In essence, go back to '70's weed -- the stuff that didn't send you into space quite so quickly.
> Curious how others are handling nondeterminism in their stack.
I have an acquaintance who has been writing around this topic and I have been finding the discussion interesting (ex. https://www.varungodbole.com/p/why-do-companies-struggle-shipping ).
Personally I don’t believe that LLMs on their own will ever achieve anything approaching sufficiently reliable/predictable/non-hallucinatory behavior. What this means for software that must harness their new potential--but must also be depended upon--is what we’re all groping with at the moment.