It is so difficult to make things like this that cannot the modified since all communication is in band.
I jailbroke the LLM then reframed the game so that I could be Fitzpatrick, or Crispi, or a superhero named Chronos. It continued to allocate 10 actions when I reframed the game, which was interesting.
Like so many things with LLMs, it’s a cool concept, but it is too easy to break.
To expand on this, the lack of a meaningful model of reality really detracts.
In one instance, I “accidentally” broke the pane of glass, which leads to an apparent dead end as the LLM will only proceed with fingerprint evidence. However, I noticed a fingerprint on a random building and it turns out it belong to Crispi! So everything is back on track and the case is solved.
The LLM is sort of reality rhyming. It doesn’t know what reality is, it just knows how to rhyme, so we get really silly situations like that one.
I jailbroke the LLM then reframed the game so that I could be Fitzpatrick, or Crispi, or a superhero named Chronos. It continued to allocate 10 actions when I reframed the game, which was interesting.
Like so many things with LLMs, it’s a cool concept, but it is too easy to break.