What confuses large language models?

It should be noted that these tools confuse themselves sometimes. This isn’t necessarily because of your prompt, but is a limitation of these LLMs and how they respond to prompts. When we use the word ‘confuse’ in this context we should be careful to not read too much into its meaning. Large language models are not conscious or aware, they do not understand and they do not have knowledge, so when we say ‘confuse’ we mean that the large language model produces something that doesn’t match the query, or provides information relating to something different, or it produces something that contradicts a previous output.

A ChatGPT fail

Here is an example of a scenario which caused ChatGPT to fail. It was asked to produce a list of 100 authors, and then asked to select only the women who had the letter L in their names:

View the full size version of the image

As you can see, ChatGPT has prioritised selecting the women first, but has forgotten to only show names with ‘L’ in them. It has also assumed that Alex Michaelides is a woman’s name although he is male. This shows that ChatGPT is making a guess at which authors are female based on their name, rather than by looking at other information about them.