I have a theory that puzzles are essentially practice for the working memory skills we need in order to perform more complicated activities.
That’s a mouthful, but what I mean is that in order to perform a complex task, like writing a book or solving a complicated math problem, we need to be able to load a lot of information in to working memory so that we can analyze it, move it around, reorganize it, etc. making the disparate connections required to arrive at creative ideas.
We can’t actually load that much information in to working memory so we have to form patterns that allow us to analyze more complex situations at a superficial level, looking for solutions. When we think we’ve found one we can dive in to the pattern, looking at it in more detail to see if it holds up beyond a superficial analysis.
I think this is why we find puzzles enjoyable. They distill down this process in to something we can practice. The same way children enjoy playing make-believe, pretending to be adults, we enjoy practicing the skills of working with patterns in our working memory.
I think this is also the key to creating a good puzzle. You need to create an environment that requires people to load some tools/facts/techniques in to memory where they can move them around, reorganize them, etc., looking for creative solutions. As people get better at the puzzle the basic tools you’ve introduced are turned in to patterns that allow for more efficient analysis. This allows you to make room for new tools to be loaded in to working memory, and eventually those more complex tools will be turned in to patterns.
I’m not sure if I’m explaining this particularly well, but as I seek to build better puzzles I think one of the keys is to create the feeling of having a number of things in working memory, where not all of which are valuable to solving the puzzle, and the ones that are valuable need to be reorganized to work toward a solution.