My thought would be that a twist should be necessary for the story to be believable, and should be as simple as possible. I think of Darth Vader being Luke's father. It was incredibly effective because it was simple, straightforward and a surprise (though foreshadowed by Obi Wan Kenobi comments). We, the audience, bought into it immediately without question. Is that what you are looking for in your plot twists or something more subtle or complex?
All the time. But that's also because most of the time, I know where I want my characters to end up but don't always know how they get where they are going so I am constantly asking myself why and how.
No. But I do sometimes find unanticipated possibilities opening up as the writing expands. My example, as requested: One of my characters (A), as I was writing her, gave me to know that she couldn't be the way I originally planned her. As she became the person she needed to be, she began to occlude another character (B) who was originally going to tick a number of boxes that A is now ticking. Thus, poor B, who was to be a major player, is now relegated to an ancillary roll and I have decided to make her too young to really fill the major roll she would have filled. Perhaps in a subsequent novel she will have grown up some and has a story of her own to tell, but not right now.
I have only just included a massive threat to humanity - so yes - before the creatures would just die - that was it - no real crisis - if a certain thing didn't happen
the aforementioned plot twist in book 2, well, its gone and twisted on me again taking a much darker turn
Freestyle doesn't work for me, proven by multiple manuscripts gone completely wrong, so I make detailed outlines beforehand. However, usually I have to end up going back and changing almost everything, because as I keep writing, the characters change by themselves. They kind of grow out of the 2-D world and become 3-D instead, with realistic flaws, fears, and other quirks that they didn't have at the beginning. Then I have to change the ending, because ending it the way I originally planned it would be OOC for the newly established MC's. In other words, I don't have unexpected twists and turns regarding the plot-- I have unexpected twists and turns regarding my characters. [example: my MC in one of my earlier stories used to be a strict, no-nonsense character, but somewhere along the way she decided to be kind of sassy, too, by herself. That was painful to fix.]
Especially in the case of imaginative work, you sometimes can't figure it out because it has a spirit or force within the work in progress book and it dictate what it will look like, how it begins or end and sometimes when you sleep a new scene unfold in your dream mysteriously to aid the existing plot creating another level of suspense even yourself did not predict. This is what actually turn the readers on. Yes is the answer to your question
