Does anybody have a rule of thumb for how many characters get to be too many for a single plot line, especially if there is a good subplot going?
No more than the author can keep distinct and memorable for the reader. This number varies depending on the writer's skill. Note: it is not uncommon for well-known writers to exceed this number, so don't feel bad if you have a hard time knowing where that limit is.
I suppose it comes down to skill and length. If you're writing a short story or novella, keep it to about three main characters. Novels will probably require many more, and epic novels (Which is my fancy improved way of saying very large books) will probably include many more. I always look at Tolkein as a good model. He has many characters, but all of them are memorable. As long as the reader can establish some initial connection to the character, whether it be pity, hate, or love, then you're pretty much solid. But your reader cannot possibly be attached to the bar-keeper who has two lines. Unless of course he saves the day.
You can have numerous ones in the plot. It may be easier to make a list of how many and who they are in your story. Try characterizing major and minor characters. The lists will vary and allow you to get a comprehensive amount. Like they all said, it is about skill and how well you can keep track of them.
As few as will allow you to tell the whole story. There may be a lot of minor characters but not all will need to be well developed. If the MC checks in with a receptionist, we don't need her name and description unless she has a larger role in the plot. A guy the MC talks to every day on the bus ride to work may not need any more than a first name and may only appear on the bus, so he doesn't need a character history if he is only a sounding board for the MC.
I think the number of characters needed for a story must be consisitant with the central premise or main plotline of the story. If you have characters that aren't intergal to te plot, I think s/he should be removed.
It might vary from genre to genre as well, because it sounds like fantasy/sci-fi tend to have more than other genres. I'm into mystery myself, and every book I've read there tends to be like 4-5 main characters, and then somewhere around 4-5 more that have something to do with the plot.
This really is genre and author dependent. Sci-fi and fantasy have a lot. Robert Jordan's Wheel of Time series has somewhere around a billion characters (ok, not that many, but it feels like it sometimes) that he did a pretty good job of keeping distinct. Course, you can do that over 14 books around 1000 pages each. Lit. fiction and detective stick to a handful of major and a handful of minor characters. Two or three is good, five is pushing it. Detective fiction can accumulate characters as the series goes on, but that's fine because, like most sci-fi/fantasy, you've had a few books to meet them. And, throwing this all out, if you're great at characterization, you can have as many as you want. It takes a master writer to do that, though.
Everyone's bang-on that's it's never more than you can handle. I try and keep a circle of four characters that the reader has to keep track of and branch off from there , but I always reign myself in. Not every character has to have there own moment or even a name - most are there to propel the plot and back up , or trip up the hero and his supporting characters.
