One of my biggest fears when writing my novel is that I'll spend months and months, possibly longer, writing what I believe to be an awesome book, but at the end of the day, it either won't be published, or if it is, it'll massively fail. Of course, I understand you can't exactly predict the future, is there any way of at least having a good idea as to whether or not your idea is even a decent one in the first place? I mean, I really like my Deadbeat's Guide stuff that I'm writing. I personally think it's funny and engaging, and occasionally, some say it's okay. But how can I be sure that agents and publishers will also like it? I've already put in so much time and effort into my story, and I can't imagine investing 50x more time and seeing absolutely zero results. I mean, sure, there are beta readers, but those come AFTER all the work. How do I know my idea/plot itself is even tenable?
It is a chance we have to take, but I wouldn't recommend worrying about it. Do your best and write the best book you can. Even if it doesn't sell right now doesn't mean that it won't sell in the future when tastes and fads change again. Plus, think of all the experience gained writing it that'll help you become a multi-series novelist sensation.
Really, the idea isn't the thing. It's the execution of the idea that will make or break a novel. Yes, there's a little bit of luck involved but a quality tale--good storytelling, including interesting characters, action, dialogue, description and more, can't do anything but help reach success--both in finding a publisher and readers.
You probably won't know until the final draft is finished. Writing is a learning process, where you improve mainly by writing--studying how to write, observing how other successful writers have accomplished certain aspects of storytelling (characterization, writing action scenes, effective dialogue, pacing, etc.), and getting feedback from knowledgeable individuals. Sometimes a writer can tell midstream that a story just isn't working and might abandon it for something better. Almost every writer has a novel or two (or more) or short stories that they finished but never made it to publication. Often they recognize why, chalk the effort up to a learning experience, and move on. Many writers focus on short stories first because there isn't as large of a time commitment for such projects. While there is a lot that is similar in writing a short story and a novel, they are not exactly the same. Characterization, plot, and pacing (for example) differ greatly between stories and novels. With every project there is no guarantee for success (however one might want to measure that). Well, for most writers. I think authors like Stephen King can pretty much expect success (by most standards) with whatever he writes. But he also is very experienced and talented, has a proven track record and large fan base. But those individuals are few in the big scheme of things. Even successful authors get rejected. It's not fun, but it is just part of the business. Since great success first time out is quite rare, you'll probably have to slog and learn and improve over time and effort.
Don't try to write to the market, just tell a good story, as @TWErvin2 said. If you enjoy writing it, and telling others about it, it will probably turn out well. I had a lo9t of beta readers reading my first book chapter by chapter as they were written, and their enthusiasm helped propel me forward. Expect to lose the taste for it, during the editing phase; the editing process is done when you can't stand to look at it one more time! Don't worry about the time invested, I put 20 years into The Eagle and the Dragon, and it turned to be rather successful. Right now, just tell your story, then when finished, polish it until it shines like a new dime.
