One of my favourite series I've ever read was Sacrifices, a seven-part Harry Potter Fanfiction of epic proportions, four books of which were over 500,000 words each. The author never seemed to run out of conflicts for the characters to deal with and it was overall amazing. I would love to write something like that, not to publish, but just to better myself as a writer without as many limits. But I don't even know how to begin to form a plot for a story like that. I know the general set-up of long-running series is the main goal for the series, the goal for each book, and then the obstacles for getting the goal in each book. But how do you do that without it getting redundant really quickly? Same with the monster/mystery of the week that many long TV series have. Sacrifices story arcs were congruent to the grade levels of the MC's, but my story lacks that kind of structure so I'm at a loss for how to work with it. Sorry if this is confusing, I was trying to get the question across without being too wordy. Any and all answers are appreciated!
Are you sure you're taking the right approach to this? I think you try to vary the type of challenge/obstacle. Having a sequence of more and more powerful enemies would get old quite fast I think. But if you throw in a variety of quest types and interpersonal issues then you can spin it out quite a bit. You can have spy stuff, assassinations, seductions, betrayals, etc. Plus have them make mistakes, lose battles, etc.
What the...Is it about Harry Potter in the WH40K universe running from the Inquisition? I know Laurel K Hamilton's Anita Blake Vampire Hunter series (a tonne of books) all have different yet extremely similar plots as the series goes on. And it is a never ending series. Not my cup of tea since it is a small universe and not my type of genre to read either. You can take pretty much any plot and then force conflict after conflict on it for as long as you want the story to go on. Though at that point it is less like a series and more like an episodic serial. The bigger the universe and more diversity or obscurity will help keep things fresh and interesting while following around the same set of characters for an extremely long length of time. So it is a doable, but you will have to be deeply invested in the story for it to keep you wanting to push it along. On a personal note, I would get bored with writing the same story for that long. So many much more interesting concepts to explore in multiple avenues to approach them. Anywho good luck with this endeavor of yours.
The only way to keep it fresh that I can come up with would be to focus on the character relations and divide it into multiple story arcs with different themes. When ending a story arc to give space for the next one, it's easy to get carried away while trying to give it a good ending, so that the next arc gets a bad start. You'll also be very restricted by the system you've created and have to remember everything from every previous chapter to avoid both direct and psychological contradictions. I'd just revisit the same universe multiple times from different character's point of view so that I can get a fresh start without too much to remember from the previous novel. This also allows you to scrap the books that were no good without missing anything in the following books.
There’s also the Known Space approach made use of by Larry Niven. Every story in that system of books makes mention of events happening elsewhere in the universe. If it’s mentioned, you can bet there’s a book in the series that takes that small mention or reference and expands it into its own story, with its own internal references that spawn other stories, etc. etc. etc. ETA: This method has the added benefit that the reader can engage any book in the series as a starting point since it’s not strictly linear.
You could take the Monster of the Week approach that TV shows often follow. The story isn't necessarily about one specific quest or goal, it's just about surviving and doing the job. Cops shows have different crimes, doctor shows have different patients/illnesses, etc. The relationships between the recurring characters are important, but they play out against a backdrop of other stuff.
Yeah, right? Look at Law and Order. There's got to be over a thousand episodes now between the various series and they never seem to get boring. It's the perfect show in a lot of ways because they don't rely on characters too much. The spinoffs like SVU and Criminal Intent ride character arcs a little bit, but the original show almost never made mention of any of the cops or lawyers personal lives except for a blip here or there, and that was usually when they were preparing to write them off. I'm not even a huge fan of the show, but I can't remember TV without it. And I always know that no matter what happens, there'll be a Law and Order episode on at any given point and I'll always have something to watch.
I write because I have something to say and I use my characters in proxy. I guess if you are moved by another writer and you want to emulate them, maybe that's ok providing it is still you. I would suggest at the onset to do a short story, in the genre you want and go through the process of developing scenes, characters and something exciting to read about. When you are done with that, go through the editing and rewriting. Finally you may be saying something like Maria Muldaur said to me, "I can't believe I have to sing that f##n song again."