The main character in my novel is a medieval aristocrat. I currently have her palling around with a handmaid. I want her to have a reliable friend who will later betray her. The closeness of a lady-in-waiting would have been common in this period, but is it too cliche? Is it too much "sidekick"? It is unlikely she would have had close friends her own age within the peerage as her home was rather remote.
I wouldn't regard this as a cliche. The important thing is to write compelling, interesting characters. If your characters are well-developed and well-written, then it definitely will not be cliche. It's perfectly normal, as you pointed out, for an aristocratic woman of that time to be close to her handmaid. Is it a cliche in a Western movie for the cowboy to ride a horse? Of course not. Just create excellent characters and you'll be fine.
Might I refer you to TV Tropes: Tropes are Tools, which discusses that it's not the clichés (tropes) that are bad/wrong/problematic - it's how they are used.
I agree with what's been said. A side character only becomes a "sidekick" when all they ever do, say, hope for, and feel for revolves around the main character. Even if she's not a viewpoint character she should still have the characteristics of being her own person. As long as you write her well, there's nothing wrong with it.
A cliche is a phrase or metaphor that is so overused as to have lost most or all of its meaning or impact. Characters are not cliches, and neither are story ideas. Whether or not they appeal to the reader is not a function of how similar they are to other characters or stories, but how well the writer has presented it to the reader.