Ok, here's the deal. My wife is an avid reader, 95% murder mystery. It is her main and almost only hobby. when she isn't doing something she has her phone in her hand and her mind in a novel. I give her the first 7 chapters of a series murder book I'm trying to write and she couldn't get past the first 3 chapters. Her complaints are that the main protagonist (PT) is not particularly likable. She says that you have to be able to relate to her. I get that, but can the PT be abused and abrasive at the beginning then grow into something else as the novel progresses? What about movie characters that have a main character like Amy Schumer in Train Wreck, who's life is a train wreck? I just don't want to be put in a box and have to write characters a certain way. But she does have a point. She has read 1000's of books and I guess she knows likable and sellable books have relatable characters. Ok, enough of that. I also have a somewhat significant character that is kind of ugly. She also says that this is undesirable. Why?
It depends what you write. Girl on the Train manages to make an unlikable, not that attractive, unreliable narrator MS likeable. You should read it because it's such a good example of how to treat a complex character. BTW, some critics complain they never did like the character.
The thing is, different people can relate to different kind of characters. So there is one-size-fits-all protagonist-writing rules. Meaning, you are not put in a box, simply some people's preferences are closer to your preferences and some people's are farther. That said, to me the most important thing is that characters be believable. When I read and feel the urge to tell the author: "Get real, have you ever seen anyone act like that?" then it's a badly-written book. Being able to relate to the main character is also important, but second to believability.
I'm not entirely conviced your wife is right - "the light of day" and its sequel "dirty story" were both best sellers for eric ambler, and the former was also made in to a film. The protagonist arthur is an unlikeable pimp. They are also told from an unreliable narrator point of view so you have to filter arthurs justifications for his behaviour
Your protag has to be interesting, but that doesn't mean likeable. I'm sure we've all met people who are so unlikeable that you just want to leave and never talk to them again, but just the same there are plenty of people that you wouldn't want to be friends with whose brains you'd still like to pick. The beauty of a novel is that we can see inside their heads, and as much as someone might be an abrasive asshole on the outside, they don't have to be just as irritating on the inside.
Your wife is only one reader, no matter how many books she's read. Perhaps give your story to a few other people and see what the reaction is. If the reaction from most readers is that the character isn't believeable, that's something you might want to work on. If, however, the character is simply somebody a certain reader doesn't like, that's another thing. If you intended that character to be unlikeable, then you have achieved your purpose. Whether somebody wants to read the story or not is another issue. Just make the situation so interesting that they will overcome their dislike of the character and continue reading. The only time you really need to worry is if you intended your character to be likeable right from the start, and that's not happening. If that's the case, find out from your reader (if you can) exactly WHY they don't like that character. See what you can do to tweak the situation. You don't necessarily need to change the character, just alter the way you've written the character. Sometimes just a little bit of extra information ...even just a line or two ...can make all the difference. Strive to make the character's actions or attitude more understandable. Even a few lines of dialogue added (or removed) can change a reader's perception of a character. Play around with it. However, there is NO story on the planet that everybody likes. This story may never appeal to your wife. Sounds like she likes a more idealised character—not wanting to read about a character who is 'ugly' is a clue, isn't it? She'll probably prefer a different kind of book.
Characters don't have to be likeable to be relatable. Rorschach from The Watchmen is an ugly, disgusting, human being, but he's also fascinating, highly intelligent, and his dedication not only to his work, but also to his morality are characteristics most of us either relate to, or aspire to. Even if his methods are extreme, his character is presented in such a way that readers would feel justified in doing the same if put in a similar situation.
funny you should say that. I was just about to come back to this conversation and comment that the PT might just need to be compelling. You beat me to it with "fascinating." Well played.
I would consider having other people read your work to get a better idea if what you're doing is working. Not everyone is going to like the same type of writing or development. Personally, I like conflicted main characters that are awful people—so long as they grow and change over time. I think for most readers, if you show them a redeemable quality in the character, and they're interesting, then they feel they can connect with them and continue reading.
When people identify characters as unlikeable, it is often because the character is not exerting a strong personality, rather than the ethics of what they do. People like villains too and it's not because they agree that the world should be dominated by a single dark lord. Try making the character's personality stronger and have them take initiative more.
One way to create a lot of interest in a story, if you have an unlikeable POV /main character, is to make most of your secondary characters VERY likeable. The readers won't like your main character, but they will like the others—and want to find out what happens to them.
Does she? Not every protagonist needs to be likable. As others have said, they need to be interesting or entertaining, but not likable. Satan in Paradise Lost, Newland Archer in Age of Innocence, and arguably Lisbeth Salander in the Millennium series are ample proof of that. She may just not like ugly characters. I can't see a literary reason for it. But traits do need to work with the characterisation, not against it. If the character is ugly, other people in the world can't act like he's attractive because that's what readers expect of a protagonist. Buyers of an accurately-marketed book can, if it's done well, like nearly anything.