I hate to say it, but it sounds as if this person is critiquing YOU rather than your work. Maybe gently ask where in the work itself he thinks you've gone wrong? (If you haven't already done that.) Maybe take him a step further. What he's saying, in essence, is that no writer can write outwith their own experience. Which, of course, is nonsense. Any writer who writes a story with multiple characters is writing outwith their own experience. Is he saying that nobody can write a historical novel? Or that nobody can write a futuristic novel? Nobody can write a story set in another country, or with a character who comes from another country? Tell him that's what research is for. Also, writing from your own perspective doesn't allow for the kind of insight that outsiders can give. To paraphrase Robert Burns: Oh, if some power would give us the gift to see ourselves as others see us!