In the present tense, you might say, "If this doesn't work, I give up." How would you write that in the past tense? "If this didn't work, I gave up"? "If this didn't work, I would give up"? "If this didn't work, I would have given up"? The second seems most correct to my instincts, but I'm not sure since "I give up" is a set phrase of sorts.
This is subjunctive voice. Your initial example is future subjunctive. More accurately, you would say, "If this doesn't work, I will give up." The underlined part is the future subjunctive clause, and the second clause is simple future tense. For a past event, you would say, "If this hadn't worked, I would have given up." The underlined clause is pluperfect subjunctive or past conditional, and the second clause is conditional perfect tense. The event which differs from the recorded past event is referred to as the counterfactual event. Because the first clause is counterfactual, the ordinary past tense forms do not apply. Since the triggering event did not occur, neither did the consequence.
That isn't correct for the sense of the sentence. It would be correct if the novel were written in the narrator's past (i.e. "I'm telling you a story that happened to me"), but this is narrated as if recording the protagonist's thoughts at the moment. (I believe that's called 'progressive', but I may be mistaken.)
Sorry, but that makes no sense. His thoughts at that moment are as you originally stated, or the "corrected" version ("If this doesn't work, I will give up."). Trying to make it past tense requires the past conditional form, for the reason I pointed out in the last paragraph. Progressive tenses indicate ongoing action, not hypothetical actions.
If I were writing a formal report I would opt for "If it hadn't worked, I would have given up." I'm quite fussy about conditionals actually, and think that US English* is leading the way in some disasterous misuses nowadays. However, there are situations in which I would consider this to be too formal. If I were writing about a hardened criminal from a first-person perspective, I would personally go with "If this didn't work, I was giving up." I think that in practice different writers would write the sentence in different ways. *There are some ways in which I prefer US English though. Get-got-gotten (as opposed to the British get-got-got) seems more logical and fits better with, for example, forget-forgot-forgotten. The spelling is generally more logical too (excepting "aluminum", obviously).
it's hard to be specific with a sentence taken out of context... you need to post what came before it, so we can make sense of what it is you're referring to... in fact, the whole paragraph needs to be looked at to adequately judge the effectivenes/correctness of that one sentence...
Okay, I can see what you mean -- If he were saying, "Whew, if that hadn't worked, I would have given up", that would be different. But in this case, it's saying, "I was about to try something crazy, and if it didn't work, I was going to give up". Outcome unknown.
I agree with Cog and maia. Cog shows how to use the 'if' situation, but as maia says, we need to know the context properly to help with a specific example. Is it reported dialogue, a character's thoughts, or basic reporting of fact?
