The context is a conditional sentence where regret happened (or failed to happen) in the past and is, therefore, impossible to change. So, the third type of conditional structure followed by past perfect will be used.
So, the blank will have the past perfect form of the verb Had/had not + V3.
"Didn't miss" in option A is incorrect as this is used to indicate the simple past tense. 
"Hadn't missed" in option B is correct as it takes the past perfect form. This indicates an action that is impossible to change. 
"Wouldn't miss" in option C is incorrect as it is used in imagined conditions in conditional sentences. 
"Wouldn't have missed" in option D is incorrect as the conditional sentences use this in imagined conditions in future perfect tense.
Thus, option B is correct.