This is an example of constructive processing.
Constructive processing is the concious examination and absorbtion of outside stimuli that results in knowledge gained or action taken. This basically means that our memories can sometimes contain errors or distortions. Therefore, both your memory and your brother's memory might differ from one in another based on how each of you have distorted the memory over time. Even though both of you are claiming you were the innocent victim of the other, it is likely that this is not the case. One, if not both of you, have likely distorted the memory which has led you to believe that you were the innocent victim when this was not actually the case.