Pardoners are ones with faults, or are saying sorry. Thus they are looked down upon. You ever had that moment in life where you made a mistake in front of everyone, and they all just start staring at you as if you were a stranger? Exactly why pardoners are looked down upon.