The pressure of society even has an effect on an adult's self-esteem. ... Men with a family and job are more likely to have a higher self-esteem than those who don't. The images that society portrays on what is or isn't right have affected everyone in some way, shape, or form. It can either take you down or build you up.