Poor physical health can result in an unhealthy body composition, which in turn, may lower your self-esteem. A low self-esteem makes it more difficult to create healthy social bonds. Poor physical health may also make it more difficult to participate in many social activities.