Acid rain can be very harmful, mildly harmful, or not harmful at all. It mainly depends on the density of the rain, and how bad the current air pollution in the area is. Also, it depends on how often the area gets acid rain. The first few times an area gets acid rain, it's nearly harmless. Then, the next few it can be very dangerous, but eventually gets weaker over time. More air polluted areas are more likely to get much more harmful acid rain.
Summary: Acid rain is usually mildly harmful, mostly only harmful to infrastructures.