As water is cooled from 4 to 0, its density "decreases." Density is the amount of substance per a specific volume. So, if the water cools it's density decreases, but as the temperature rises it's density will increase because liquid to a gas and a gas has a high density then a liquid.
Hope this helps.