You can demonstrate that a piece of chalk has mass by putting it on a pan scale. Pan scales determine mass in grams. Also if the mass feels like it has some weight to it (although not to be confused with weight, they are two different things!), it most likely has mass.
Hope this helps!!