The constitutional authority of congress to forbid discrimination in employment is based on the power of Congress to regulate interstate commerce.
When it came to interstate commerce, there was no employment discrimination there - both men and women could work, regardless of the race and skin color. This was the basis on which the Congress managed to ban discrimination in employment elsewhere.