the reason why the swineherd and cowherd described as downcast is: they are afraid Penelope will marry someone else.
Penelope is really well known to have a lot of suitors lining up to marry her and since Odysseus seems to be always have to travel, She's kinda inclined to choose another instead