you would use the word “but” since two very different actions are being described! “went to work” is the exact opposite of “went home.”
basically, you would use “or” if you were CHOOSING something (for example, “Do i go to work or do i go home?” and “and” if the statements were similar (for example, “Stacy and I went to work.”).