Answer:
estudiar
Step-by-step explanation:
estudiar is the correct answer because it is the act of going to study
in the sentence (translating from Spanish to English) it says "my brother will ____ in the library."
you wouldn't put estudio because it is in past tense, meaning that person has already studied and you wouldn't put estudia because it is present tense meaning that person is currently studying.