Columbus didn't think he was in India as we think of India today. Instead he thought he was in the Indies (what we'd call Indonesia today). Upon his return to Europe, he wrote a letter to be sent ahead to King Ferdinand, saying "Since I know that you will be pleased at the great victory with which Our Lord has crowned my voyage, I write this to you, from which you will learn how in thirty-three days I passed from the Canary Islands to the Indies."
With later voyages by additional European explorers, it was eventually learned that the Caribbean Islands and Indonesia were not part of the same archapelago, resulting in a distinction being made between the East Indies (Indonesia) and the West Indies (the Caribbean). During Columbus' first voyage, however, this distinction wasn't made yet.