The connection that the author Ralph ellison draw between symbols and language itself is that Language is naturally symbolic. The exact same words could carried out a different meaning depend on the context, and who are the participants.
Hope this helps