You can listen to music as background noise. Basically, you are not listening at all, but using it as a sonic backdrop for some other activity. You might not even want music with lyrics, because you are studying or concentrating on something else.
The second kind of listening is when you are actually listening. That is your primary activity. Say, when you are at a concert and sitting there. Your mind might wander all over the place, but the music is not secondary to you.
A third kind might be when you are dancing, or doing something else for which the music is a necessary accompaniment.
A fourth kind of listening is when you are attending closely to music in an analytical way. Suppose you were writing a review of an album, or trying to transcribe a solo, or comparing two interpretations of the same work.
None of these kinds of listening is bad, per se, and I practice all four. My point is that what we call listening to music encompasses a wide range of degree of attentiveness, from not hearing the music at all to dedicating one's full attention to it.