If I ask myself to sing a note in the middle of my speaking / singing voice, I will come up with a G3 most of the time. It will be around there, sometimes an F#. My ear is not particularly good, but the voice itself acts like an ear. People who can play an instrument by ear can essentially treat another instrument as though it were their voice. When people can't match a pitch played on a piano in their voice, we say they have no ear. In other words, if you cannot match the pitch with your voice, we say that you can't hear it.
The ability to hear is more important than the ability to produce a sound, so music is more about perception than production.