So aside from perception of pattern, meter, and tempo, there have to be other more nuanced perceptual qualities to rhythm. Just off the top of my head, let's say there is expectation and anticipation, or the forward projection of where the rhythm is going, along with the fulfillment or disappointment of these expectations. There is the perception of being on top of the beat, dead on it, or a bit behind (rhythmic feel), and the sensation of a soloist floating over the meter with a broader beat. (How broad or narrow is the beat?). There are sensations of kinetic energy (I want to dance! is the music driving forward, swinging, bouncy maybe? Is the energy more vertical, bouncy, or horizontal [forward moving]) or rhythmic interest or boredom (monotony, variety of patterns). There is harmonic rhythm (how fast the chords are changing or not changing underneath the melody), and the rhythmic phrasing of melodic ideas. There is the complex play of symmetry and asymmetry, tension and release, regularity and fluidity, the perception of structure over longer periods of time (feeling the piece's entire structure as a rhythm.) We could say a very crude idea of perception would ask the questions of fast and slow (tempo), triple or binary pattern (meter) and repetition of patterns. If the brain cannot perceive all of that on a basic level, then these higher level nuanced perceptions won't come into play.