Improvisation as a kind of extra-territoriality on the other side or on this side of time and space, these (time and space) are necessarious/unavoidable coordinates for the rationality and for the physicality of the acustic event, but paradoxically they could be ‘broken’ by improvisation…
In that you seem to be asking if time and space can be broken by improvised music, I believe it can “break” the usual way in which we are used to perceiving time and space.
There is a pulling together of events using sounds or words or visuals. . . a juxtaposition of emotion as it connects to events, or juxtaposition of events (time and space) as it connects to emotion. Improvisers do allow themselves the territory of freedom of perception. We do go beyond what society allows us to perceive and express, generally speaking. We also may express what others do not have the words for. We have the ability to put their quandaries and perceptions into a setting where the listeners dilemmas can be spoken about and unraveled. Music is, by definition, is an expression of experience that cannot be articulated in words. Answers and vision in life can come from so many different places. Music can simultaneously access these places.

Now let’s consider the drums respect to the visual level: drums consists in different parts and can be played with different instruments – sticks, brushes… and in different ways and directions. There is a continuous  lights  reflections absorptions effects from lucidity to opaque. And also there are the shadows or shadow fragments effects and movements. And all this being a kind of movement on the place: so a continuous dialectic between movement and stasis…
You have to remember and consider that what the drummer does is not, generally speaking, visually premeditated. He is, for the most part, doing his job and hopefully what comes out audibly is what he intended but what comes out visually is only a by product. The stage and lighting can make what the drummer does look very physical, magnetic and appear to be a physical dance to the music that’s going on but it is usually not intended to appear that way. At the same time in stage music production they may want the drummer to appear as if he is being physical but that is usually just for the audience.