Ostensibly, even an audience comprised of reverential listeners would have plenty to hear, but in every performance I've attended the silence has been broken by the audience and become ironically noisy.
It should be noted that each performance was held in a concert setting, where any muttering or clearing one's throat, let alone heckling, was a breach of decorum. Thus, there was already in place in these settings, as in other settings of Western art music, a culturally specific mandate to be silent, a mandate regulating the behavior that precedes and accompanies musical performance. As with prayer, which has not always been silent, concertgoers were at one time more boisterous; this association was not lost on Luigi Russolo, who remarked on "the cretinous religious emotion of the Buddha-like listeners, drunk with repeating for the thousandth time their more or less acquired and snobbish ecstasy". 4'33" by tacitly instructing the performer to remain quiet in all respects, muted the site of centralized and privileged utterance, disrupted the unspoken audience code to remain unspoken, transposed the performance onto the audience members both in their utterances and in the acts of shifting perception toward other sounds, and legitimated bad behavior that in any number of other settings (including musical ones) would have been perfectly acceptable. 4'33" achieved this involution through the act of silencing the performer. That is, Cagean silence followed and was dependent on a silencing. Indeed, it can also be understood that he extended the decorum of silencing by extending the silence imposed on the audience to the performer, asking the audience to continue to be obedient listeners and not to engage in the utterances that would distract them from shifting their perception toward other sounds. Extending the musical silencing, then set into motion the process by which the realm of musical sounds would itself be extended.