It's the minor character Marcellus, and not Hamlet, who coins the phrase. There's a reason he says "state of Denmark" rather than just Denmark: the fish is rotting from the head down—all is not well at the top of the political hierarchy.
The Church of England is "Established" and thereby the State Church.
It's not just Denmark that has its problems. Prayers needed here.