"Mary on a Cross" features a distinctive musical structure, characterized by a soaring chorus and a dark, driving verse. The song's instrumentation is marked by the interplay between Tobias Forge's (Ghost's primary songwriter) crunching guitar riffs, Papa Nihil's (the band's bassist) pulsing bassline, and Cardinal Copia's (the band's vocalist) expressive vocals. The song's arrangement is notable for its use of dynamics, shifting from quiet, atmospheric moments to explosive, anthemic choruses.