He cannot come to the party, more's the pity.
more's the pity
idiomIt is a pity; it is unfortunate.
The library is closed on Sundays, and more's the pity, as that is my only free day.
The old theater will be demolished to make room for a parking lot, and more's the pity, for it holds so much history.