"This avenging justice belongs to God as a judge, and he can no more dispense with it than he can cease to be a judge, or deny himself; though at the same time he exercises it freely. It does not consist in the exercise of a gratuitous power, like mercy, by which (whether it be exercised or not) injustice is done to no one. It is that attribute by which God gives to every one his due, and from the exercise of which, when proper objects are presented, he can no more abstain, than he can do what is unjust. This justice is the constant will of punishing sinners, which in God cannot be inefficient, as his majesty is supreme and his power infinite " (Turretin's Atonement, Translated by Wilson. New York, 1859).
"So long as he is holy he must be just; he must repel sin, which is the highest idea we can form of punishment" (Hodge's Essays and Reviews, p. 137).
"For whatever else God may be, or may not be, he must be just. It is not optional with him to exercise this attribute, or not to exercise it, as it is in the instance of that class of attributes which are antithetic to it. We can say: "God may be merciful or not as he pleases," but we cannot say: "God may be just or not, as he pleases." It cannot be asserted that God is inexorably obligated to show pity; but it can be categorically affirmed that God is inexorably obligated to do justly" (Bib. Sacra, Vol. XVI., p. 738).
These quotations were brought to my attention by Pastor Daniel Fisk in his article "The Necessity of the Atonement," in Biblia Sacra, from April 1861 (link).