Very often, when I discuss the issue of exclusive psalmody with people, once we are past: "But would that mean wouldn't be able to sing 'Amazing Grace'?" The next question is something to the effect of "What about Jesus? His name is not mentioned in the Psalms!"
Sometimes this question is even phrased as an argument, something to the effect that singing only the Psalms is somehow wrong or anti-trinitarian.
Pastor Ian Campbell provides an excellent answer to this question/objection (link).
To give you a taste of the article, here is it's opening paragraph:
I detect an increasing interest in the Psalms. Paul. S. Jones, in the essay, 'Hymnody in a post-hymnody world' says that "Singing psalms in worship is a biblical mandate -- not an optional activity' (Give Praise to God, p255). In the same volume Terry Johnson, in 'Restoring Psalm Singing to our Worship', asks: "What are the implications of a psalter in the canon of Scripture?" (p259), and concludes that "Our ancestors were psalm singers! The Psalter gave to their faith the bold, robust quality that we still admire today. A revival of their use has begun in our time... may they become a fixed element in the worship of evangelical Christians once more" (p286).
May the God of David and Asaph shine his blessings on us,