I've been experimenting today with the review features. The close vote review queue is in bad shape, as it contains over 100 items. This makes it not terribly useful because most of the questions that appear are old. It would be much more useful if there were fewer questions on it. In particular, there were several of old questions that were obviously duplicates but nonetheless were stuck at 3 or 4 close votes.
I was wondering if anyone had any ideas about how to clean this queue out a little so that it will become more useful. I had a couple ideas myself which I will mention as answers to see if people want to vote them up.
[Update]: As of January 2019, the close vote review queue is normally about 300 now. Is it possible that folks who have access to the review queues get their 1k reviews in for the Steward badge, and then stop reviewing? Perhaps we could add more badges at higher levels of reviews?