I get a little red dot on the top bar and when I click on it I see another red dot for "review close votes".
How/when/why does this nag me to do so? What's the deal with this?
As mentioned in a comment this is to draw your attention to the fact that there are relatively many pending review tasks.
Thus, if you see it and are willing to help out click on it to visit one of the review queues.
The details of what relatively many means are explained in this answer: How does the review-needed indicator work exactly?
This site is in the "large" site category. Briefly, for each queue there is a threshold and if one is above it the global indicator appears (for close votes it's 200 for a red dot, and for the others it's 10 (reopen, edits), 20 (LQ), or 30 (LA, FP)). A gray dot just means there is an item.
Note that the feature itself that the top-bar nags you when there are many items in the queues is not new to the redesigned top-bar. In the old one there appeared an orange box with the rough number of items to be reviewed written in it.