For example, in my case, why am I user119598? Does this mean that the number of registered users on this site is now around 120,000 and I was the number 119598?
The numbers are assigned consecutively to all accounts created on the site. With one exception, they are positive integers. The first accounts are created by SE developers in the process of putting the site together. Geoff Dalgas is user number 2, and there is no user number 1 (I don't know why). The first "real" (non-SE-affiliated) user of Math.SE was Noah Snyder, who has number 8.
Because many user accounts are deleted or merged for various reasons, the highest existing user number is not equal to the number of accounts. According to SE stats, there are currently 98227 accounts on Math.SE: this includes both registered and unregistered accounts. For more on what it means to be "unregistered", see this answer where I counted all unregistered users as of two weeks ago.
The main reason to know your user number is that many SE Data Explorer queries take it as a parameter. (There is a quasi-authentication system that pre-fills the number for logged-in users, but it does not always work.)
Another related point: if you use the "share" link under a post to generate its short URL, your user number is included at the end of that URL. This is done for issuing certain badges, but it's something to be aware of if you share the link at a place where you maintain a separate identity. If you want to share the link without being identified as a particular SE user, delete that part of the URL.