If you want some part of text not to be rendered, simply include it in backticks, like this: 
    
    `$ \begin{pmatrix} a & b \\ c & d \end{pmatrix} $`.

You will get: `$ \begin{pmatrix} a & b \\ c & d \end{pmatrix} $`.

So you could leave a comment like:

> To write down a matrix you can use the syntax `$ \begin{pmatrix} a & b \\ c & d \end{pmatrix} $` - like here $ \begin{pmatrix} a & b \\ c & d \end{pmatrix} $.

Double dollars are not a way to achieve this - they are used for centered formulas. Writing `$$ \begin{pmatrix} a & b \\ c & d \end{pmatrix} $$`  gives you  this: $$ \begin{pmatrix} a & b \\ c & d \end{pmatrix}$$

And - as you checked yourself - `\$ ... \$` does not work in this way either. You probably used this because you are used to this from LaTeX. (Still you can use `\$` inside math mode - as in `$\$$` which renders as $\$$ - but that's not what you need here.)