Your Question has been placed on-hold as too broad. As one of those voting to close as "too broad", my main suggestion for improvement is to narrow the problem statement substantially. Before going into that in more detail, though, here is a more basic advice:
1. Do more research
The title and tag locally-compact-groups can be taken as indicating this as your primary focus. The alternate tag harmonic-analysis might also be your focus, given the mention of a book on Fourier series.
Your final sentence is "I still don't understand much what that group mean though." I would suggest starting with the Wikipedia article on Locally compact group, which supplies a definition and some surrounding discussion of examples and properties.
You will need to understand topological groups first to get the definition of a locally compact group, i.e. a topological group which is locally compact and Hausdorff as a topological space.
2. Narrow the problem statement
The current body of your Question charges off in all directions:
It also seems to me that talking about it is talking about locally compact groups. So to have a big picture, can you explain to me how the local and non-local compact, compact and non-compact groups play a role in many branches of math?
Once you've made an effort to research definitions and so forth, I think you'll appreciate better the futility of trying to respond to such broad requests. Math.SE is designed to delivery excellent content regarding the "leaves" of mathematical knowledge rather than the "tree limbs" in totality. From the site FAQ:
Your questions should be reasonably scoped. If you can imagine an entire book that answers your question, you’re asking too much.
This is not to say there might not be other Web resources out there that can help with a "top down" search for information. In your Comment above you "remember that there is a website listing all math topics in a network." This reminds me of Dave Rusin's Mathematical Atlas (Wayback Machine snapshot), which is not currently online. In any case the "bubble" 22 is for topological groups and Lie groups, and it has a subtopic 22D for locally compact groups and their algebras.
In summary Math.SE does not aim to replace all the Internet math-related resources or to do your work in searching them out. Instead it aims to give curated content that addresses Questions in reasonably concise and definitive Answers.