Simply relocate those icons next to the awesome bar, for example.
To customize your Firefox UI go to Add-ons then in the Options dialog choose the desired options.
I hardly ever use this button.

You just need to do a little editing in your about:config page.
Setting it to ‘0’ will mean only the active tab has a close button.
For a few other Firefox-related tweaks you’re able to check out our previous tips: