After you have logged into the site, the administration toolbar appears at the top of the screen. An additional administration menu is available on the left and it can be exposed / hidden by pressing the icon.
It is not necessary to log out of your site when you are not using it. However pages may load more quickly when you are not logged in due to page caching.