Templates are the building blocks for a Sitecore installation, they're amazingly flexible and with capabilities such as inheritance you can produce an elegant architecture, or alternatively a complete mess. Here's how I like to structure things:
First off I have a folder of base templates. These are the building blocks for all the fields that will end up in a component data source or an actual page via inheritance. By setting them up as a base template we can re-use the same field definition for things like headings, but more importantly it keeps them focused on a specific purpose rather including details of the entire eventual item.
One additional rule I have for base templates is that they should only contain one data section within them. This in turn helps keep them focused on a specific purpose.
Next we have pages. You guessed it, a page template is what a content editor will create an instance of when they make a page. It's responsibility is to bring together default presentation details, the set of fields on the page and insert options for the pages beneath it.
All fields are inherited from base templates and standard values are used to define the defaults for the page.
Additionally to make the CMS experience as easy as possible for the content editors an icon should be set so that a page type can be visually identifiable.
A component template is the equivalent of a page template but for data sources.
Like page templates all the fields are inherited from base templates and standard values are used to define the defaults for the page. An icon should also be set to make content types easily identifiable by content editors.
Folder templates are often overlooked but they are an essential part of creating a decent user experience.
Folder templates are created to define the insert options for components and site settings rather than having them set on the site content tree.
Where relevant a folder template should also include itself as one of the insert options so that content editors can organise their content into sub-folders.
When a component needs some config to customise its look and feel that is not content, it can be better to use Rendering Parameters rather than a data source.
Site Setting Templates
A Site configuration template is useful to contain various global settings for a site. This could include things such as the Site Logo, Google Analytics account details etc. Settings should be segmented into logical sections that have been defined in Base Templates and then inherited.
It can also be useful to separate some settings into their own template item depending on the scenario of what the setting is for.
Group Into Folders
With each these template types created, you'll end up with a tree structure that looks something like this.