Containers can be of all forms, including chests, cabinets, urns, barrels and more. When you open a container, the containers menu appears. Items above the divider line are in the container, and items below the line are in your inventory.

You can select individual items and take them, or, you can take all of the items which can save you a lot of time. However, if you are reaching your carry weight limit, you might want to just take the gold or the lighter items, and leave the heavier items in the container.

You can also store items in a container. Just select the item and press the action button to transfer an item from your inventory to the container.

WARNING!
Do not leave things in containers you do not own! They may not be there where you come back from them. If you want a place to store items permanently, consider buying a house.

It is useful to store things in containers which  you do not need at the moment, but think that you could use in the future. Since there is no point carrying an item not in use, as it will limit the other items you can carry. Makre sure to put it in a container that belongs to you, otherwise they could be gone next time you check. Your containers are usually in a house you buy, and some quests also result in you getting additional living quarters which you can store items in. Remember which containers you are using, since Skyrim is a big world and if you forget then you might spend a lot of time having to search through all the containers...