The Bitcoin network is a distributed ledger (blockchain) distributed by a P2P network and updated by miners, that creates and tracks digital coins. Every coin has a face value (size) and note indicating which credentials are needed to spend it (address).

Transactions happen in Bitcoin by destroying old coins and creating new ones. If Alice wishes to send money to Bob, first she gathers coins whose total face value matches or exceeds the amount she wants to send and adds them to a transaction. In the transaction she then uses her credentials (private key) to prove her right to destroy the coins she wants to spend. She also creates a new coin for Bob that has a face value of the amount she wants to send and is tied to his credentials. If there is any leftover value in the coins she has destroyed beyond what she wants to send to Bob, she also creates a new coin for herself containing the excess value. This is called a "change output".

If Alice wants the transaction to be processed by miners with a higher priority, she can create a slightly smaller change output for herself than what she would otherwise be allowed. That difference is known as a "transaction fee" and the miner who adds her transaction to the blockchain is allowed to keep it.

For privacy and security reasons, addresses should never be used more than once. This means that a Bitcoin user will accumulate a large number of credentials over time. These credentials are stored in software known as a "wallet". A wallet is a program that securely generates and stores credentials, and also watches the blockchain to keep track of which coins are yours to spend. Sometimes the word "wallet" is also used to describe the file on your computer that stores the credentials themselves.

A wallet typically displays the total face value of all your coins as a balance, without showing you the individual coins themselves. Wallets allow you to send and receive money without worrying about the details of each specific coin because they handle all the details for you.

Because wallets store the credentials needed to spend your coins, it's very important to protect them from loss by backing them up. If the credentials associated with a coin are lost, those coins will be unspendable forever. What you need to do to backup your wallet depends on the type of wallet you are using.

Random wallets like Bitcoin-Qt generate new random credentials in groups (typically 100 at a time) as old credential are used up. Because of this you must constantly keep making new backups or else you are at risk of your backups becoming outdated.

Deterministic wallets like Armory generate new credentials from a secure formula, and only need to back the parameters for that formula.  Backups of a deterministic wallet are valid forever, and so only need to be created once.