To understand the ways in which a quantum computer is different from a classical computer you must first understand the rudiments of the classical computer. The most fundamental building block of a classical computer is the bit. A bit is capable of storing one piece of information, it can have a value of either 0 or 1. Any amount of information can be encoded into a list of bits. In a classical computer a bit is typically stored in a silicone chip, or on a metal hard drive platter, or on a magnetic tape. About 1010 atoms were used to represent one bit of information in 1999. The smallest conceivable storage for a bit involves a single elementary particle of some sort. For example, for any particle with a spin-1/2 characteristic (such as a proton, neutron, or electron), it's spin-1/2 characteristic on measurement will be either +1/2 or -1/2. We can thus encode a bit using a single particle by mapping 1 to be spin +1/2 and 0 to be -1/2, assuming we can measure and manipulate the spin of such a particle. If we were to try to use this spin-1/2 particle as a classical bit, one that is always in the 0 or 1 state, we would fail. We would be trying to apply classical physics on a scale where it simply is not applicable. This single spin-1/2 particle will instead act in a quantum manner. (Williams, Clearwater)
This spin-1/2 particle which behaves in a quantum manner could be the fundamental building block of a Quantum computer. We could call it a qubit, to denote that it is analogous in some ways to a bit in a classical computer. Just as a memory register in a classical computer is an array of bits, a quantum memory register is composed of several spin-1/2 particles, or qubits. There is no particular need for the spin-1/2 particle, equally well we could use a Hydrogen atom, and designate its electron being measured in the ground state to be the 0 state, and it being in the first excited state to be the 1 state. There are a multitude of possible physical qubit representations that could work. For simplicity I will discus only the spin-1/2 particle from here on, but analogous arguments could be made for many things.