The sample rate is the rate at which a "slice" of a sound wave is sampled per second. To be able to reproduce a wave, you need at least 2 samples of it, so your sampling rate determines up to what frequency you can sample a sound (it's roughly half the sampling rate, if we are at 44.1kHz the highest frequency you can capture is around 20kHz, look for the nyquist frequency as mentioned by unsystematic). The advantage to having a higher sample rate is that you have more high frequencies saved, so more harmonics, and better quality. Of course sample rate doesn't mean much without the (EDIT) bit depth (the size of each sample).