If you're new to this, I would suggest you start by reading
Tweak's Guide. It will point you in the right direction.
As far as what I would suggest, I think you would probably need a MIDI keyboard, audio interface, and some software. I would suggest getting the
Alesis PhotonX 25.
It's a MIDI controller keyboard, and it has a built in audio interface. It also comes with a free copy of
Cubase LE to get you started. The whole package only costs $200 brand new. You could learn on that version, and even make some hits with it, and once you get the hang of it you can upgrade to a better version of
Cubase. You can see what the differences between the version are
here. They're all really good with MIDI, and also really good with audio. You could run software plugins/instruments with it, and if you ever get any hardware, it works really well with that too. You can also record vocals with it, and mix, master, etc. Here's a screenshot (and link to their website if you click it):
And some good software to run with
Cubase would be Native Instruments
Komplete 3. It's expensive though, so some other good options would be:
Native Instruments
Kontakt
And Native Instruments
Battery.
Another really good program that works well with
Cubase is IK Multimedia
Sampletank
Here's some videos of them too, so you can see how they work:
Video of Cubase.
Video of Kontakt.
Video of Battery.
Video of full beat being made using an old version of Cubase, Sampletank, and SR-202 (a free program similar to Battery but not as good).