Aside from what is said above, rules are made to be broken is something that really applies to music making A LOT!
On the one hand, you can use mainstream techniques and sounds. If it sounds correct and if it fits neatly into a certain genre, you'll have a good chance masses will like your music. If it's quality ( a lot of musicians just go by if they or a friend likes what they hear, others will probably also like it). On the other hand, you can also look for that new sound. A totally new genre of music or a general sound or feel that makes you stand out from the rest. In trying to achieve the above, making music, there are rules but they're more like a general outlining. Everything is possible and you can do everything. Good chance you'll find something that really wouldn't fit in the basic rule book but that does appeal to a lot of people.
Think of people like Jimmi Hendrix for example. He basically broke the rule book with his style of playing the guitar.
Another example, look at how many sorts of scales there are out there. Usually, we work in a major or minor scale that consists of 7 notes. That scale has a pentatonic scale of 5 notes that'll pretty much always sound good when played in that scale. That will accommodate a very large proportion of modern music. But, with so many countries around and so much music around, not everyone is set so much in major or minor and doesn't come further then that. Take india for example (oriental music vs non oriental music). In Hollywood, a movie is considered succesful when 300 million ppl hit the theatres. In India, when only 300 million ppl hit the theatres, the movie is a failure
. Back to scales, there's a ton of different scales that'll work as well. Normally, in a C Major scale, it'll sound weird if you start playing a flat or sharp note. In certain other scales, it'll sound alright. Think of scales like the dominant 7th, an oriental scale, a cowboy scale, etc. There are about 29 scales that I can think of. There's a certain theory behind what sounds good together, but if tomorrow somebody comes with a new scale that works, makes a new track with that scale and get's massive playback, big chance we'll have a new scale
Music theory (rules sounds so compulsory) is a nice guide line and it'll help you to get where you want to get. But, don't see it as something set in stone. Trying to deviate from something well known can help you be creative and outstanding. It's easier said then done though. If you want to be famous and make music for the masses, there's always a fine balance between keeping it simple, real and recognizable vs being innovative.