well coming from a softsynth->hardware->softsynth background i can tell you this: It really depends on the power of your computer and the sounds that you want. fantom patches load instantly, software in some cases not so much and it can bottleneck your PC/mac if you dont have enough CPU/RAM. a few years ago i ran into that problem a lot because i love to layer so i went and got a motif ES rack. this was before quadcores became affordable. I got rid of my rack when I bought a quadcore because I could really work just as fast and not worry about performance issues.

with that out of the way, heres my next bit of advice. A fantom has its own sound to it as far as stuff like brass, pads, FX for guitars and such go. you either like them or you dont. some hate the brass and dont think it's "real" enough. Others love it and it's the typical sound you hear in much of the southern rap singles people put out on the radio. The same goes for plugins like Philharmonik, again, you either like their brass/string/orchestral patches (catered more towards realism IMO) or you dont. which ever one sounds "better" to your ears is very subjective so I cant necessarily say go plugins or go Fantom just based on sounds. You should really demo the fantom out, if that inspires you to make tracks and it goes well with your workflow, get it and add on the plugins to complement it at wherever it lacks (ex: Silk for the ethnic stuff, although there is an SRX card for that). you can always sell/trade/take back the fantom after you have bought enough plugins that address whatever it lacks, not so easy if you are exclusively plugins.