Oh man, New York has more vinyl per capita than anywhere else in the world! At least, that's what someone at a bar told me once. Either way, there are A TON. I still discover new ones every so often just walking around. Here some easy ones off the top of my head:
Turntablelab Manhattan/East Village @ 7th St near Ave B.
Fat Beats Manhattan(I think it's in the West Village, I haven't been here in forever, but I hear it's still cool for DJs)
Academy Vinyl Manhattan/East Village @ 12th btw 1st and Ave A.
The Sound Library Manhattan/LES, Ludlow Street, (Maybe about two blocks south of Houston St)
A-1 Records Manhattan/East Village @ 5th or 6th? Between A and 1st? I always walk there, but I never pay enough attention to the actual location.
Good Lookin' Manhattan/LES @ somewhere near 1st and 1st? ("How can the same street intersect with itself? I must be at the nexus of the Universe!")
Big City Records 12th near Ave B (I've been picking up some great classic Hip Hop LPs from this place lately)
These are a little more rock related but maybe that's what you're after:
RockitScientist Manhattan East Village, St Marks Place (8th Street) near 1st Ave (I believe. It's next to that tattoo shop with the naked blue mannequin on the curb)
Cake Shop Manhattan/LES @ Ludlow Street a few blocks south of Houston. It's a coffee shop in the front, record store in the back, and a music venue downstairs.
Other Music Manhattan/WestVillage @ you can look it up for the exact location, I don't know off the top of my head, but it's a really popular place. It should be easy to find.
Some Place I Don't Know the Name Of... On Avenue A, south of Tompkins Square Park, but before the the little strip of bars that run to Houston (Double Down, The Library, Nice Guy Eddie's etc...). East side of the street.
Oh man, I could keep going, and I haven't even gotten out of my neighborhood, yet. Those are all pretty close to each other, you can hit them all in an afternoon.
One more for the road: If you've got the nuts, and don't mind making one of those painful commutes on the G Train, you can take it to the Greenpoint Ave stop in Greenpoint (the neighborhood), walk north on Manhattan Ave to somewhere between Huron and Freeman St until you find a place called The Thing. West side of the street. Their back room is flooded with bookshelves stacked to the brim three times with records for $2. And then you go downstairs to the basement where you'll find the most overwhelming collection of used vinyl you've ever seen in your life. Hundreds of thousands of records easily. No order, a few little record players for checking stuff out. It's so packed, you can move a bookshelf, and find another bookshelf full of records behind it. I used to spend whole Saturdays at the place, and still never made it through all the stacks.
EDIT: I notice you're the guy looking for Tutorial DJ DVDs as well. I thought I'd point out that Turntablelab has a nice collection of DJ related DVDs for sale. They've got the qbert stuff, plus a lot of others. They also have a pretty complete collection of break records that are still in print.