It's so easy to walk from a good neighbourhood, into a bad one. Did that a few times but being a little street wise doesn't go astray. Everything just rolls into each other and where we stayed was a little rough at night (West Village...14th Street between 7th & 8th Ave) but only 2 blocks away was the Highline, Chelsea market, Meatpacking district and a few blocks south were areas filmed in Sex In The City (Famous cupcake store there, Browstones and the comedy stretch to name a few).
My wife tells me that the East village was less seedy at night but the West Village alters from block to block. Just do as New Yorkers do and DON'T look at anyone, just walk straight, no eye contact, a little sidewalk swapping late at night and you'll be fine. Always a challenge but I enjoyed acting like a rude local bastard for authenticity....
Harlem can be rough but becoming more touristy these days. Take a Harlem Bus Tour in numbers which will be fine too. I've walked as far as 110th street but Krudds is pretty much on the ball, although we found an awesome Japanese resturant a few blocks west from the park. $45.00 for a feast of sushi/sashimi, soup and a free glass of wine at lunchtime. Went there twice it was that good!
Oh, and wander Times square within close proximity to the Ed Sullivan Theatre to hopefully get free tickets to David Letterman. Not a fan of the show but being in the audience was upbeat and a fun way to burn up a few hours. Bruce Willis was one of the guest when we were there in July. They film around 2:00pm for memory.
So many things to do in New York and felt the 3 weeks we were there was just enough to see and feel much of it. Never a dull moment and always something to do.