i don't plan on living in the property but would like to make sure its safe for my tenants.My realtor thinks i should look to a nicer part of town instead but this means i would have to spend more time saving to come up with the down payment since those properties are at least 50-60K more and dont necessarily generate that much more cash flow.

Is real estate something that I can do and just get started on my own, or do I need to spend all this money to be part of a community?   

@Christopher Short I would say if you plan to be a buy and hold investor instead of spending money on a management company, learn as much as you can about systematizing your business so that you can get great returns and not have to spend much time running your properties, especially when you only have one.

I would research your particular materials and construction and determine the likely risk, but I would seriously consider if your tenant is worth spending this kind of money on and what your plan would be for the tenant and project if you actually have a documented case of asbestos, which may require professional mitigation if you think you might have a liability situation with your tenant.  

When you're paying off in 5 years without changing spending habits, it's really not an interest rate sensitive loan.

I was listening to a BP podcast and the speaker (I think his name was Charles) was saying that if you want to be a good investor you should spend more time learning/figuring out the financing for the property over the property itself.

I would suggest though that you spend some money on a good real estate attorney and have him draw the documents up, meaning the note and the deed of trust. 

I was seeing that my maintenance head was spending 5  plus hours a week at Home Depot, Lowe's and other vendors, mostly picking up the same items. 

If this is a one off flip it may be different but if you plan on creating a long term business doing that your reputation could take a big hit justified or not.With court there is the time you spend on filings,attorneys,preparing for a case, going to court,etc.

With enough passive income to support your lifestyle, you can spend more time doing what you enjoy.