I have other loan officers challenge me on this all the time.

I live in the area and have found it challenging to make the numbers work on a 2-4 flat. 

I want to be able to make them an offer for their house but it's a challenge for me when the realtor is involved.

Although, the difficulty in finding buildings that will allow (or won't find out) about airbnb'ing can be the most challenging part!  

I completely understand your frustration and am very sorry for the challenges you have encountered. 

It will be a challenge when your SO isn't on board (yet).I am kinda in the same boat, my SO isn't into real estate investing, but I am.

When it comes to older office, in particular multistory office buildings, the challenge is often looking beyond 'it pays a decent return' and digging in to understand what is required to day to keep vacancies low, and equally or more importantly what will be required tomorrow to attract or keep tenants?

I can tell you that my most difficult or challenging thing is contractors.  

If yes, that's where the challenges begin.

You actually can decide to reject a applicant for any reason you choose provided when challenged in court you can show the applicant you did pick was better qualified.