well then if its zoned for that and utls are there then carry forward.. its pretty tough in our markets though with build cost to make sense of building a duplex.. vis a vi rent.but I don't know your rental rates.. building costs are 100 a foot  so probably 300k or so to build it to rental grade plus lot.. so maybe your all in for 400klets say rents are 1500 a side  that's not bad..

However you are correct that you need to consider the cost of that money ie the payment when deciding if purchase makes sense. 

My lender says it doesn't make sense and to just keep paying more on principal if I wanna pay it down.

You also need to make sure you know your numbers its easy to think a deal makes sense but forget a cost or two. 

I have seen cars turned into BBQ grills, tables turned into desks, all kinds of things turned into light stands, and so forth.Just because a word or phrase is used for multiple things doesn't mean they are related in the sense that we are criminals because we take an old tire and turn it into a coffee table vs someone who breaks into the DMV and creates fake IDs. 

@Tom HermanUsually not a good idea.You'll lose the capital gains exemption.You'll only have access to hard money or commercial lenders.You would need to speak with an accountant and get a serious opinion on it to see if it makes sense...

When you put it that way, maybe selling makes sense.

So, the high property tax can make (somewhat) more sense.

Probably not.Of course, this would be different if you were looking for finishing money and the broker offered you something that made sense with an expectation to make it back after the sale.

Do you have budget for some minor cosmetic improvement on the property to "make sense of" the rent increase?