Is it a good time to refinance - detailed

[Daniel Winsor]

Currently living in a Duplex in Key West, Florida. Purchased on a VA loan, so required to live in the property until I move. Just read up that I can refinance using an IRRRL which would allow me to move out of the property to purchase a new one. Rates have come down slightly since I bought, so I could hopefully keep my PITI the same. I may have to pay $3000 in points in order to secure the loan, but that would quickly be made up in additional rental costs.

Duplex is a 3/2 and a 1/1. I live in the 1/1 and rent out the 3/2 for $3300 a month which pays the entirety of the PITI + $100. By living in the 1/1, I am living in KW, FL for nothing out of pocket, keeping my entire $3300 BAH. However, it's a large 1/1 and would reasonably be expected to rent for $1500-1800 a month. I self manage for the time being, so my expenses wouldn't rise appreciably. I am here for another two years at a minimum, but may have the option to stay for longer after my commitment is up.

If I do this, it would possibly free me up to purchase another home in the area using the rest of my VA money. Currently nothing on the market that would fit my requirements, but the one I bought never hit the market either. If lightning struck twice (I think I got very lucky), I would own two duplexes and be adding $1500-1800 a month to my cash flow.

Any downside to the IRRRL at this time? Are rates expected to fall and I could get a lower rate if I wait?