Worth Paying 50% of Remaining Loan to Refinance to Lower Rate?

[Bill Baldwin]

Refinancing question for the BP community.

I currently have roughly $197k remaining on a seller finance loan for a recently purchased $300k property in Bakersfield, CA, at 5% amortized over 15 years (but I need to pay off the remaining principal balance at the 8 year mark).  I've been speaking to a local bank in the Boston area (where I'm originally from) and they've agreed to loan me $100k at 3.07% over 10 years for a refinance.

My question is, do people think it's worth a refinance to a much lower rate if I have to put down roughly $97k?  Have to consider this payment is in addition to my initial down payment when I purchased a few months back, and would eat up most of my available cash at the moment.  Pros would be higher cash flow, paying off the loan quicker, and significantly lower monthly interest payments; cons would be less capital to spend on possible future projects in the short term.  There are obviously a lot of other factors in play here, but taking the above scenario in a vacuum, and given the current market and financing options, etc., I'd be interested to hear people's opinions.