The downside will have more to do with your business personality.

You will get a ton more business, and their fees are easy to make up for...

Personally I think Josh himself is innocent but the company definitely is involved in shady and deceptive business practices.

Your IRA can borrow, but you have to be careful to not personally guarantee it.

Yours appears to be a business-purpose (investment) loan.

That would be a separate mortgage and not what amounts to a second loan on your personal residence, which if you have spouses, they might not be that into. 

I have a 2-person unit with a glass top dining table and it has worked well.

Which is a tax when you do an active business in your IRA but it is essentially the same thing. 

It would for sure have to be a higher hourly rate than a typical W2 job however as agents are 1099 contractors and there are expenses associated with maintaining our licenses and operating our businesses that usually come out of our commissions.

Have you calculated what your (monthly nut) is you need to survive business wise and personally?