Peter,Have you ever considered partnerships or joint ventures using a Self-Directed IRA? 

IRA service providers don't set the rules - IRS does.

 I am looking to become a private lender and have heard about setting up a self directed IRA so that the interest can go back to the IRA and grow tax free, but....What about the set up fee, which I understand can be $1200+ and then quarterly fees?

You are ultimately creating a transaction between yourself and your IRA.  

There are couple major problems: IRA account holder is considered to be a "disqualified person" to the IRA itself and the rules prohibit any direct or indirect transaction between qualified plan and a disqualified person.

Where do you have your Self Directed IRA?

They can also be a great way to build your retirement funds by investing in them using a Self-Directed IRA.

I wrote about this topic before: https://www.biggerpockets.com/forums/51-tax-legal-issues-contracts-self-directed-ira/topics/996587-explained-tax-consequences-of-k-1-syndicationsPreferred return refers to distributions.

Use the money I have in one of the IRAs to open up a self-directed IRA, have the self directed IRA purchase the property and immediately seller the property tax free, the. move the money back into the original IRA, close the self directed IRA.3.

You may want to talk to your accountant on whether it makes sense to some of these strategies in a Self-Directed IRA.