I do have a landlord/tenant policy and carry an umbrella - but the LLC is not a named insured.

You may need to contact another lender to see their policy. 

One of the things you can do to protect yourself is get an umbrella policy. 

If I own two rental properties in the same LLC, and the liability limit of the policy is, for example $500,000, does each property get it's own separate liability limit of $500,000, or is the $500,000 the max for the entire LLC and it's assets?

Try Allied and see what they say.One problem I have had is having my umbrella policy cover my LLC's but I guess that kind of makes sense given an LLC is technically a separate entity.

The cost of the seller's title policy as well as your own. 

Or should we buy it under our names and purchase insurance policy + umbrella policy?

If however you have other partners in the LLC or the LLC is used for other business activities you can simply remove liability from the landlord policy and write a separate commercial liability policy. 

Landlord and tenant are co-insureds under the fire policy.

The lender will have a sales policy, laid out in a bid document, made available to person interested in making offers.