Can I split off a portion of my IRA owned land to build a home?

[RA McManis]

If my self directed IRA owns raw land, is there any way I can build on part of this property? I own about 100 acres and would like to build a home of the property. Currently it is just a raw land used for nothing other than an investment. I would like to split a portion, maybe 5-10 acres off and build a home. I would still hang on to the remainder to sell later. Would it be possible to split off a small portion to build a home, if I pay taxes and the penalty (before 59 1/2)?