Recommendations on building a more useful baluster for these stairs

[Justin C.]

We replaced the shoddy work of the previous homeowner and built these lovely steps.  From floor to top of landing is 32.5".  Rental compliance requires me to have a 34" Baluster on the right side of these steps.  My concern is its narrow already and putting a permanent baluster setup is going to make maneuvering furniture around the corner even more difficult.  Is there a good removable setup that is strong enough to maintain the structural integrity?   If not, the only other good and bullet proof option we have come up with is a permanent 2x4 wall with drywall with yellow pine cap.

I am aware that the gap needs to be closed as well.  A wall will solve this problem obviously and I can easily finish this out with sanded plywood.  Everything will be painted eventually.

Thoughts??