I would say go for it.
Don't try to talk yourself out of the deal. If the numbers make sense and there is some cashflow, pull the trigger. The probability of failing is very high but if you never fail then you won't learn.
Obstacles to overcome:
1. Financing
To get a mortgage from a bank might be tricky given your age but with your part time job, you might be able to fins some money or you could get a parent to cosign. Without bank financing your other option would be to convince the owners to owner finance the property to you which is very probable given your relationship with them. My best advice would be to go talk to several lenders in your area and just see what is possible, they will have a better idea of what you are capable of.
2. Converting back to a duplex
You will need to check with local zoning restrictions to make sure that the property hasn't been rezoned since it was converted. Then you will need to find a GC you trust and that is capable of the work. I think finding a GC is the hardest part of the RE investing puzzle and many experienced investors continue to struggle with it.
3. The pool
You already had a sense that the pool would be a liability. Personally, I would probably keep it there while I would be living there (if my insurance allowed) and then once I moved out, I would fill it in. I would having a harder time sleeping at night having the pool there while I was not on site. Additionally, the pool just presents added maintenance expense and cuts into your cash flow.
Best of luck to you and feel free to reach out with any other questions. I am always glad to help.
-BA