There are a few things in your post to clarify first but...
"Annuity from an old job" typically means a pre-tax 401k. More likely a 403b from a school, or hospital if it's in an annuity. What type of employer was this from? IE- this is likely a "qualified" account.
Most people's definition of a brokerage account is a taxable investment account. IE- non qualified.
You "can" roll your old qualified account to a non-qualified account, but you will be taxed on the entire amount at ordinary income rates. Plus 10% if you're younger than 59.5.
Most common situation is you would roll your old 401k/403b "qualified" plan into a "qualified" traditional IRA. This would avoid taxes. You could potentially withdraw $10k for a first time home purchase and avoid the 10% penalty.
Look at the account statement from your old plan. If it says "Jesse Goswick, (401k, 403b, 457 etc.)", it's a qualified plan.
Look at the account statement from your brokerage account. If it says "Jesse Goswick, tod, jnt or jtwros", it's a non qualified plan.
There's a lot more to this, hard to put all in a response. Reach out to me if you have questions. I'm always willing to help out a CC resident :)
