EACH piece needs back buttered as well as wall then applied and spaced and the thinset (NOT MASTIC) scraped out of grout joints. 

If you go tile, start small with a backsplash or something and don't use mastic, but use real thinset that you must mix.

I agree that tile is the way to go but even with a good mastic, if the subsurface isn't solid the new surface won't be either.

I doubt this is a cheep fix,, where you gonna find a tile to match..Your going to have to grind out mortar, mastic a  new one in that I doubt you will find a match for it and then re grout..it's in a worse than awful spot on an inside and in a corner.. you'll probalby have to take several tiles out..tile work is tricky once you start to force one tile the others chip..  

Many of the older vinyl floor tiles and any associated paper-like backing, mastic, adhesive or glue, may contain asbestos.

That said, the liability of knowing about the presence of asbestos in the living area or HVAC system (ie in plaster or floor tiles, duct wrap/mastic), but not disclosing it could come back to bite you if something ever did occur.Much of our building stock is old here in the east - 1920s-1930s would be newer buildings in some neighbourhoods.  

So I tried removing some of the mastic and glue with Citristrip last night.

Mastic" is a generic term for an adhesive.

I would bet that block was tarred with a waterproof mastic coating before the ground was backfilled around it. 

I've been looking into Mastic and Shirley area but your current about Long Island, tough to get starting in.