Identify heirloom mount drivers without list of IDs

[MunkDev]

So I have this peculiar issue where I need to be able to identify heirloom mount drivers, mercenaries and other NPCs that are glued to you or behaves sort of like a pet.

In essence, what I'm looking for is a robust way to distinguish NPCs that are tethered to me without a blacklist.
I'm using the following code to process units and check if they seem to be NPCs.

local BLACKLIST = {
    ['89713'] = true;   -- Koak Hoburn, heirloom mount chauffeur
    ['89715'] = true;   -- Franklin Martin, heirloom mount chauffeur 
}
 
local function IsNPC(unit)
    local guid = UnitGUID(unit)
    if not guid then return end
    local unitType, _, _, _, _, ID = strsplit('-',guid)
    return  not UnitIsBattlePet(unit) and   -- unit should not be battlepet
            not UnitIsPlayer(unit) and      -- unit should not be player
            not UnitIsMercenary(unit) and   -- unit should not be mercenary
            UnitIsFriend('player', unit) and    -- unit should be friendly
            (unitType == 'Creature') and    -- GUID should start with Creature
            (not BLACKLIST[ID])         -- check with blacklist
end

Looking through the API, I can't seem to find a function that tells me if a unit/GUID is part of the game world or following me around. Is this at all possible to find out?