Ensure local invited & knocking users leave before purge. (#16559)

This is mostly useful for federated rooms where some users
would get stuck in the invite or knock state when the room
was purged from their homeserver.