You do not need to remove invalid tags because they are never added to the value of the original input nor they are saves in tagify.value (your Tagify's instance).
Also, the keepInvalidTags is by default set to false which removes the tags' DOM nodes (after a minor delay due to animation).

For tags' validation you can use the pattern settings (works just like HTML5 pattern attribute):

var tagify = new Tagify(inputElm, {
    // Validate typed tag(s) by Regex. Here maximum chars length is defined as "20"
	pattern: /^.{0,20}$/,  
}) 

Another option is to use the transformTag settings, which is a function which is called for every tag being added, before is has been validated internally, so here you can intervene and manipulate the tag data to force it to be marked as invalid:

var tagify = new Tagify(inputElm, {
	transformTag: function(tagData){
        // example of basic custom validation
		if( tagData.value != 'whatever' )
			tagData.value = '' // an empty value will force the tag to be "invalid"
    }
}) 

To learn move about how tagify validates tags, see validateTag method

Update (Nov 2020) -

The above method (tagData.value = '') is discouraged and will not work. Instead set it to tagData.__isValid = false) or use the validate function setting which is dedicated to complex scenarios where a Regex pattern will not suffice.

Thank you for quickly answer!

Actually something i want like this:
User copy ids from excel file, then paste to tag, default it will display value with loading icon beside each tags.
Then a debounce on add event will run a function which: load all ids from server then replace loading tags with actual value (replace id to username+avatar), also marked invalid ids then remove it like what keepInvalidTags does

transformTag look like what i need, but if user copy 500 ids, then it will trigger 500 ajax requests?

You are the one who's making the 500 requests, and you can choose to do 1 request for all those id's, if you listen to the add event and add the tag just added to a queue which you'll empty very quickly, like so:

tagify.on('add', onAddTag)

const waitForAllTags = ((allTags, timeout) => { 
	const validate = () => {
		// call ajax once and send "allTags"
        console.log(allTags)
        // reset "allTags"
    	allTags.length = 0
	}

	return data => {
		clearTimeout(timeout)
		timeout = setTimeout(validate, 200)
		allTags.push(data)
	}
})([]) 

function onAddTag(e){
    waitForAllTags(e.detail.data)
}

Yes, i'm doing like that, but i'm getting issue with replaceTag on ajax response.

when got response from ajax for 500 ids, example we have 400 valid id and 100 invalid

$.ajax(...).done(function (users) {
        tagify.value.each(function (tag) {
            if (!tag.uid) {// process value which missing id only
                if (user = users[v.value]) {// found user, replace tags with username
                    tagify.replaceTag(tagify.getTagElmByValue(tag.value),
                        {value: user.display_name + '(' + user.user_login + ')', uid: user.ID}
                    );
                } else {
                    tagify.removeTag(tag.value);//In valid user, remove/remove with invalid effect tried with replaceTag(...,{__isInvalid:true}) as well but seem not working
                }
            }

        })
    });

Both, replaceTag/removeTag not working as expected

I see the problem. well, it's also not a good idea at all to call replaceTag hundreds of times, it does DOM manipulations, which aren't batched, which means it might cause a slowdown.

Suggestion

What you could do instead is to remove all the tags entirely (removeAllTags()) and re-add them, after your AJAX call is successful.

If you already have 100 "good" tags, which you've done the above for, and now the user adds 100 more, which needs to be transformed also, then you can manually save the state and concatenate to it the newly transformed tags, and again remove all and re-add

calling replaceTag repeatedly

There's is a currently problem with calling replaceTag multiple times, there are over a hundred scenarios which can happen in this plugin, and one of them requires a trick to not allow calling replaceTag consequently.

I will try to see if I can come up with another way, to lift the "ban" and allow calling it repeatedly.