We do not currently know of any antonyms for have influence with.
For similar words to have influence with, try: