This is just not provided by the EGit/JGit plugin.
Where other plugins/tools delegate their Git commands to the system, JGit has implemented it internally and doesn't support (OAuth) Token based authentication. There is a topic open here: https://github.com/eclipse-jgit/jgit/issues/94
So until that plugin is update, a workaround is required.
I did need to configure the system git to use OAuth for azure devops by setting the following properties (these set the Azure authentication to oauth and delegate the request flow to the system browser, make sure that one is not IE anymore)
git config --system credential.azreposCredentialType oauth
git config --system credential.msauthFlow system
Apart from that, I have created Eclipse launch files so we can still fetch, pull and push from within the Toolkit.
Just dropping the information here, should anyone ever run into the same issue.