The general 'recursive routing' problem in IP networking
There's a general problem in certain sorts of IP network setups that I will call the recursive routing problem. The recursive routing problem comes up when you want to send some inner traffic to an IP address in protected form, by taking the traffic, encrypting it, and then sending it to the same IP address as outer traffic. You need this outer traffic to be treated differently from the inner traffic so it's not recursively re-routed back through the encryption process again, because otherwise your traffic to the IP will just spin around in a little circle and never get anywhere.
The simplest way to break the recursive routing problem is to do the encryption at a level above IP routing. You establish a (TCP) connection, then you have the application level use the connection to arrange an encryption layer and transport your data over that. This is how a huge amount of encrypted data crosses the Internet every day, in the form of HTTPS (and some SSH and other application level protocols). This has the problem that it's not a particularly general solution, and so you wind up with people working out all sorts of ways to tunnel traffic over HTTPS partly because they don't have an accessible, usable general purpose encrypted transport layer they can count on.
One of the things that IPsec was supposed to be used for was encrypting such direct IP to IP traffic. IPsec solved the resulting recursive routing problem with, basically, magic. Systems knew what IPsec was and knew that once traffic had gone through IPsec, it was fired out onto the network without looping back through again. You could write an IPsec policy that said 'everything to host X must be IPsec'd', and your system would make this work so that IPsec traffic didn't actually count as 'everything' (including, I believe, IPsec setup and negotiations). IPsec got this support partly because it was 'part of IP(v6)', and so was obviously worthy of special and more or less standardized treatment (and, for example, its own IP protocol along side things like TCP, UDP, and GRE). Other encrypted protocols were not so lucky.
(You could restrict IPsec's scope so it only applied to certain traffic, but I don't think you had to.)
You can solve a related version of this, what I call the VPN recursive routine problem, by using a highly specific IP route as well as a more general one. You can say '128.100.X.0/24 is routed through the VPN tunnel, but 128.100.X.VPN/32 is routed through <my default gateway>'. Your traffic to everything on 128.100.X.0/24 except the VPN itself will be shoved through the VPN tunnel, but the more specific route for the VPN's IP diverts its traffic away from the tunnel. When you don't actually talk to the VPN server other than for the VPN itself, this is fine, but it's potentially less fine if you'd like a mesh-capable VPN solution where you can make individual hosts VPN endpoints.
(One hack around this in a mesh-capable VPN environment is to not use the same IP. VPN endpoints have an inside IP as well as an outside IP, and all of their services are accessed using the inside IP. But this gives you other issues.)
Otherwise you are looking at operating system specific hacks. Linux has a 'fwmark' system that you can use to tag specific packets and route them specially, using Linux's general system for policy based routing. I believe that Windows, macOS, and iOS all have their own approaches. Being Linux based, Android may use fwmark and other things under the hood, or it may have a different approach.