Thanks for the report, and no, this isn't expected behaviour. We've had a number of these bugs that introduce non-deterministic behaviour, which is definitely undesirable. GPU has been particularly affected by this but sometimes it's crept into the CPU models as well.
As you point out it's an issue in spaCy rather than Prodigy. If you like you could make an issue on the tracker and link this thread there. I'm also happy to make the thread instead if you don't have a Github account or would rather not take the time.
I think spaCy v2.3 has some fixes for non-determinism that might address this case. But if not, we definitely want to track it down.
In spaCy v3 we're adopting an integration with Data Version Control, which checksums trained assets. This will make non-determinism much more prominent in our workflows, which should allow us to address this problem.