My situation is a little different than most here, and as such the answer is an easy "yes" -
Every move I've made has involved leaving projects in the middle, the end, and the beginning. In my world I manage multiple projects (20+) on a rolling basis so there is never a point when I'm 'done' because there are always others in various stages.
As others have said, it's job, nothing more. I would have no more problem leaving than when one of my team tells me they're leaving. I always wish them the best, ask about the new opportunity, and then download as much as possible to their replacement. Same for me. I give appropriate notice, make sure all of my responsibilities are up to date and handled, and work as much as I can with my replacement before I leave to get them up to speed. Then it's "here's my number, call me if you have any questions", and we part ways.
I always have a vested interest in the projects success, even after I'm gone. I still often drive past many of the projects I built or worked on and stop and look. But ultimately the project will end and belong to someone else. And while I would like to be able to successfully finish every project I have ever touched, I would be foolish to pass on an opportunity out of a (to me) faulty sense of loyalty.
As long as I've done my job correctly, am leaving the project in good shape, and make a smooth transition to the new PM, then I have no qualms about leaving.