Always a fan of your songs and vocal, and the core ideas in this one certainly don’t disappoint! 
Structurally speaking I’m not convinced… you know me!
It’s quite a long wait to get to the meat of the song and meanders too much I reckon. And I think in this day and age of instant gratification it would work better - certainly dynamically, to have those earlier choruses “I can Never Let You Go” much bigger, i.e. full arrangement, then collapse back down into a more open stripped-back verse. Generally I think you could edit this down to under 4 minutes, create more highs and lows, at the moment it feels a bit flat and too “bitsy” - and the verse/chorus/bridge etc are not so well-defined. I’d lose the interlude at 1:54 - 2:17.
Fundamentally It’s a really good song that for me currently has a lot of untapped potential. I really think there’s a more optimal structure to be had, one with a more commercial pop-song sensibility. Now I realise that may not be what you want but for me, this like many of your tunes has commercial pop written all over it albeit a somewhat 80’s vibe, which of course largely comes down to arrangement/production choices etc.
In fact you’re a very good writer, you regularly come up with some real catchy, hooky lines which I often envy, and a voice that is pleasing to the ear. It’s another one of those songs that when I hear it I think 'I’d love to get my hands on this!"
Anyway, as always, take or leave anything I say! Still a fan… and if you ever want to chance me having another crack at “Sherzifying” one of your tunes, just let me know. 