I just did another test with a different vc backend: the Mercurial backend.
I created a local Mercurial repo and submitted a file that is not completely rendered by vc-annotate with the SVN backend.
I modified the Mercurial hosted file, committing my changes in the Mercurial repo. Then I used vc-annotate on that Mercurial hosted file and every line was annotated as it should be.
So it does seem to be specific to the SVN backend.
--
/Pierre