Hi Colton, what you're describing is not the bug in eshell but rather a bash feature; no other shell will perform expansion on literal tilde in PATH (you can disable it in Bash using POSIXLY_CORRECT environment variable). If you drop the quotes then it's going to work as expected because the shell will expand the tilde to HOME before storing it in PATH:
PATH=~/test-bin:$PATH emacs -Q
Best,
Jakub Ječmínek