yeah if you just press delete the verts will be left behind. You have to use edit mesh>delete edge/vertex. You can also hold down the shift-RMB and a marking menu will pop up with the delete edge option so you don't have to go to the menu tabs.
"If I have seen further it is by standing on the shoulders of giants." Sir Isaac Newton, 1675