There is clearly not enough room for a first time layer to manipulate under the door. so take it off

If you take a door off put the screw in some Masking tape and stick them over the top of the door. This could save you time trying to find them. Also save the embarrasment of finding one underneath the vinyl you have otherwise successfuly installed