Should I just quit my job (No Hours)?

Yeah, your job is obviously not working for you. You should find another job before you quit this one though. Five hours is better than zero hours until you find another job.