I cannot read from serial port until I login

I have a python script that reads from serial:
The serial port is receiving data from an external device and everything works fine when I manually run it via SSH. However, when I try to make this script run at boot with systemd or rc.local, it first reads a bunch of 0 bytes and then nothing. I tried to initialize the serial port with xonxoff=False and dtr=False but the result is the same. This script runs on a Linux build with serial console disabled. What could be the cause of this problem?