I'm not sure i understand how you do it, the files from the playlist are not imported yet in rdj? If so, the program will first import them and when playing it will read it again, this indeed can cause some delay, but i'm baffled about the 2 minutes delay. It's a lot.
If you have the chance the import the files first, you can disable the high precission cue point from the track editor. Then whenever you load the track, even when using playlists, the settings will apply.