Hi, did you manage to solve this problem? I am facing similar issue while training on my dataset. If the conversation is long (more number of steps), the model starts outputting special symbols.