Bug Problem while training an agent with sac and demonstration.

Hi,

I've been trying to train an agent using sac+behavioral cloning. After a while I've noticed that the environment started freezing so I've ran mlagents-learn using --debug. The training process starts normally but, after a certain amount of time, the environment gets stuck and the debugger informs that the sac policy is being updated at the same step over and over.


Configuration file:

behaviors:

  ShipTraining:

    trainer_type:   sac

    hyperparameters:    

      learning_rate:    0.0003

      learning_rate_schedule:   constant

      batch_size:   1024

      buffer_size:  1000000

      buffer_init_steps: 20000

      tau:  0.005

      steps_per_update: 8

      save_replay_buffer: false

      init_entcoef: 0.9

      reward_signal_steps_per_update: 8

    network_settings:  

      normalize: true

      hidden_units: 256

      num_layers:   3

      vis_encode_type: simple

      goal_conditioning_type:   none

    reward_signals:

      extrinsic:    

        gamma:  0.99

        strength:   1.0

        network_settings:  

          normalize: false

          hidden_units: 128

          num_layers: 2

          vis_encode_type: simple

          goal_conditioning_type: hyper

    behavioral_cloning:

      strength: 0.4

      demo_path: Demos\AimbotRecMovement.demo

      steps: 30000

    keep_checkpoints: 5

    max_steps:  2000000

    time_horizon: 1024

    summary_freq: 10000

    threaded: false

engine_settings:

  width: 1800

  height: 1600

  quality_level: 2

  time_scale: 10

  target_frame_rate: -1

  capture_frame_rate: 60

  no_graphics: false

I've also attached a screenshot of the debugger output.
Useful informations:
mlagents version: 0.26.0
mlagents-env version: 0.26.0
unity version: 2021.3.12f1
unity package: 2.0.1

Is there a workaround to this problem?

Thank you in advance.