Skip to content

Fine tuning with RL #105

Closed
Closed

Description

@terryfrankcombe

Hi folks

Is there a recommended way to load a pretrained model for RL without loading the entire checkpoint?

I'm training with TD3 and I want to refine the model with new data and a modified reward function. If I call torchfort_rl_off_policy_create_system("td3_system",... then torchfort_load_model("td3_system",... with the filename for a saved checkpoint policy/model.pt I get an Invalid usage. (Invalid model name provided.) error.

I guess one approach would be to load the entire checkpoint then try to kill the history, but I don't know how to do that either.

How do I set up for fine tuning?

Ciao
Terry

Activity

  1. azrael417 commented on Jun 3, 2026

    @azrael417
    Collaborator

    Hello Terry,

    that is indeed a missing functionality. I implemented that into this branch:
    #107

    can you have a look and evaluate and let me know what you think?

    Best Regards
    Thorsten

  2. terryfrankcombe commented on Jun 4, 2026

    @terryfrankcombe
    Author

    Wow, that was fast! I will take me a couple of days to make time to test.

  3. terryfrankcombe commented on Jun 17, 2026

    @terryfrankcombe
    Author

    Looks to work just fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions