Skip to content

Framework/bitbots rl motion#788

Merged
jaagut merged 111 commits into
mainfrom
framework/bitbots_rl_motion
May 11, 2026
Merged

Framework/bitbots rl motion#788
jaagut merged 111 commits into
mainfrom
framework/bitbots_rl_motion

Conversation

@sWintermoor
Copy link
Copy Markdown
Collaborator

@sWintermoor sWintermoor commented Apr 1, 2026

Summary

This change provides a framework for rl policies. Main purpose is to minimize boiler plate code and facilitate the integration of rl models.

Proposed changes

  • New rl_motion component binds rl running logic into a parent class.
  • So-called handlers bind subscription and associated processing logic .
  • Policy nodes are reduced to customizable parts such as observation, publisher and 'allowed states' logic. Execution logic is inherited from rl parent class.
  • Integrate walking and kicking/dribbling policy config files (wolfgang) as examples.
  • Create a new behavior action called RLKick. RLKick sends via kick capsule control messages to kick policy. RLKick decides whether kick policy is allowed to publish messages.
  • Replace WalkKick with RLKick in Behavior dsd.
  • Quintic walk won't be executed by default.

Related issues

This component has to be tested on a robot.

Checklist

  • Run pixi run build
  • Write documentation
  • Test on your machine
  • Test on the robot
  • create issues for future work
  • Triage this PR and label it

@sWintermoor
Copy link
Copy Markdown
Collaborator Author

I removed test.launch and rl_walk.launch in bitbots_rl_motion, they don't fit anymore. Maybe the new launch file rl_motion.launch has to be adapted.

@sWintermoor sWintermoor self-assigned this Apr 26, 2026
@sWintermoor sWintermoor moved this from 👀 In review to 🏗 In progress in Software Apr 26, 2026
@sWintermoor sWintermoor moved this from 🏗 In progress to 👀 In review in Software Apr 26, 2026
@jaagut
Copy link
Copy Markdown
Member

jaagut commented Apr 29, 2026

How do we want to progress with this? This is currently targeting main and Wolfgang. Can we reuse this architecture for Pi+? Or do we already do that?

@sWintermoor You keep using Wolfgang for your thesis, right?

@sWintermoor
Copy link
Copy Markdown
Collaborator Author

This architecture is reusable for Pi+. We would just add a config file.

No, I don't use any Wolfgang.

self.blackboard.kick.start_rl_kick()

elapsed = self.blackboard.node.get_clock().now() - self._start_time
if elapsed >= Duration(seconds=self._duration):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How / why is it guaranteed that after this duration, the kick is actually finished?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no guarantee, I just assumed it. I don't have an ad-hoc solution for measuring whether a kick is completed

Comment thread src/bitbots_misc/bitbots_bringup/launch/motion.launch Outdated
type: double
description: "End pose trunk height. Depends on walkready of walking."
default_value: 0.38
default_value: 0.372
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you change these?

Comment thread src/bitbots_motion/bitbots_rl_motion/package.xml Outdated
Comment thread src/bitbots_motion/bitbots_rl_motion/setup.py Outdated
Comment thread src/bitbots_motion/bitbots_rl_motion/nodes/rl_node.py Outdated
Comment thread src/bitbots_motion/bitbots_rl_motion/nodes/rl_node.py
Comment thread src/bitbots_motion/bitbots_rl_motion/nodes/rl_node.py Outdated
Comment thread src/bitbots_motion/bitbots_rl_motion/nodes/rl_node.py Outdated
Comment thread src/bitbots_motion/bitbots_rl_motion/handlers/joint_handler.py Outdated
@sWintermoor sWintermoor force-pushed the framework/bitbots_rl_motion branch from 8381a7d to 51ce0ee Compare May 3, 2026 14:38
@jaagut jaagut merged commit 0df9442 into main May 11, 2026
2 checks passed
@jaagut jaagut deleted the framework/bitbots_rl_motion branch May 11, 2026 12:21
@github-project-automation github-project-automation Bot moved this from 👀 In review to ✅ Done in Software May 11, 2026
@Flova
Copy link
Copy Markdown
Member

Flova commented May 11, 2026

Are we sure this is compatible with the piplus right now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

3 participants