feat: initial ROS2 AsyncAPI contribution by SIEMENS AG #270
Conversation
…ort and is corporate internal legally cleared for OSS contribution.
gramss
requested review from
derberg,
fmvilas,
dalelane,
smoya,
char0n,
asyncapi-bot-eve and
GreenRover
as code owners
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Welcome to AsyncAPI. Thanks a lot for creating your first pull request. Please check out our contributors guide useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.
gramss
changed the title
gramss
changed the title
ros2/README.md
Outdated
|
|
||
| ###### Fixed Fields | ||
|
|
||
| Field Name | Type | ROS 2 Versions | Description |
There was a problem hiding this comment.
I see you're including a "ROS 2 Versions" column but this column always has the "all" value. Should we get rid of it or do you expect future properties to appear only for certain versions of ROS 2?
There was a problem hiding this comment.
currently this list is not comprehensive. This could be much larger but a better place to look for changes is here: e.g. for Jazzy: official ROS doc for release changelog
For an initial PR I would leave it that way and depending of the acceptance and usage of AsyncAPI in ROS this can be extended more and bring back the ROS 2 Version column that way.
Leaving this open to feedback from others for now, but the change is reflected in the latest commits
ros2/README.md
Outdated
| ## Channel Binding Object | ||
|
|
||
| A channel represents a ROS 2 topic, service or action interface. | ||
|
|
||
| This object DOES NOT contain any ROS 2 specific properties. | ||
|
|
||
| Example - ROS 2 topic: | ||
|
|
||
| ```yaml | ||
| address: /turtle1/cmd_vel | ||
| messages: | ||
| TwistMsg: | ||
| $ref: '#/components/messages/TwistMsg' | ||
| ``` | ||
| Example - ROS 2 action: | ||
|
|
||
| ```yaml | ||
| RotateAbsoluteRequest: | ||
| address: /turtle1/rotate_absolute | ||
| messages: | ||
| RotateAbsoluteActionRequest: | ||
| $ref: '#/components/messages/RotateAbsoluteActionRequest' | ||
|
|
||
| RotateAbsoluteReply: | ||
| address: /turtle1/rotate_absolute | ||
| messages: | ||
| RotateAbsoluteActionResult: | ||
| $ref: '#/components/messages/RotateAbsoluteActionResult' | ||
| RotateAbsoluteActionFeedback: | ||
| $ref: '#/components/messages/RotateAbsoluteActionFeedback' | ||
| ``` |
There was a problem hiding this comment.
If there's no channel binding, I recommend leaving it out of the document. E.g., the way we're doing with the AMQP 1.0 channel binding: https://github.com/asyncapi/bindings/tree/master/amqp1#channel-binding-object.
There was a problem hiding this comment.
hm.. Just had time to dig a little bit deeper into channels.
Would you describe this picture here as a channel?
Source from OSRF
Or another example here (circle = node == application and box = topic == message)
The message /joint_states is consumed by 2 nodes..
For me, topic translates to an asyncAPI message and a ros node translates to an asyncAPI application.
There can be one-to-many communications in ROS between one node (pub) and many other nodes (subs). Same applies for the other ros2 roles.
But I am unsure how this can be represented as a channel here. The specifics how the message is transported to many clients is defined inside of the concrete rmw implementation(?). A node can only tune how to react to a new event but not influence the transport mode other than specifying the QoS level.
Does @Achllle maybe has an idea about that? Here is the asyncAPI definition: link
I am having a hard time especially with translating this sentence from the doc to ROS:
The producer sends a message through the channel, which then queues the message for delivery to the appropriate consumers.
There was a problem hiding this comment.
mmm 🤔 from what I'm reading here, it seems that AsyncAPI channels map to ROS2 topics and AsyncAPI messages map to ROS2 messages (or data types, data units). So, from what I'm learning so far:
- ROS2 node -> AsyncAPI application (i.e., the whole AsyncAPI file)
- ROS2 middleware (DDS, Zenoh) -> AsyncAPI server.
- ROS2 topic -> AsyncAPI channel
- ROS2 message/data type -> AsyncAPI message
Feel free to correct me as I'm looking at this for the first time.
There was a problem hiding this comment.
From the docs here:
self.subscription = self.create_subscription(
String,
'topic', # <-- This is the AsyncAPI's channel address
self.listener_callback,
10)
self.subscription # prevent unused variable warning
def listener_callback(self, msg): # <-- `msg` is the AsyncAPI's message
self.get_logger().info('I heard: "%s"' % msg.data) # <-- msg.data is the AsyncAPI's message payloadAnd I'm assuming, in listener_callback, the msg variable will contain a msg.header which maps directly to AsyncAPI's message headers.
There was a problem hiding this comment.
@gavanderhoorn thank you for the proper fact checking and updating (at least) me on the current state of some ROS developments. 👍
We can bring the topic about a ROS2 release-versioned database of common-msgs in the mentioned discourse discussion.
I am open to discuss this topic. The binding should include a guideline how to deal with common messages regardless of the outcome of this discussion.
There was a problem hiding this comment.
A specific (recent) distribution of ROS 2 + the package name + interface name is the only thing needed to fully define a message. It feels somewhat silly to redefine such messages in the asyncAPI. Without the message headers or import it's not possible (AFAIK) to create the payload like one would do with a json blob. To me the ideal experience would involve linking to the interface definition rather than redefining it. When someone intends to update to a newer distribution, they would have to manually validate all AsyncAPI messages to match the new distribution. Most ROS applications heavily depend on those interface definitions included in ros-base or other common packages, so this would mean a lot of duplication and room for user error.
I don't really see the point with 'black box code'. Such repos will have at least their interface descriptions made available, so whether the code is accessible doesn't really matter.
Perhaps this 'linking' is possible through a binding where if it's a standard message, you'd link to it with:
$ref: https://docs.ros.org/en/$ROS_DISTRO/p/geometry_msgs/msg/Twist.html
# OR
$ref: https://github.com/ros2/common_interfaces/blob/$ROS_DISTRO/geometry_msgs/msg/Twist.msg
# OR
$ref: my_custom_interfaces/msg/MyMsg.msgand we include some magic to render the definitions when they're found.
Looking at NoDL, it seems that this approach is happening there as well.
There was a problem hiding this comment.
@Achllle First, thank you for your thoughtful feedback on this discussion. I appreciate your expertise in ROS2 and would like to address some of the points you raised.
Regarding References in AsyncAPI
I understand your suggestion about using direct URL references to ROS2 message definitions. While this approach seems intuitive from a ROS2 perspective, the AsyncAPI specification has a specific way of defining message payloads, which relies on the $ref keyword to reference reusable schema definitions. From my understanding, it's not possible to use a direct URL in this context
The $ref syntax you proposed would require custom extensions or resolvers that aren't part of the standard AsyncAPI functionality. This could create compatibility issues with existing AsyncAPI tools and validators.
On Message Definition Duplication
You make an excellent point that a ROS2 distribution + package name + interface name is sufficient to fully define a message. What might seem like redundant redefinition actually serves an important purpose in our approach.
By including complete message definitions within the AsyncAPI document, we enable:
- Automated Code Generation: Tools can generate client/server code, blocks for low code tools as behavior tree or node red, directly from the AsyncAPI document if it contains all the necessary information.
- Proper HTML documentation: Having complete message definitions embedded in the AsyncAPI document ensures that the generated documentation is self-contained and fully informative, allowing even those unfamiliar with ROS2 to understand the API structure and message formats without needing to reference external resources.
Our Approach: Automation Over Manual Definition
I believe we might be aligned on the core issue but approaching it differently. Our goal isn't to have developers manually redefine ROS2 messages in AsyncAPI (which would indeed be tedious and error-prone). Instead, we envision automated tools that:
- Extract message definitions from ROS2 packages
- Convert them to AsyncAPI format
- Generate the complete AsyncAPI document with all necessary definitions included
This way, developers don't need to manually maintain these definitions - the tooling handles the synchronization between ROS2 interfaces and AsyncAPI documents
There was a problem hiding this comment.
@fmvilas can you please comment on
would require custom extensions or resolvers that aren't part of the standard AsyncAPI functionality.
specifically whether this would be achievable today through e.g. bindings? Is there any precedent for this as well, such as including existing json/yaml schemas that are defined outside AsyncAPI? Are those re-defined within the AsyncAPI spec?
I understand it may be harder to do this, but it's hard to argue that it's also not superior. Automated generation doesn't solve the issue of duplicating information, thus leading to potential conflicts, especially since messages can vary between ROS 2 distros.
generated documentation is self-contained and fully informative, allowing even those unfamiliar with ROS2 to understand the API structure and message formats without needing to reference external resources.
Documentation generation can take care of linking or even embedding the message definitions so that would address self-containment. That would also solves the issue for those unfamiliar with ROS 2. I'm also not sure why someone unfamiliar with ROS 2 would want to deeply interact with a ROS 2 API without first learning ROS 2 basics.
That being said, if it proves unattainable or would require updating the spec in a major way, I'm fine with the duplication approach, though I might consider using NoDL in that case.
There was a problem hiding this comment.
agree on getting @fmvilas opinion on this.
I think we cannot alter it from this binding-spec but rather would need to make contributions to the larger asyncAPI body.
@Achllle we should also look at the problem from another angle. ROS already does a great job at defining its messages through a standard approach on the code level with the dedicated .msg / .srv and .action files. Are other supported protocols also doing that way? I was not able to gather such information about MQTT for instance.
Regarding your concerns:
issue of duplicating information, thus leading to potential conflicts, especially since messages can vary between ROS 2 distros
Can we dig deeper into this?
Are you concerned about the duplication itself or more about the missing reference to the original origin?
The datatypes need to be translated anyway from ROS into AsyncAPI at some point. Either at a rendering stage of HTML or before by holding the information in files.
We could influence this from the ROS UX so that those files are only generated on-demand and deleted afterwards.
If your only concern might be regarding different ros distros, this information is also available at the head asyncAPI file. But this is not pinned on the message level for mix-bistro-robots (that I would like to neglect if possible at a first stage?).
I'm also not sure why someone unfamiliar with ROS 2 would want to deeply interact with a ROS 2 API without first learning ROS 2 basics.
Reading a documentation of a robot and finding information (where and if e.g. a camera raw stream is made available) is a step before going into the details of a specific application. Could be that a robot offers such information only as a websocket, then one does not need to go deeper into the ROS universe.
Generally, I would love to bring this discussion on a use-case based requirement level. I totally respect your view on this subject but would love to bring it into use-case scenarios that we can either accept or reject (non-relevant) and with it the proposing requirements.
This is something we could focus on the meeting next week.
ros2/README.md
Outdated
| AsyncAPI operations with their `send` and `receive` actions map directly to ROS 2 subscribers, publishers, actions or services. | ||
| - send -> `publisher`, `action_client`, `service_client` | ||
| - receive -> `subscriber`, `action_server`, `service_server` |
There was a problem hiding this comment.
I'm not sure I understand this. Could we improve this description?
There was a problem hiding this comment.
It will be put in the description field of the table
There was a problem hiding this comment.
Not sure if I'm nitpicking, but technically speaking:
| AsyncAPI operations with their `send` and `receive` actions map directly to ROS 2 subscribers, publishers, actions or services. | |
| - send -> `publisher`, `action_client`, `service_client` | |
| - receive -> `subscriber`, `action_server`, `service_server` | |
| AsyncAPI operations with their `send` and `receive` actions map directly to ROS 2 subscribers, publishers, actions or services. | |
| - producer -> `publisher`, `action_client`, `service_client` | |
| - consumer -> `subscriber`, `action_server`, `service_server` |
send and receive refers to publisher.publish(), receive would point to a callback()/register_callback()
There was a problem hiding this comment.
The reason why we used send/receive is because those are the only options for the field action in AsyncAPI and we were tying to map it to the different ROS2 types, since we cannot change the valid options for the field action
ros2/README.md
Outdated
|
|
||
| Field Name | Type | ROS 2 Versions | Description | ||
| ---|:---:|:---:|---| | ||
| `type` | string | all | Specifies the ROS 2 type of the node for this operation. Valid values are: `publisher`, `subscriber`, `service_client`, `service_server`, `action_client`, `action_server`. This defines how the node will interact with the associated topic or action. |
There was a problem hiding this comment.
So, if I understand correctly, when the action field is send, this value can be publisher, action_client, or service_client. And when action is receive, this value can be subscriber, action_server, or service_server. Is that right? If so, this relationship should be expressed here instead of above.
There was a problem hiding this comment.
Yes, we will put the explanation here
ros2/README.md
Outdated
| $ref: "#/channels/CmdVel" | ||
| bindings: | ||
| ros2: | ||
| role: subscriber |
There was a problem hiding this comment.
It would be great to make this value the default so people don't have to specify it.
There was a problem hiding this comment.
In my opinion does not make sense since in ROS2 there is no default value for roles. It depends if you are using topics, actions, services... and it depends if you are publishing, subscribing... But we are completely open to suggestions. @Achllle what is your opinion here?
There was a problem hiding this comment.
Yeah, not a strong opinion on this.
There was a problem hiding this comment.
I'm not sure if I understand role. I can't find it in the adding bindings doc.
There was a problem hiding this comment.
The idea is to have a ros2 binding named as "role" inside the operation bindings to define if a node is publisher/subscriber or service/action client/server. Right now you don't see it in the binding documentation because it does not exist yet. What we are trying here is to define the necessary bindings to put them as official bindings (in the same way that you see right know the mqtt ones) and then you will be able to see them. Right know all ros2 bindings that don't use the existing ones, are experimental.
Summarizing, in bindings we can get "anything" we want.
| format: double | ||
| ``` | ||
|
|
||
| ROS 2 Type | AsyncAPI Type | AsyncAPI Format | |
There was a problem hiding this comment.
| ROS 2 Type | AsyncAPI Type | AsyncAPI Format | | |
| ROS 2 Type | AsyncAPI Type | AsyncAPI Format | |
There was a problem hiding this comment.
What's this table about? When should it be used and what for?
There was a problem hiding this comment.
I agree that we should delete the table from here since it is not about the bindings but we should have this table somewhere. We think it is important that ros developers have this table to know how to express the ros2 messages types in asyncAPI format.
@Achllle do you think that we should put this information in the ros2 documentation?
There was a problem hiding this comment.
Sure, I'm all in with having it, I just think we have to explain what is for. Right now it's just there without any further explanation.
There was a problem hiding this comment.
I do believe this belongs here, agreed with adding explanation. Suggest including a link to the design document
There was a problem hiding this comment.
We included the design document suggested by @Achllle .
But @fmvilas question is not answered:
explain what [this table] is for
I think such an information could be needed to properly integrate/visualize different bindings in one tool. That is why you have your AsyncAPI message types and formats for.
Right now, this is used as our Rosetta Stone table for the siemens PoC AsyncAPI generator reading ros message file definitions and producing respecting AsyncAPI files.
I hope that the current wording reflects this already. If not, we can make it more distinctive.
@Achllle while we think that could be really interesting to include the parameters, we don't think that it is a hard requirement to have them to be able to have the first version of the ROS2 bindings. |
|
Considering the ROS 2 version of this effort, node_idl / NoDL includes them in their spec, I would argue this is a must-have for a complete spec. It's fine by me if it happens in two phases, but the point of a spec is that it's complete. |
yes. I agree that parameters need to be added to the ros2 bindings. In our use cases, we did not use them so far. That is why they slipped our implementation. I would like to add them in a second step. From what I've gathered, parameters are interfaced through fixed services that all nodes have, unless pro-actively deactivated. What can be transported is also fixed in terms of data / message types. So like you put it, it might only be necessary to add to a node==application what parameters it has and probably not list all the additional We need to investigate this more. I still am having trouble how it embeds into an asyncAPI binding.. Is somebody in the ros-infrastructure team already familiar with asyncAPI @Achllle ? |
|
The best way to get involvement from the ROS community would be via a post in ROS discourse! |
https://discourse.ros.org/t/invitation-to-collaborate-on-asyncapi-specification-for-ros2/42915 @fmvilas if you have the time, it would be a pleasure to have you join such a meeting. |
| ## Server Binding Object | ||
|
|
||
| This object contains information about the server representation in ROS 2. | ||
| ROS 2 can use either DDS or Zenoh as its middleware. |
There was a problem hiding this comment.
Minor perhaps, but while the main RMWs in use may be DDS-based, there are certainly others that aren't (rmw_zenoh being a recent addition, but also eclipse-ecal/rmw_ecal fi).
There was a problem hiding this comment.
Thank you for your comment!
As you know, there are a quite acouple of them with my favourite (for the sake of this exact argument) the christophebedard/rmw_email.
We also had this argument earlier with @fmvilas , that for the moment we only want to focus on DDS and Zenoh based systems. But I think that the majority of the RMW implementations should also be representable through this binding. But this current implication should be put out exactly in this sentence. Therefor, thank you for highlighting it!
Additionally, we are already indicating the vast options one can choose from with ROS2 here:
Line 27 in 463fb3d
1. Include in the server binding links and change to none instead of localhost. 2. Include the link to the non-negative less than 232. 3. Explain better the message binding with links. 4. Put at the end a whole example of a ROS application with its explanation.
|
|
||
| ## Message Binding Object | ||
|
|
||
| While this object DOES NOT contain any ROS 2 specific properties, it is important to understand how to express the different [ROS2 data types](https://design.ros2.org/articles/legacy_interface_definition.html#:~:text=to%20IDL%20types-,ROS%20type,string,-The%20mapping%20of) in [AsyncAPI message types and formats](https://www.asyncapi.com/docs/reference/specification/v3.0.0#dataTypeFormat:~:text=The%20formats%20defined%20by%20the%20AsyncAPI%20Specification%20are%3A). |
There was a problem hiding this comment.
| While this object DOES NOT contain any ROS 2 specific properties, it is important to understand how to express the different [ROS2 data types](https://design.ros2.org/articles/legacy_interface_definition.html#:~:text=to%20IDL%20types-,ROS%20type,string,-The%20mapping%20of) in [AsyncAPI message types and formats](https://www.asyncapi.com/docs/reference/specification/v3.0.0#dataTypeFormat:~:text=The%20formats%20defined%20by%20the%20AsyncAPI%20Specification%20are%3A). | |
| While this object DOES NOT contain any ROS 2 specific properties, it is important to understand how to express the different [ROS 2 data types](https://design.ros2.org/articles/legacy_interface_definition.html#:~:text=to%20IDL%20types-,ROS%20type,string,-The%20mapping%20of) in [AsyncAPI message types and formats](https://www.asyncapi.com/docs/reference/specification/v3.0.0#dataTypeFormat:~:text=The%20formats%20defined%20by%20the%20AsyncAPI%20Specification%20are%3A). |
| It is important to understand that the message header of the AsyncAPI specification does not map with the message header in ROS2. | ||
| For this reason, the AsyncAPI message header will be ignored. If you want to define the header of a message in ROS2, you should put in the payload of the message. [Example](https://docs.ros.org/en/ros2_packages/rolling/api/point_cloud_interfaces/msg/CompressedPointCloud2.html): |
There was a problem hiding this comment.
| It is important to understand that the message header of the AsyncAPI specification does not map with the message header in ROS2. | |
| For this reason, the AsyncAPI message header will be ignored. If you want to define the header of a message in ROS2, you should put in the payload of the message. [Example](https://docs.ros.org/en/ros2_packages/rolling/api/point_cloud_interfaces/msg/CompressedPointCloud2.html): | |
| It is important to understand that the message header of the AsyncAPI specification does not map with the message header in ROS 2. | |
| For this reason, the AsyncAPI message header will be ignored. If you want to define the header of a message in ROS 2, you should put in the payload of the message. [Example](https://docs.ros.org/en/ros2_packages/rolling/api/point_cloud_interfaces/msg/CompressedPointCloud2.html): |
|
|
||
| ## Complete example: | ||
| From the AsyncAPI specification example it could be extracted that: | ||
| - It consist on a ROS2 jazzy application running with Fast DDS as RMW. |
There was a problem hiding this comment.
| - It consist on a ROS2 jazzy application running with Fast DDS as RMW. | |
| - It consist on a ROS 2 Jazzy application running with Fast DDS as RMW. |
| - It consist on a ROS2 jazzy application running with Fast DDS as RMW. | ||
| - `/turtlesim` node is a subscriber of the `/turtle1/cmd_vel` topic and its qos policies. | ||
| - The interface of the `/turtle1/cmd_vel` topic is `Twist` that has a nested type: `Vector3`. Both of them are part of the standard package `geometry_msgs`. | ||
| - `Vector3` has already the types converted to AsyncAPI types and formats (number and double), instead of using the ROS2 types. |
There was a problem hiding this comment.
| - `Vector3` has already the types converted to AsyncAPI types and formats (number and double), instead of using the ROS2 types. | |
| - `Vector3` has already the types converted to AsyncAPI types and formats (number and double), instead of using the ROS 2 types. |
Co-authored-by: Christophe Bedard <[email protected]>
Description
Maintaining will be done through @amparo-siemens and me.
Further maintainers and improvements suggested in #254 are highly welcome.
This binding is tested already in a few internal tools and generators as seen in this presentation: ROSCon DE
cc: @renzo-sie (thank you for the preparations!) and @fmvilas