6.4. Shared Memory Transport¶

The shared memory (SHM) transport enables fast communications between entities running in the same processing unit/machine, relying on the shared memory mechanisms provided by the host operating system.

Note

Fast DDS utilizes the DomainParticipant’s GuidPrefix_t to identify peers running in the same host. Two participants with identical 4 first bytes on the GuidPrefix_t are considered to be running in the same host. is_on_same_host_as() API is provided to check this condition. Please, take also into account the caveats included in GUID Prefix considerations for intra-process delivery.

SHM transport provides better performance than other network transports like UDP / TCP, even when these transports use loopback interface. This is mainly due to the following reasons:

Large message support: Network protocols need to fragment data in order to comply with the specific protocol and network stacks requirements, increasing communication overhead. SHM transport allows the copy of full messages where the only size limit is the machine’s memory capacity.

Reduce the number of memory copies: When sending the same message to different endpoints, SHM transport can directly share the same memory buffer with all the destination endpoints. Other protocols require to perform one copy of the message per endpoint.

Less operating system overhead: Once initial setup is completed, shared memory transfers require much less system calls than the other protocols. Therefore, there is a performance/time consume gain by using SHM.

6.4.1. Definition of Concepts¶

This section describes basic concepts to help explain how the Shared Memory Transport works in order to deliver the data messages to the appropriate DomainParticipant. The purpose is not to be an exhaustive reference of the implementation, but to be a comprehensive explanation of each concept, so that users can configure the transport to their needs.

Many of the descriptions in this section will be made following the example use case depicted in the following figure, where Participant 1 sends a data message to Participant 2. Please, refer to the figure when following the definitions.

../../../_images/shm_comm_sequence_diagram.svg

Sequence diagram for Shared Memory Transport¶

6.4.1.1. Segment¶

A Segment is a block of shared memory that can be accessed from different processes. Every DomainParticipant that has been configured with Shared Memory Transport creates a segment of shared memory. The DomainParticipant writes to this segment any data it needs to deliver to other DomainParticipants, and the remote DomainParticipants are able to read it directly using the shared memory mechanisms.

Note

Launching any of the processes with a higher privileged user (for instance, root) can lead to communication problems, as processes run by non-privileged users may not be able to write into the memory segment.

Every segment has a segmentId, a 16-character UUID that uniquely identifies each shared memory segment. These segmentIds are used to identify and access the segment of each DomainParticipant.

6.4.1.2. Segment Buffer¶

A buffer allocated in the shared memory Segment. It works as a container for a DDS message that is placed in the Segment. In other words, each message that the DomainParticipant writes on the Segment will be placed in a different buffer.

6.4.1.3. Buffer Descriptor¶

It acts as a pointer to a specific Segment Buffer in a specific Segment. It contains the segmentId and the offset of the Segment Buffer from the base of the Segment. When communicating a message to other DomainParticipants, Shared Memory Transport only distributes the Buffer Descriptor, avoiding the copy of the message from a DomainParticipant to another. With this descriptor, the receiving DomainParticipant can access the message written in the buffer, as is uniquely identifies the Segment (through the segmentId) and the Segment Buffer (through its offset).

6.4.1.4. Port¶

Represents a channel to communicate Buffer Descriptors. It is implemented as a ring-buffer in shared memory, so that any DomainParticipant can potentially read or write information on it. Each port has a unique identifier, a 32 bit number that can be used to refer to the port. Every DomainParticipant that has been configured with Shared Memory Transport creates a port to receive Buffer Descriptors. The identifier of this port is shared during the Discovery, so that remote peers know which port to use when they want to communicate with each DomainParticipant.

DomainParticipants create a listener to their receiving port, so that they can be notified when a new Buffer Descriptor is pushed to the port.

6.4.1.5. Port Health Check¶

Every time a DomainParticipant opens a Port (for reading or writing), a health check is performed to assess its correctness. The reason is that if one of the processes involved crashes while using a Port, that port can be left inoperative. If the attached listeners do not respond in a given timeout, the Port is considered damaged, and it is destroyed and created again.

6.4.2. SharedMemTransportDescriptor¶

In addition to the data members defined in the TransportDescriptorInterface, the TransportDescriptor for Shared Memory defines the following ones:

Member	Data type	Default	Accessor / Mutator	Description
`segment_size_`	`uint32_t`	`512*1024`	`segment_size()`	Size of the shared memory segment (in octets).
`port_queue_capacity_`	`uint32_t`	`512`	`port_queue_capacity()`	The size of the listening port (in messages).
`healthy_check_timeout_ms_`	`uint32_t`	`1000`	`healthy_check_timeout_ms()`	Timeout for the health check of ports (in milliseconds).
`rtps_dump_file_`	`string`	`""`	`rtps_dump_file()`	Full path of the protocol dump file.
`default_reception_threads`	ThreadSettings		`default_reception_threads`	Default ThreadSettings for the reception threads.
`reception_threads`	`std::map<uint32_t, ThreadSettings>`		`reception_threads`	ThreadSettings for the reception threads on specific ports.
`dump_thread()`	ThreadSettings		`dump_thread()`	ThreadSettings for the SHM dump thread.

If rtps_dump_file_ is not empty, all the shared memory traffic on the DomainParticipant (sent and received) is traced to a file. The output file format is tcpdump hexadecimal text, and can be processed with protocol analyzer applications such as Wireshark. Specifically, to open the file using Wireshark, use the “Import from Hex Dump” option using the “Raw IPv4” encapsulation type.

Note

The kind value for a SharedMemTransportDescriptor is given by the value LOCATOR_KIND_SHM.

Warning

Setting a segment_size() close to or smaller than the data size poses a high risk of data loss, since the write operation will overwrite the buffer during a single send operation.

6.4.3. Enabling Shared Memory Transport¶

Fast DDS enables a SHM transport by default. Nevertheless, the application can enable other SHM transports if needed. To enable a new SHM transport in a DomainParticipant, first create an instance of SharedMemTransportDescriptor, and add it to the user transport list of the DomainParticipant.

The examples below show this procedure in both C++ code and XML file.

DomainParticipantQos qos;

// Create a descriptor for the new transport.
std::shared_ptr<SharedMemTransportDescriptor> shm_transport = std::make_shared<SharedMemTransportDescriptor>();

// [OPTIONAL] ThreadSettings configuration
shm_transport->default_reception_threads(eprosima::fastdds::rtps::ThreadSettings{-1, 0, 0, -1});
shm_transport->set_thread_config_for_port(12345, eprosima::fastdds::rtps::ThreadSettings{-1, 0, 0, -1});
shm_transport->dump_thread(eprosima::fastdds::rtps::ThreadSettings{-1, 0, 0, -1});

// Link the Transport Layer to the Participant.
qos.transport().user_transports.push_back(shm_transport);

<?xml version="1.0" encoding="UTF-8" ?>
<dds>
    <profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
        <transport_descriptors>
            <!-- Create a descriptor for the new transport -->
            <transport_descriptor>
                <transport_id>shm_transport</transport_id>
                <type>SHM</type>
                <default_reception_threads> <!-- OPTIONAL -->
                    <scheduling_policy>-1</scheduling_policy>
                    <priority>0</priority>
                    <affinity>0</affinity>
                    <stack_size>-1</stack_size>
                </default_reception_threads>
                <reception_threads> <!-- OPTIONAL -->
                    <reception_thread port="12345">
                        <scheduling_policy>-1</scheduling_policy>
                        <priority>0</priority>
                        <affinity>0</affinity>
                        <stack_size>-1</stack_size>
                    </reception_thread>
                </reception_threads>
                <dump_thread> <!-- OPTIONAL -->
                    <scheduling_policy>-1</scheduling_policy>
                    <priority>0</priority>
                    <affinity>0</affinity>
                    <stack_size>-1</stack_size>
                </dump_thread>
            </transport_descriptor>
        </transport_descriptors>

        <participant profile_name="SHMParticipant">
            <rtps>
                <!-- Link the Transport Layer to the Participant -->
                <userTransports>
                    <transport_id>shm_transport</transport_id>
                </userTransports>
            </rtps>
        </participant>
    </profiles>
</dds>

Note

In case that several transports are enabled, the discovery traffic is always performed using the UDP/TCP transport, even if the SHM transport is enabled in both participants running in the same machine. This may cause discovery issues if one or several of the participants only has SHM enabled and other participants use some other transport at the same time. Also, when two participants on the same machine have SHM transport enabled, the user data communication between them is automatically performed by SHM transport only. The rest of the enabled transports are not used between those two participants.

Hint

To configure discovery traffic through Shared Memory, the default builtin transports must be disabled. In that way, communication is performed completely using Shared Memory. The snippet examples below show this procedure in both C++ code and XML file. See HelloWorldExampleSharedMem for a complete example.

DomainParticipantQos qos;

// Create a descriptor for the new transport.
std::shared_ptr<SharedMemTransportDescriptor> shm_transport = std::make_shared<SharedMemTransportDescriptor>();

// Link the Transport Layer to the Participant.
qos.transport().user_transports.push_back(shm_transport);

// Explicit configuration of SharedMem transport
qos.transport().use_builtin_transports = false;

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <transport_descriptors>
        <!-- Create a descriptor for the new transport -->
        <transport_descriptor>
            <transport_id>shm_transport_only</transport_id>
            <type>SHM</type>
        </transport_descriptor>
    </transport_descriptors>

    <participant profile_name="DisableBuiltinTransportsParticipant">
        <rtps>
            <!-- Link the Transport Layer to the Participant -->
            <userTransports>
                <transport_id>shm_transport_only</transport_id>
            </userTransports>
            <useBuiltinTransports>false</useBuiltinTransports>
        </rtps>
    </participant>
</profiles>

6.4.4. HelloWorldExampleSharedMem¶

A Shared Memory version of helloworld example can be found in the HelloWorldExampleSharedMem folder. It shows a publisher and a subscriber that communicate through Shared Memory.