2 .\" Copyright (c) 2012 The DragonFly Project. All rights reserved.
4 .\" This code is derived from software contributed to The DragonFly Project
5 .\" by Nuno Antunes <nuno.antunes@gmail.com>.
7 .\" Redistribution and use in source and binary forms, with or without
8 .\" modification, are permitted provided that the following conditions
11 .\" 1. Redistributions of source code must retain the above copyright
12 .\" notice, this list of conditions and the following disclaimer.
13 .\" 2. Redistributions in binary form must reproduce the above copyright
14 .\" notice, this list of conditions and the following disclaimer in
15 .\" the documentation and/or other materials provided with the
17 .\" 3. Neither the name of The DragonFly Project nor the names of its
18 .\" contributors may be used to endorse or promote products derived
19 .\" from this software without specific, prior written permission.
21 .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
22 .\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
23 .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
24 .\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
25 .\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
26 .\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING,
27 .\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
28 .\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
29 .\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
30 .\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
31 .\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
38 .Nm lwkt_initport_thread ,
39 .Nm lwkt_initport_spin ,
40 .Nm lwkt_initport_serialize ,
41 .Nm lwkt_initport_panic ,
42 .Nm lwkt_initport_replyonly_null ,
43 .Nm lwkt_initport_replyonly ,
44 .Nm lwkt_initport_putonly ,
50 .Nm lwkt_initmsg_abortable ,
58 .Nd LWKT message passing interface
62 .Fn lwkt_initport_thread "lwkt_port_t port" "struct thread *td"
64 .Fn lwkt_initport_spin "lwkt_port_t port"
66 .Fn lwkt_initport_serialize "lwkt_port_t port" "struct lwkt_serialize *slz"
68 .Fn lwkt_initport_panic "lwkt_port_t port"
70 .Fn lwkt_initport_replyonly_null "lwkt_port_t port"
72 .Fn lwkt_initport_replyonly "lwkt_port_t port" "void (*rportfn)(lwkt_port_t, lwkt_msg_t)"
74 .Fn lwkt_initport_putonly "lwkt_port_t port" "int (*pportfn)(lwkt_port_t, lwkt_msg_t)"
76 .Fn lwkt_sendmsg "lwkt_port_t port" "lwkt_msg_t msg"
78 .Fn lwkt_domsg "lwkt_port_t port" "lwkt_msg_t msg" "int flags"
80 .Fn lwkt_forwardmsg "lwkt_port_t port" "lwkt_msg_t msg"
82 .Fn lwkt_abortmsg "lwkt_msg_t msg"
85 .Fn lwkt_initmsg "lwkt_msg_t msg" "lwkt_port_t rport" "int flags"
87 .Fn lwkt_initmsg_abortable "lwkt_msg_t msg" "lwkt_port_t rport" "int flags" "void (*abortfn)(lwkt_msg_t)"
89 .Fn lwkt_beginmsg "lwkt_port_t port" "lwkt_msg_t msg"
91 .Fn lwkt_replymsg "lwkt_msg_t msg" "int error"
93 .Fn lwkt_getport "lwkt_port_t port"
95 .Fn lwkt_waitport "lwkt_port_t port" "int flags"
97 .Fn lwkt_waitmsg "lwkt_msg_t msg" "int flags"
99 .Fn lwkt_checkmsg "lwkt_msg_t msg"
101 .Fn lwkt_dropmsg "lwkt_msg_t msg"
103 Light weight kernel threads in DragonFly may use a message passing interface to
104 communicate with each other.
105 Messages are sent to message ports.
106 All light weight kernel threads have a built-in message port, but you may create
107 additional ports if necessary.
108 The following types of message ports are available:
118 Ports of type 'thread' are owned by a single light weight kernel thread.
119 When a message is sent to a port of type 'thread', only the owner of that port
120 is allowed to retrieve the message from it.
121 When a message is sent to a port of type 'spin' or to a port of
122 type 'serializer', multiple threads are allowed to check that port for new
123 messages and compete to retrieve them.
124 You define the port type when you initialize the port.
125 By default, the built-in port of every light weight kernel thread is
126 automatically initialized to type 'thread'.
128 When a message is sent, the receiver should normally send back a reply.
129 The reply is sent to the reply port that is registered on the original message.
130 Messages can be replied to synchronously or asynchronously.
131 The sender may request a synchronous or asynchronous reply to the message,
132 however the target port will ultimately decide how the message will be treated.
133 .Sh MESSAGE FUNCTIONS
134 Messages must be initialized before being used.
137 function initializes a message.
140 argument identifies the reply port which will be used for asynchronous replies.
143 argument sets any required flags for this message.
144 Flags passed this way will simply be or'ed to any already existing flags on the
148 .Fn lwkt_initmsg_abortable
149 function is similar to
151 but it takes an additional parameter
153 which defines the abort function for this message.
157 function requests an asynchronous reply, sends the message and returns
159 Under normal circumstances, users of this function may always expect the reply
160 to be queued to the reply port registered on the message.
163 argument defines the target port to which the
165 message will be sent to.
169 function requests a synchronous reply, sends the message and does not return
170 until the message has been replied to.
171 If the target port supports synchronous reply, this function will return that
173 If not, and this is the most common case, this function will block and wait for the
174 reply to arrive and then return it.
177 argument defines the target port to which the
179 message will be sent to.
183 function replies to a message that was processed asynchronously by the target
185 This function is used by the thread on the receiving side.
188 argument is the message being replied to and the
190 argument is the actual response to send back.
194 simply forwards a message to another port.
197 argument defines the target port to which the
199 message will be sent to.
201 If a message has been initialized as abortable, you can use the
203 function to try to abort it.
206 passed upon the initialisation with
207 .Fn lwkt_initmsg_abortable
208 will be called by this function.
212 will dequeue the specified message from the target port it was sent to and makes
213 it look like it was never sent.
216 .Fn lwkt_initport_thread
217 initializes the specified
219 with the default 'thread' port type handlers.
222 argument defines the owner thread of the port and only that thread is allowed to
223 receive messages on it.
226 .Fn lwkt_initport_spin
227 initializes the specified
229 with the default 'spin' port type handlers.
230 It will also initialize the embedded spinlock within the
231 lwkt_port structure which will protect subsequent port access.
234 .Fn lwkt_initport_serialize
235 function initializes the specified
237 with the default 'serializer' port type handlers.
238 The subsequent port access will be protected by the passed
244 function checks the specified
246 for available messages, dequeues the first one and returns it.
247 If no messages are available then
250 This function is used by threads on the receiving side.
254 function checks the specified
256 for available messages, dequeues the first one and returns it.
257 If no messages are available then the caller thread will sleep until a message
258 arrives on the specified port.
261 argument defines the flags used for the sleep.
262 This function is used by threads on the receiving side.
263 .Sh SPECIAL PORT INITIALIZERS
265 .Fn lwkt_initport_replyonly
266 function initializes a
268 which is used only as reply port and may have a custom reply port handler.
269 The reply port handler is specified with the
272 All the other handlers will panic the system if they are called.
273 This initializer is normally used on ports for freeing resources after the
274 messages have fulfilled their purpose.
277 .Fn lwkt_initport_replyonly_null
278 function initializes a
280 which is used only as reply port.
281 The reply port handler will simply mark the message as being done and will not
283 All the other handlers will panic the system if they are called.
286 .Fn lwkt_initport_putonly
287 function initializes a
289 which is used only as target port.
290 The putport handler is specified with the
293 All the other handlers will panic the system if they are called.
296 .Fn lwkt_initport_panic
297 function initializes a
299 which will panic the system if any of its handlers are called.
300 This function is sometimes used to initialize a reply-only port which does not
301 expect the messages to be replied to, e.g. when the messages should be consumed
302 by the receiving thread and never replied back.
303 .Sh INTERNAL MESSAGE FUNCTIONS
304 The following functions are used only by the infrastructure, you should not
305 need to use them directly unless in very rare cases.
309 function simply calls the target port's putport handler.
310 This function is only called by the
315 The putport handler returns
317 for messages processed asynchronously or any other value for messages processed
319 That return value of the putport handler is propagated by this function.
322 argument defines the target port to which the
324 message will be sent to.
328 function puts the caller to sleep until the specified
330 message has been replied to.
333 argument defines the flags used for the sleep.
335 The LWKT msgport implementation resides in
336 .Pa sys/kern/lwkt_msgport.c .
340 * Example 1: per CPU threads.
344 #include <sys/thread.h>
345 #include <sys/msgport.h>
346 #include <sys/msgport2.h>
348 static void my_service_loop(void *dummy);
349 lwkt_port_t my_service_portfn(int cpu);
350 void my_service_sendmsg(lwkt_msg_t lmsg, int cpu);
351 int my_service_domsg(lwkt_msg_t lmsg, int cpu);
353 /* Array of per-CPU target ports */
354 struct lwkt_port *my_service_ports[MAXCPU];
357 * Create per-cpu threads for handling msg processing. Remember that built-in
358 * lwkt ports are automatically initialized to type 'thread' so we don't need
359 * to initialize them explicitly.
362 my_per_cpu_service_init(void)
367 for (i = 0; i < ncpus; ++i) {
368 lwkt_create(my_service_loop, NULL, &td,
369 NULL, 0, i, "myservice_cpu %d", i);
370 my_service_ports[i] = &td->td_msgport;
375 * This is the routine executed by the service threads on each CPU.
378 my_service_loop(void *dummy __unused)
381 thread_t td = curthread;
382 int cpu = curthread->td_gd->gd_cpuid;
384 while ((msg = lwkt_waitport(&td->td_msgport, 0)) != NULL) {
385 /* Do some work in the receiver thread context. */
386 kprintf("Received message on CPU %d.\en", cpu);
388 /* And finally reply to the message. */
389 lwkt_replymsg(msg, 0);
394 * Given a CPU id, return our respective service port.
397 my_service_portfn(int cpu)
399 return my_service_ports[cpu];
403 * Send an asynchronous message to the service thread on a specific CPU.
406 my_service_sendmsg(lwkt_msg_t lmsg, int cpu)
408 KKASSERT(cpu < ncpus);
409 lwkt_sendmsg(my_service_portfn(cpu), lmsg);
413 * Send a synchronous message to the service thread on a specific CPU.
416 my_service_domsg(lwkt_msg_t lmsg, int cpu)
418 KKASSERT(cpu < ncpus);
419 return lwkt_domsg(my_service_portfn(cpu), lmsg, 0);
423 * Example use case. Initialize the service threads and send each one a
430 lwkt_port_t builtin_port = &curthread->td_msgport;
433 my_per_cpu_service_init();
434 for (i=0; i<ncpus; ++i) {
435 kprintf("Sending msg to CPU %d.\en", i);
436 lwkt_initmsg(&lmsg, builtin_port, 0);
437 my_service_domsg(&lmsg, i);
442 * Example 2: Dynamic allocated message passing with automatic free.
444 * This scenario is used when resources need to be freed after the message
445 * has been replied to. Features:
446 * - An argument is passed within the message.
447 * - Messages are allocated with kmalloc(). Replying to the msg, kfree()s it.
450 #include <sys/thread.h>
451 #include <sys/msgport.h>
452 #include <sys/msgport2.h>
454 void my_service_queue(void *arg);
456 lwkt_port my_autofree_rport;
457 lwkt_port_t my_service_port;
460 * Use this function to send messages with a void * argument to our
464 my_service_queue(void *arg)
468 msg = kmalloc(sizeof(*msg), M_TEMP, M_WAITOK);
470 /* Set reply port to autofree. */
471 lwkt_initmsg(msg, &my_autofree_rport, 0);
473 /* Attach the argument to the message. */
474 msg->u.ms_resultp = arg;
477 lwkt_sendmsg(my_service_port, msg);
481 * This is the routine executed by our service thread.
484 my_service_loop(void *dummy __unused)
487 thread_t td = curthread;
489 while ((msg = lwkt_waitport(&td->td_msgport, 0)) != NULL) {
491 * Do some work in the receiver thread context. In this
492 * example, the sender wrote his name in the argument he
493 * sent us. We print it here.
495 char *arg = msg->u.ms_resultp;
496 kprintf("%s: Hi %s! Got your msg.\en", curthread->td_comm, arg);
498 /* And finally reply to the message. */
499 lwkt_replymsg(msg, 0);
504 my_autofree_reply(lwkt_port_t port, lwkt_msg_t msg)
506 kfree(msg->u.ms_resultp, M_TEMP);
511 my_service_init(void)
515 /* Initialize our auto free reply port. */
516 lwkt_initport_replyonly(&my_autofree_rport, my_autofree_reply);
518 /* Create our service thread on CPU 0. */
519 lwkt_create(my_service_loop, NULL, &tdp, NULL, 0, 0, "myservice");
520 my_service_port = &tdp->td_msgport;
524 * Example use case. Initialize the service and send the current thread name
525 * to the service thread.
534 len = strlen(curthread->td_comm);
535 arg = kmalloc(len + 1, M_TEMP, M_WAITOK);
536 bcopy(curthread->td_comm, arg, len + 1);
537 kprintf("%s: Sending message.\en", curthread->td_comm);
538 my_service_queue(arg);
542 All the default putport handlers (used when a message is sent) currently
543 implement asynchronous putports only, i.e. all *_putport() handlers return
545 You can still have synchronous putport handlers (which are run in the sender's
546 context) but you have to implement the function yourself and then override the
549 Port handler functions can be overridden with custom functions if required.
550 You can override the default putport handler by either using the
551 .Fn lwkt_initport_putonly
552 initializer, or by manipulating the mp_putport handler pointer directly on the
555 There is one such case where the putport handler is overridden in
556 .Pa sys/kern/netisr.c.
557 In that case, the putport handler is overridden to detect a loopback message
558 (when the target port belongs to the sending thread).
559 This special putport handler turns the sent message into a direct function call
560 instead of queueing it to the port.
564 function works differently depending on the original message request. If the
565 message was originally an asynchronous request, the reply will be queued to the
566 sender's reply port. If the message was originally a synchronous request, then
567 this function will just write the error response on the message and wake up the
568 waiter without queueing the message to the reply port.
569 There is no need to queue in the synchronous request case because the original
570 sender had blocked waiting on this specific message with
573 As is the case with putport handler, the replyport handler can also be
575 You override the default replyport handler by using the
576 .Fn lwkt_initport_replyonly
578 .Fn lwkt_initport_replyonly_null
579 port initializers, or by manipulating the mp_replyport handler pointer directly
580 on the lwkt_port structure.
582 The sent message structure is reused for replies.
583 When a message is replied to, the error response is written on the message
584 which is subsequently sent to the reply port.
589 The LWKT msgport interface first appeared in
595 message passing interface implementation was written by
597 This manual page was written by