Multi-Threaded Libevent Server Example

Recently I had a need to write a socket server in C. In the past I’ve done lots of these in Java, and some in C. Being a huge fan and avid user of memcached, and knowing that memcached uses libevent at its core, I decided to go the libevent route this time. So I looked for some examples. I found cliserver and echoserver, both of which were helpful. So, I went about implementing my server using these two as examples for how to put libevent to work dispatching events and doing non-blocking I/O for me. So far, so good.

Libevent is a nice library for handling and dispatching events, as well as doing nonblocking I/O. This is fine, except that it is basically single-threaded — which means that if you have multiple CPUs or a CPU with hyperthreading, you’re really under-utilizing the CPU resources available to your server application because your event pump is running in a single thread and therefore can only use one CPU core at a time.

The solution is to create one libevent event queue (AKA event_base) per active connection, each with its own event pump thread. This project does exactly that, giving you everything you need to write high-performance, multi-threaded, libevent-based socket servers.

There are mentionings of running libevent in a multithreaded implementation, however it is very difficult (if not impossible) to find working implementations. So, I went about developing a working implementation of a multi-threaded, libevent-based socket server.

The overall architecture is a standard, multi-threaded socket server. The main thread listens on a socket and accepts new connections, then farms the actual handling of those connections out to a pool of worker threads. Each connection has its own isolated event queue and runs on a single worker thread.

You can download the code from the sourceforge project page here: Multi-Threaded Libevent Server Example.

The server itself simply echoes whatever you send to it. Start it up, then telnet to it:
telnet localhost 5555
Everything you type should be echoed back to you.

One advantage of using libevent with a handful of event pump threads, is that in many cases you don’t need hundereds or thousands of threads to achieve good performance under load. In theory, for maximum performance, the number of worker threads should be set to the number of CPU cores available. Feel free to experiment with this. If your request processing code tends to block while waiting for database or other blocking operations to complete, you’ll need hundereds or thousands of threads to achieve good performance at speed, because when a thread blocks, it isn’t doing any work. But if your request processing is more of a compute-intensive task which does not block while waiting for external operations or resources, matching the number of threads to the number of CPU cores should theoretically give you optimum performance.

Also note that the server includes a multithreaded work queue implementation, which can be re-used for other purposes.

Since the code is BSD licensed, you are free to use the source code however you wish, either in whole or in part.

Some inspiration and coding ideas came from echoserver and cliserver, both of which are single-threaded, libevent-based servers.

Multi-Threaded Libevent Server Example: https://sourceforge.net/projects/libevent-thread/
Direct Link to Download Page: https://sourceforge.net/projects/libevent-thread/files/
Echoserver is located here: http://ishbits.googlecode.com/svn/trunk/libevent-examples/echo-server/li…
Cliserver is located here: http://nitrogen.posterous.com/cliserver-an-example-libevent-based-socket-se

 

15 thoughts on “Multi-Threaded Libevent Server Example

  1. Francis

    When I compile your multi-threaded-libevent-server, I got following error.
    Could help me to check this erorr. Thank you.
    $ gcc -o echoserver_threaded echoserver_threaded.c workqueue.c -levent -lpthread
    echoserver_threaded.c: In function ‘buffered_on_read':
    echoserver_threaded.c:129:19: error: dereferencing pointer to incomplete type
    echoserver_threaded.c:131:23: error: dereferencing pointer to incomplete type
    echoserver_threaded.c:131:56: error: dereferencing pointer to incomplete type
    workqueue.c: In function ‘workqueue_init':
    workqueue.c:74:3: warning: passing argument 3 of ‘pthread_create’ from incompatible pointer type [enabled by default]
    /usr/include/pthread.h:225:12: note: expected ‘void * (*)(void *)’ but argument is of type ‘int (*)(struct workqueue_t *, int)’

    Reply
    1. ron Post author

      That’s very strange. I’ve compiled on both CentOS 64 bit and Ubuntu 64 bit, and never encountered any errors. Can you please post the details of your OS?

      Reply
          1. Michael

            Hi guys,

            you can modify the while loop as following:

            do {
            nbytes = evbuffer_remove(bev->input, data, 4096);
            if (nbytes == -1) break;
            evbuffer_add(client->output_buffer, data, nbytes);
            if (nbytes < 4096) break;
            }while(1);

            It should work.

          2. ron Post author

            Are you seeing a bug? I’m using this code in a high-volume server and it’s been working fine as-is for over two years. If you’re seeing a problem, please provide code to reproduce it and I’ll see if I can fix it. Should be good as-is though.

  2. anil

    Even I face same issue. here are the details. Thanks in advance!

    /Tools/libevent-thread$ gcc -o echoserver_threaded echoserver_threaded.c workqueue.c -levent -lpthread
    echoserver_threaded.c: In function ‘buffered_on_read’:
    echoserver_threaded.c:132:19: error: dereferencing pointer to incomplete type
    echoserver_threaded.c:134:23: error: dereferencing pointer to incomplete type
    echoserver_threaded.c:134:56: error: dereferencing pointer to incomplete type

    /Tools/libevent-thread$ lsb_release -a
    No LSB modules are available.
    Distributor ID: Ubuntu
    Description: Ubuntu 11.10
    Release: 11.10
    Codename: oneiric

    Reply
  3. adrian

    You don’t wait for the created threads to finish, or i am missing something… ? no ptherad_exit(). The workqueue.jobs_mutex, and workqueue.jobs_cond shouldn’t be destroyed as well when the program ends?

    First, i got those errors too, i am using libevent 2.0.21-stable. For getting data from socket, i used bufferevent_read(), for sending data i used bufferevent_write().

    Reply
    1. ron Post author

      event_base_dispatch(evbase_accept) should be blocking until the server is terminated by calling killServer(), which is called when the process receives a signal. I just pkill -9 the thing when I want it to die.

      Reply
  4. adrian

    Forgot one little thing… .

    while (worker->workqueue->waiting_jobs == NULL) {
    pthread_cond_wait(&worker->workqueue->jobs_cond, &worker->workqueue->jobs_mutex);
    }

    You should check for worker->terminate inside this loop, otherwise it gets stuck here. This is observable when you terminate the program using C-c and the jobs linked list is empty.

    Quite nice and useful piece of code though.

    Reply
    1. ron Post author

      The workqueue_shutdown() function should be taking care of this by setting all workers to terminate, removing all jobs, then notifying all workers by calling pthread_cond_broadcast(&workqueue->jobs_cond).

      void workqueue_shutdown(workqueue_t *workqueue) {
      worker_t *worker = NULL;

      /* Set all workers to terminate. */
      for (worker = workqueue->workers; worker != NULL; worker = worker->next) {
      worker->terminate = 1;
      }

      /* Remove all workers and jobs from the work queue.
      * wake up all workers so that they will terminate. */
      pthread_mutex_lock(&workqueue->jobs_mutex);
      workqueue->workers = NULL;
      workqueue->waiting_jobs = NULL;
      pthread_cond_broadcast(&workqueue->jobs_cond);
      pthread_mutex_unlock(&workqueue->jobs_mutex);
      }

      Reply
      1. nearmeng

        I think it is really a problem.

        As your code presents, in the workqueue_shutdown() function, you want`workqueue->waiting_jobs = NULL;` and notify the work threads by `pthread_cond_broadcast(&workqueue->jobs_cond);` but i think it can’t be seccessful actually. Because if the work thread is blocked in `pthread_cond_wait()`,when he recved the broadcast signal, he will wake up and lock the mutex (if we do not consider the contend), then he will check the `woker->workqueue->waiting_jobs == null`again, and it is not satisfied, so he will blocked by `pthread_cond_wait()` again. And you can’t shutdown the worker thread as you wish.

        So I think if we test if the `worker->terminate == 1` in the loop of `while (worker->workqueue->waiting_jobs == NULL)` as the adrian said, it will solve the problem.

        Thanks for your code anyway :)
        If there is something wrong with my thought, please let me kown.

        Reply
        1. ron Post author

          I believe you are correct. I’ve added an extra check for the terminate boolean, so it should be fixed now. I committed the change into the git repo, and also created a new release download.

          I’m noticing that newer versions of libevent have multi-threading support built in. Not all Linux distros have these newer libevent versions though. So…the usefulness of my little example may be diminishing as time goes by. All I can say for sure is that it works for me, at least in the limited (but high volume) project in which I used it.

          Reply
  5. Mike

    Thanks for the example code. I try to learn libevent and am still a complete noob.

    However, I don’t get why there is an event_base per client? Doesn’t that mean that you can only deal ‘number of threads’ clients in parallel? Because ‘event_base_dispatch’ is blocking until the client is done. You use something like libevent to not have that problem, right? Or do I understand the code in a wrong way?

    Wouldn’t having an event_base per thread and then assigning threads to clients be better? Or even better get all events on a single thread and give all the work to the work queue (or does this not work because libevent is not thread safe?)

    Thanks and regards

    Reply
    1. ron Post author

      The events are asynchronous and non-blocking. The idea is that you initiate your processing and then immediately return. When the processing completes, you get a callback, then you populate a buffer for the response and queue it up to be asynchronously delivered to the client. This is the point of asynchronous I/O and libevent: to eliminate the overhead of hundreds or thousands of thread (and the costly context switches which go with them) and to achieve higher levels of throughput.

      If you’ve ever used Node.js, the asynchronous I/O model should be very familiar. Node.js does nearly everything asynchronously.

      For example, let’s say you had a server which takes in some parameters in the request, queries MySQL, and then returns some data to the client. Maybe a header, a bit of data per row returned from the query, and a footer containing some totals. With the asynchronous model, when you get the data-ready callback, you queue up the header to be sent back to the client, then you initiate the MySQL query and immediately return. As rows come back from the MySQL query, you get callbacks to another function which you provided when you initiated the query. In each of those callbacks, you take the row data, format it, and queue it up to be sent back to the client. When the query completes, you get a callback to another function. In that function, you send out the footer with totals, then end the response and return. At that point, all that’s left is for the asynchronous I/O to finish draining the buffers you queued up containing the response to the client, then close the connection.

      This is an over-simplification, I realize. But it serves as an illustration of how a server works when using asynchronous I/O in an event-based model. Because everything is asynchronous and nothing ever blocks, you need far fewer threads. Because you save the expensive thread context switches, you get higher concurrency and throughput than with a threaded server.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *


3 + five =

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>