stefan's blag and stuff

Blog – 2015-01-21 – Using ipv6only in Nginx

The Objective

Some days ago I fiddled with the nginx (version 1.6.2) configuration of our server. While the last debian update to jessie we had to change all files in /etc/nginx/sites-available/ to something like

server {
        listen 80;
        listen 443 ssl;
        listen [::]:80;
        listen [::]:443 ssl;
        server_name stefanchrist.eu www.stefanchrist.eu;
	[…]

Otherwise the nginx daemon wasn't starting and serving webpages on IPv4 and IPv6. Using this configuration nginx opens two sockets for each port. One socket for IPv4 and one socket for IPv6 connections. The command netstat shows

$ netstat -tlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address  Foreign Address  State   PID/Program name
tcp        0      0 *:http         *:*              LISTEN  20658/nginx -g daem
tcp        0      0 *:https        *:*              LISTEN  20658/nginx -g daem
tcp6       0      0 [::]:http      [::]:*           LISTEN  20658/nginx -g daem
tcp6       0      0 [::]:https     [::]:*           LISTEN  20658/nginx -g daem

Everything is fine with this setup, excepted … my perfection and early adopter parts of my brain weren't satisfied. The IPv6 address space embeds the IPv4 address space and the linux kernel socket supports the socket options IPV6_V6ONLY (manpage 7 ipv6) to serve IPv4 and IPv6 connections with a single socket. It's possible for the userspace to treat IPv4 addresses and connections as another kind of an IPv6 connection. The difference is only the unusal IPv6 address.

The goal is serving both IP versions with only one (not two) socket per port. It's nicer because the number of listing sockets is reduced on our server and we continue the next step to abandon IPv4 completely. So let's do it.

The Configuration

There are some configuration challanges. First you should check the default value of the IPV6_V6ONLY flag. Use

$ sysctl net.ipv6.bindv6only
net.ipv6.bindv6only = 0

Ok. If an application doesn't provide any information about the flag, the socket will be opened as IPv4+IPv6. Some programs work out of the box, but some others may fail with the error EADDRINUSE (Address already in use), because they try to open the same port for IPv4 and then for IPv6. It's generally the best that the application is specific about his intend and provides this information to the socket syscall, e.g. nginx in version 1.6.2

Second you should read the nginx documentation for the listen directive and especially the text about the parameter ipv6only=on|off. After that I tell you about a detail in it:
The parameter is different from e.g. the ssl flag. The flag ssl can be used in multiple server contexts and be switch on and off as you wish. The flag ipv6only can only be set once per port (and address). Only a single listen directive my contain the parameter and it will be valid for all server contexts using this port. If you use it twice, the nginx daemon won't start and will write the following error messages to his error log

$ tail /var/log/nginx/error.log
2015/01/18 22:01:13 [emerg] 20645#0: bind() to [::]:80 failed (98: Address already in use)
2015/01/18 22:01:13 [emerg] 20645#0: bind() to [::]:443 failed (98: Address already in use)

I didn't noticed this while skipping through the documentation, but maybe you have done it:

These parameters can be specified in any listen directive, but only once for a given address:port pair.

(It's also mentioned in a forum post Re: startup error -> "[emerg] duplicate listen options" if 2nd IPv6 listener is enabled). You have to pick one of your servers contexts and use the listen directives as follows:

server {
        listen [::]:80 ipv6only=off;
	listen [::]:443 ssl ipv6only=off;
	server_name stcim.de www.stcim.de;
	[…]

For every other server context use

server {
	listen [::]:80;
	listen [::]:443 ssl;
        server_name stefanchrist.eu www.stefanchrist.eu;
	[…]

You can use

$ grep listen -R /etc/nginx/sites-available/

to find all of them. After that you can restart the nginx daemon and hope for the best ;-)

If everything is successful and nginx doesn't fill his error log with messages, you may check with tool netstat that it opens only one socket per port.

$ netstat -tlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address  Foreign Address  State   PID/Program name
tcp6       0      0 [::]:http      [::]:*           LISTEN  20828/nginx -g daem
tcp6       0      0 [::]:https     [::]:*           LISTEN  20828/nginx -g daem

Ah neet … and don't forget to check reachabilty with both IPv4 and IPv6.