Improve IPv6 support in globalzone #503

sjorge · 2015-10-31T22:46:29Z

summary

This PR further improved IPv6 support for compute nodes/smartos.

sysinfo now also lists IPv6 addresses
/etc/hosts now also gets IPv6 entry if needed

detailed explanation of changes

svc-ndp got added to the overlay
We give in.ndp time to stabilize, 3 seconds seems to be the sweet spot. (At least in my testing).
After that we update the sysinfo cache, when using addrconf or static IPv6 ndp will make some changes to the network. Like binding the allocated IPv6 address and updating the routes.

I chose not to pull in the sdc config into svc-ndp and just do the small 3 second delay regardless if we have admin_ip6 set or not. Although I am willing to change this if this is preferred to skip it or not.

Boot delay seemed to be none existing, probably due to smf starting some services in parallel.

sysinfo
ipaddr variable is renamed to ip4addr and ip6addr has been added. Additional properties for ipv6 addresses have been added. IPv6 info can now be queried like IPv4 info, this will be used later.
identity-node
With the updated sysinfo we can now obtain the IPv6 address and also add host entries to /etc/hosts. This should allow sendmail to again resolve all it's own addresses.

The 3 second wait for sysinfo to stabilize here is optional as the CONFIG_ is available, we skip this when admin_ip6 is not set.

test data

no IPv6
/usbkey/config

#
# This file was auto-generated and must be source-able by bash.
#

# admin_nic is the nic admin_ip will be connected to for headnode zones.
admin_nic=0:15:0:5:d9:c1
admin_ip=dhcp
admin_netmask=
admin_network=
admin_gateway=dhcp

#admin_ip6=addrconf
#admin_ip6=2001:zzzz:yyy:xxxx::234/48

#external_nic=12:b0:5b:e:28:4e
#external_ip=127.0.1.1
#external_netmask=255.255.255.0
#external_ip6=addrconf

headnode_default_gateway=

dns_resolvers=8.8.8.8,8.8.4.4
hostname=soth
dns_domain=acheron.be

ntp_hosts=ntp.acheron.be
compute_node_ntp_hosts=dhcp

/etc/hosts

# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
# Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# Internet host table
#
::1             localhost
127.0.0.1       localhost loghost
172.16.xx.227   soth soth.acheron.be

output of sysinfo

Live_Image='20151031T170124Z'
System_Type='SunOS'
Boot_Time='1446332086'
SDC_Version='7.0'
Manufacturer='Joyent'
Product='SmartDC HVM'
Serial_Number='618a01b8-c37b-473f-8cdb-ca2a66d6adf7'
SKU_Number='001'
HW_Version='7.20151029T053122Z'
HW_Family='Virtual Machine'
VM_Capable='true'
CPU_Type='Unknown'
CPU_Virtualization='none'
CPU_Physical_Cores=0
Nic_Tags=external,admin
Setup='false'
UUID='618a01b8-c37b-473f-8cdb-ca2a66d6adf7'
Hostname='soth'
CPU_Total_Cores=2
MiB_of_Memory=2047
Disk_c0d1_size_in_GB=21
NIC_external='vioif1'
NIC_admin='vioif0'
Network_Interface_vioif0_MAC_Address='00:15:00:05:d9:c1'
Network_Interface_vioif0_IPv4_Address='172.16.xx.227'
Network_Interface_vioif0_IPv6_Address=''
Network_Interface_vioif0_NIC_Names='admin'
Network_Interface_vioif0_Link_Status='up'
Network_Interface_vioif1_MAC_Address='12:b0:5b:0e:28:4e'
Network_Interface_vioif1_IPv4_Address=''
Network_Interface_vioif1_IPv6_Address=''
Network_Interface_vioif1_NIC_Names='external'
Network_Interface_vioif1_Link_Status='up'
Virtual_Network_Interface_external0_MAC_Address='02:08:20:1c:5a:24'
Virtual_Network_Interface_external0_IPv4_Address=''
Virtual_Network_Interface_external0_IPv6_Address=''
Virtual_Network_Interface_external0_Link_Status='up'
Virtual_Network_Interface_external0_VLAN='0'
Virtual_Network_Interface_external0_Host_Interface='vioif1'
Bootparam_console='vga'
Bootparam_vga_mode='115200,8,n,1,-'
Bootparam_root_shadow='$5$2HOHRnK3$NvLlm.1KQBbB0WjoP7xcIwGnllhzp2HnT.mDO7DpxYA'
Bootparam_smartos='true'

addrconf
/usbkey/config

#
# This file was auto-generated and must be source-able by bash.
#

# admin_nic is the nic admin_ip will be connected to for headnode zones.
admin_nic=0:15:0:5:d9:c1
admin_ip=dhcp
admin_netmask=
admin_network=
admin_gateway=dhcp

admin_ip6=addrconf
#admin_ip6=2001:zzzz:yyy:xxxx::234/48


#external_nic=12:b0:5b:e:28:4e
#external_ip=127.0.1.1
#external_netmask=255.255.255.0
#external_ip6=addrconf

headnode_default_gateway=

dns_resolvers=8.8.8.8,8.8.4.4
hostname=soth
dns_domain=acheron.be

ntp_hosts=ntp.acheron.be
compute_node_ntp_hosts=dhcp

/etc/hosts

# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
# Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# Internet host table
#
::1             localhost
127.0.0.1       localhost loghost
172.16.xx.227   soth soth.acheron.be
2001:zzzz:yyy:xxxx:215:ff:fe05:d9c1       soth soth.acheron.be

output of sysinfo

Live_Image='20151031T170124Z'
System_Type='SunOS'
Boot_Time='1446332236'
SDC_Version='7.0'
Manufacturer='Joyent'
Product='SmartDC HVM'
Serial_Number='618a01b8-c37b-473f-8cdb-ca2a66d6adf7'
SKU_Number='001'
HW_Version='7.20151029T053122Z'
HW_Family='Virtual Machine'
VM_Capable='true'
CPU_Type='Unknown'
CPU_Virtualization='none'
CPU_Physical_Cores=0
Nic_Tags=external,admin
Setup='false'
UUID='618a01b8-c37b-473f-8cdb-ca2a66d6adf7'
Hostname='soth'
CPU_Total_Cores=2
MiB_of_Memory=2047
Disk_c0d1_size_in_GB=21
NIC_external='vioif1'
NIC_admin='vioif0'
Network_Interface_vioif0_MAC_Address='00:15:00:05:d9:c1'
Network_Interface_vioif0_IPv4_Address='172.16.xx.227'
Network_Interface_vioif0_IPv6_Address='2001:zzzz:yyy:xxxx:215:ff:fe05:d9c1'
Network_Interface_vioif0_NIC_Names='admin'
Network_Interface_vioif0_Link_Status='up'
Network_Interface_vioif1_MAC_Address='12:b0:5b:0e:28:4e'
Network_Interface_vioif1_IPv4_Address=''
Network_Interface_vioif1_IPv6_Address=''
Network_Interface_vioif1_NIC_Names='external'
Network_Interface_vioif1_Link_Status='up'
Virtual_Network_Interface_external0_MAC_Address='02:08:20:51:9f:72'
Virtual_Network_Interface_external0_IPv4_Address=''
Virtual_Network_Interface_external0_IPv6_Address=''
Virtual_Network_Interface_external0_Link_Status='up'
Virtual_Network_Interface_external0_VLAN='0'
Virtual_Network_Interface_external0_Host_Interface='vioif1'
Bootparam_console='vga'
Bootparam_vga_mode='115200,8,n,1,-'
Bootparam_root_shadow='$5$2HOHRnK3$NvLlm.1KQBbB0WjoP7xcIwGnllhzp2HnT.mDO7DpxYA'
Bootparam_smartos='true'

static
/usbkey/config

#
# This file was auto-generated and must be source-able by bash.
#

# admin_nic is the nic admin_ip will be connected to for headnode zones.
admin_nic=0:15:0:5:d9:c1
admin_ip=dhcp
admin_netmask=
admin_network=
admin_gateway=dhcp

#admin_ip6=addrconf
admin_ip6=2001:zzzz:yyy:xxxx::234/48

#external_nic=12:b0:5b:e:28:4e
#external_ip=127.0.1.1
#external_netmask=255.255.255.0
#external_ip6=addrconf

headnode_default_gateway=

dns_resolvers=8.8.8.8,8.8.4.4
hostname=soth
dns_domain=acheron.be

ntp_hosts=ntp.acheron.be
compute_node_ntp_hosts=dhcp

/etc/hosts

# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
# Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# Internet host table
#
::1             localhost
127.0.0.1       localhost loghost
172.16.10.227   soth soth.acheron.be
2001:zzzz:yyy:xxxx::234   soth soth.acheron.be

output of sysinfo

Live_Image='20151031T170124Z'
System_Type='SunOS'
Boot_Time='1446332403'
SDC_Version='7.0'
Manufacturer='Joyent'
Product='SmartDC HVM'
Serial_Number='618a01b8-c37b-473f-8cdb-ca2a66d6adf7'
SKU_Number='001'
HW_Version='7.20151029T053122Z'
HW_Family='Virtual Machine'
VM_Capable='true'
CPU_Type='Unknown'
CPU_Virtualization='none'
CPU_Physical_Cores=0
Nic_Tags=admin
Setup='false'
UUID='618a01b8-c37b-473f-8cdb-ca2a66d6adf7'
Hostname='soth'
CPU_Total_Cores=2
MiB_of_Memory=2047
Disk_c0d1_size_in_GB=21
NIC_admin='vioif0'
Network_Interface_vioif0_MAC_Address='00:15:00:05:d9:c1'
Network_Interface_vioif0_IPv4_Address='172.16.xx.227'
Network_Interface_vioif0_IPv6_Address='2001:zzzz:yyy:xxxx::234'
Network_Interface_vioif0_NIC_Names='admin'
Network_Interface_vioif0_Link_Status='up'
Network_Interface_vioif1_MAC_Address='12:b0:5b:0e:28:4e'
Network_Interface_vioif1_IPv4_Address=''
Network_Interface_vioif1_IPv6_Address=''
Network_Interface_vioif1_NIC_Names=''
Network_Interface_vioif1_Link_Status='up'
Bootparam_console='vga'
Bootparam_vga_mode='115200,8,n,1,-'
Bootparam_root_shadow='$5$2HOHRnK3$NvLlm.1KQBbB0WjoP7xcIwGnllhzp2HnT.mDO7DpxYA'
Bootparam_smartos='true'

test image

http://sjorge.sinners.be/illumos/joyent/platform-20151031T170124Z.iso

@melloc I guess I need to poke you for this

rmustacc · 2015-10-31T22:50:56Z

I haven't looked at this in detail, but I'm pretty much against the idea that we should sleep in the start method and then do a sysinfo update. This is problematic for a couple reasons. It seems to suggest that there's a static set of addressing, which, especially if using ndp, there's no reason to believe. Instead, we should go through and cause sysinfo to be updated dynamically based upon changes to the assigned addresses in a similar fashion to what was recently done for datalink state changes. Basically, assuming that this is static is asking us to have problems and get out of sync with reality.

sjorge · 2015-10-31T23:06:25Z

I agree that there are better solutions than just sleeping. Although I did not see a cleaner solution that (I with my skill set) could implement.

Having sysinfo dynamically update for each address discovered with in.ndp would indeed be better. That still leaves identity-node that needs to be refreshed too though.

Feel free to cherry pick for example only the sysinfo change. (Or I can submit a PR for this separately)

melloc · 2015-11-04T08:25:15Z

Hey @sjorge , just wanted to let you know that I haven't forgotten about this, and I'll take a look over it as soon as I can.

sjorge · 2016-01-04T14:17:01Z

Oops I forgot this was linked to a PR, branch restored.

melloc · 2016-01-07T07:29:59Z

After talking to @rmustacc , I agree that the way this should be done is by generating a sysevent after the address has been assigned and finished DAD, the same way that links are in OS-4840 (TritonDataCenter/illumos-joyent@5dccc1e) and OS-4841 (4c1efa6). This would ensure that the information accurately reflects the system's configuration. This is really how we should be doing it right now for ip4addr, but we aren't, and it's a correctness issue. (For example, when using DHCP, your address could change
and sysinfo wouldn't notice.)

After giving it some thought, I feel like we would want sysinfo to print out all addresses for that interface. If I get multiple addresses because the local network segment has multiple prefixes associated with it, that should be reflected somehow. The link-local address should probably also show up. (I feel that this is also true for IPv4, too, when there are multiple addresses.) Unfortunately, I haven't given too much thought to what multiple IPv4 and IPv6 addresses in the SDC admin network should look like, so I'm not yet sure if it's better to have two separate fields, like 'ip4addrs' and 'ip6addrs', or one field like 'ipaddrs'.

I think that this PR represents several different issues that need to be fixed separately:

We need to generate sysevents when IP addresses are set
sysinfo needs to represent multiple IP addresses on an interface using the information it gets from the sysevents
Regarding the sendmail issue, I believe the core issue is that its SMF manifest should be moved into the overlay directory and depend on svc:/system/identity:node
/etc/hosts needs to contain hostname mappings for IPv6, but there's a potential race condition when using NDP.

The paths for 1, 2, and 3 seem pretty straightforward, but I haven't thought of a good way to do the last one yet.

sjorge · 2016-01-07T07:56:47Z

For 4, can't sysevent also be used? Regenerate the hosts file everytime an ip gets added/removed?

Perhaps this should be moved to an issue and the PR closed, as this sounds a lot above my skill level.

sjorge · 2017-12-27T22:34:17Z

@melloc any progress on this?

sjorge added 5 commits October 31, 2015 10:43

allows sysinfo to discovered the ipv6 address

0542ce0

also print the variables instead of just setting them

7ddd3b4

make sure addrconf ipv6 addresses also show up in sysinfo

55cd57e

/etc/hosts should also contain ip6 addresses

deb5ac4

small correction for clarity

00ae623

rmustacc assigned melloc Oct 31, 2015

sjorge closed this Dec 25, 2015

sjorge deleted the more-gz-ipv6 branch December 25, 2015 10:54

sjorge restored the more-gz-ipv6 branch December 25, 2015 10:55

sjorge deleted the more-gz-ipv6 branch December 25, 2015 10:55

sjorge restored the more-gz-ipv6 branch January 4, 2016 14:16

sjorge reopened this Jan 4, 2016

melloc mentioned this pull request Jun 1, 2016

vmamd/sysinfo doesn't handle dhcp ip change #625

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve IPv6 support in globalzone #503

Improve IPv6 support in globalzone #503

sjorge commented Oct 31, 2015

rmustacc commented Oct 31, 2015

sjorge commented Oct 31, 2015

melloc commented Nov 4, 2015

sjorge commented Jan 4, 2016

melloc commented Jan 7, 2016

sjorge commented Jan 7, 2016

sjorge commented Dec 27, 2017

Improve IPv6 support in globalzone #503

Are you sure you want to change the base?

Improve IPv6 support in globalzone #503

Conversation

sjorge commented Oct 31, 2015

summary

detailed explanation of changes

test data

test image

rmustacc commented Oct 31, 2015

sjorge commented Oct 31, 2015

melloc commented Nov 4, 2015

sjorge commented Jan 4, 2016

melloc commented Jan 7, 2016

sjorge commented Jan 7, 2016

sjorge commented Dec 27, 2017