Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

4.33.0 breaks containers like MSSQL #7368

Open
Roberto-Neto opened this issue Jul 29, 2024 · 19 comments
Open

4.33.0 breaks containers like MSSQL #7368

Roberto-Neto opened this issue Jul 29, 2024 · 19 comments
Assignees
@andrea-reale
Labels
area/kernel Linux kernel bug version/4.33.0

Comments

@Roberto-Neto
Copy link

Description

With the new Update V.4.33., the Vms started but after a few seconds, they stopped.

Reproduce

1 - Go to Dashboard
2- Start VM

Expected behavior

No response

docker version

Server: Docker Desktop 4.33.0 (160616)
 Engine:
  Version:          27.1.1
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       cc13f95
  Built:            Tue Jul 23 19:57:14 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.19
  GitCommit:        2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc:
  Version:          1.7.19
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Capturing a dump of 10
2024-07-29 12:04:06 FAILED to capture a dump. Details in paldumper log.
2024-07-29 12:04:06 Executing: /opt/mssql/bin/handle-crash.sh with parameters
2024-07-29 12:04:06      handle-crash.sh
2024-07-29 12:04:06      /opt/mssql/bin/sqlservr
2024-07-29 12:04:06      10
2024-07-29 12:04:06      /opt/mssql/bin
2024-07-29 12:04:06      /var/opt/mssql/log/
2024-07-29 12:04:06      
2024-07-29 12:04:06      17b1af70-a127-43ea-8108-952937bef880
2024-07-29 12:04:06      50073ca4-e466-455e-b8d7-e9aad4518627
2024-07-29 12:04:06      
2024-07-29 12:04:06      
2024-07-29 12:04:06 
2024-07-29 12:04:06 Ubuntu 18.04.6 LTS
2024-07-29 12:04:07 Capturing core dump and information to /var/opt/mssql/log...
2024-07-29 12:04:12 Mon Jul 29 11:04:12 UTC 2024 Capturing program information
2024-07-29 12:04:13 Mon Jul 29 11:04:13 UTC 2024 Attempting to capture a dump with paldumper for pid 10
2024-07-29 12:04:14 WARNING: Capture attempt failure detected
2024-07-29 12:04:14 Attempting to capture a filtered dump with paldumper for pid 10
2024-07-29 12:04:14 WARNING: Attempt to capture dump failed.  Reference /var/opt/mssql/log/core.sqlservr.10.temp/log/paldumper-debug.log for details
2024-07-29 12:04:14 Mon Jul 29 11:04:14 UTC 2024 Attempting to capture a dump with gdb
2024-07-29 12:04:14 Mon Jul 29 11:04:14 UTC 2024 Captured a dump with gdb
2024-07-29 12:04:14 Mon Jul 29 11:04:14 UTC 2024 Capturing program binaries
2024-07-29 12:04:16 Mon Jul 29 11:04:16 UTC 2024 Not compressing the dump files, moving instead to: /var/opt/mssql/log/core.sqlservr.07_29_2024_11_04_06.10.d
2024-07-29 12:04:06 This program has encountered a fatal error and cannot continue running at Mon Jul 29 11:04:06 2024
2024-07-29 12:04:06 The following diagnostic information is available:
2024-07-29 12:04:06 
2024-07-29 12:04:06          Reason: 0x00000001
2024-07-29 12:04:06          Signal: SIGABRT - Aborted (6)
2024-07-29 12:04:06           Stack:
2024-07-29 12:04:06                  IP               Function
2024-07-29 12:04:06                  ---------------- --------------------------------------
2024-07-29 12:04:06                  000055555569e9ec <unknown>
2024-07-29 12:04:06                  000055555569e432 <unknown>
2024-07-29 12:04:06                  000055555569da41 <unknown>
2024-07-29 12:04:06                  00007ffffcd2af10 killpg+0x40
2024-07-29 12:04:06                  00007ffffcd2ae87 gsignal+0xc7
2024-07-29 12:04:06                  00007ffffcd2c7f1 abort+0x141
2024-07-29 12:04:06                  0000555555632672 <unknown>
2024-07-29 12:04:06                  00005555556b5204 <unknown>
2024-07-29 12:04:06                  00005555556e81f8 <unknown>
2024-07-29 12:04:06                  00005555556e7fda <unknown>
2024-07-29 12:04:06                  000055555563e4ea <unknown>
2024-07-29 12:04:06                  000055555563e13f <unknown>
2024-07-29 12:04:06         Process: 10 - sqlservr
2024-07-29 12:04:06          Thread: 109 (application thread 0x1a8)
2024-07-29 12:04:06     Instance Id: 17b1af70-a127-43ea-8108-952937bef880
2024-07-29 12:04:06        Crash Id: 50073ca4-e466-455e-b8d7-e9aad4518627
2024-07-29 12:04:06     Build stamp: 388f6c66c6dbc4da12ba35a003fcb9b5e0f65b6bc460dbf45ba9cf2562768e67
2024-07-29 12:04:06    Distribution: Ubuntu 18.04.6 LTS
2024-07-29 12:04:06      Processors: 14
2024-07-29 12:04:06    Total Memory: 16749948928 bytes
2024-07-29 12:04:06       Timestamp: Mon Jul 29 11:04:06 2024
2024-07-29 12:04:06      Last errno: 2
2024-07-29 12:04:06 Last errno text: No such file or directory
2024-07-29 12:04:12 dmesg: read kernel buffer failed: Operation not permitted
2024-07-29 12:04:12 /usr/bin/timeout: failed to run command '/bin/journalctl': No such file or directory
2024-07-29 12:04:12 /usr/bin/timeout: failed to run command '/bin/journalctl': No such file or directory

Diagnostics ID

2CC47E7A-01A1-406D-ABB8-461FE9D0521F/20240729111206

Additional Info

No response
@matthewdenaburgvc
Copy link

matthewdenaburgvc commented Jul 29, 2024 •
edited
Loading

I'm also experiencing this issue (as with a number of my colleagues).

Reproduction Steps

docker compose up -d

The docker-compose file uses an image based on https://hub.docker.com/r/microsoft/mssql-server's 2019-latest tag.

I've attached a (shortened) log file from the compose up command. There's a section where it repeats ~200 lines with a different /proc/10/task/#### value that I've removed for brevity.

diagnostics from the log file

Highlighting the diagnostics I received, as they are a bit more details than the above:

2024-07-29 14:45:42.65 spid30s     Starting up database 'master'.
This program has encountered a fatal error and cannot continue running at Mon Jul 29 14:45:42 2024
The following diagnostic information is available:
         Reason: 0x00000001
         Signal: SIGABRT - Aborted (6)
          Stack:
                 IP               Function
                 ---------------- --------------------------------------
                 00005555556a1c71 std::__1::bad_function_call::~bad_function_call()+0x29f81
                 00005555556a1636 std::__1::bad_function_call::~bad_function_call()+0x29946
                 00005555556a0bbf std::__1::bad_function_call::~bad_function_call()+0x28ecf
                 00007fffff107090 killpg+0x40
                 00007fffff10700b gsignal+0xcb
                 00007fffff0e6859 abort+0x12b
                 000055555563d426 std::__1::basic_stringbuf<char, std::__1::char_traits<char>, std::__1::allocator<char>>::str() const+0xd736
        Process: 10 - sqlservr
         Thread: 145 (application thread 0x238)
    Instance Id: a266616a-c91a-4a7f-a99d-f3dc16d3e830
       Crash Id: 495475a2-1f39-4e77-bfb8-c0f59d6d024e
    Build stamp: 51e51a4dfa4635df0a9847dbd15101717167582ccfb61f8ace2ffc3b0924d54d
   Distribution: Ubuntu 20.04.6 LTS
     Processors: 10
   Total Memory: 8219537408 bytes
      Timestamp: Mon Jul 29 14:45:42 2024
     Last errno: 2

output of docker version

Client: Docker Engine - Community
 Version:           27.1.1
 API version:       1.46
 Go version:        go1.22.5
 Git commit:        63125853e3
 Built:             Fri Jul 19 17:35:01 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.33.0 (160616)
 Engine:
  Version:          27.1.1
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       cc13f95
  Built:            Tue Jul 23 19:57:14 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.19
  GitCommit:        2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc:
  Version:          1.7.19
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

output of docker info

Client: Docker Engine - Community
 Version:    27.1.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1-desktop.1
    Path:     /Users/$USER/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1-desktop.1
    Path:     /Users/$USER/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     /Users/$USER/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.14
    Path:     /Users/$USER/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/$USER/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     /Users/$USER/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/$USER/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/$USER/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/$USER/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.11.0
    Path:     /Users/$USER/.docker/cli-plugins/docker-scout

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 11
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.0-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 7.655GiB
 Name: docker-desktop
 ID: 0bebe745-5bac-45b5-b335-162c8ad607cb
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/$USER/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

Let me know if there's any other information that would be helpful. I've downgraded to 4.32.0 for now.

@andreasernst
Copy link

Also with mcr.microsoft.com/mssql/server:2017-CU31-ubuntu-18.04 and macOS 14.5 (Apple M2 Max).

@Roberto-Neto
Copy link
Author

Roberto-Neto commented Jul 30, 2024 via email •
edited
Loading

@andreasernst
Copy link

Hi Roberto,

i have to use this versions, because our customer uses this version of MS SQL. I downgraded to 4.32.0.

Best regards
Andreas

@Roberto-Neto
Copy link
Author

Roberto-Neto commented Jul 30, 2024 via email

@kmc059000
Copy link

I'm experiencing this with mcr.microsoft.com/azure-sql-edge as well. Downgrading to 4.32.0 fixed the issue.

@zhedar
Copy link

zhedar commented Aug 5, 2024

Also had this issue last week running Sonoma 14.5 on M1 Max.
An old mcr.microsoft.com/azure-sql-edge image acted up and I wasn't able to access several management features anymore. Setting up a new container failed completely, downgraded to 4.32.0.

@alexwalkerinfo
Copy link

Got this issue with SQL 2017 image also. Downgraded for now.

@bchewy
Copy link

bchewy commented Aug 6, 2024 •
edited
Loading

Having this issue too 👍

Sonoma 14.6
Docker Desktop v.4.33.0

happening to mssql 2017 image.

@Roberto-Neto
Copy link
Author

Roberto-Neto commented Aug 6, 2024 via email

@zhedar
Copy link

zhedar commented Aug 6, 2024

@Roberto-Neto Maybe consider renaming the issue to something like '4.33.0 breaks containers like MSSQL'.

@Roberto-Neto Roberto-Neto changed the title New Update 4.33.0 With MAC OS Sequoia 4.33.0 breaks containers like MSSQL Aug 6, 2024
@caseyhelbling
Copy link

I'm also experiencing this issue with 4.33.0 breaking my mssql container.

2024-08-06 13:15:43 2024/08/06 18:15:43 [launchpadd] WARNING: Failed to load /var/opt/mssql/mssql.conf ini file with error open /var/opt/mssql/mssql.conf: no such file or directory
2024-08-06 13:15:43 2024/08/06 18:15:43 [launchpadd] INFO: DataDirectories =  /bin:/etc:/lib:/lib32:/lib64:/sbin:/usr/bin:/usr/include:/usr/lib:/usr/lib32:/usr/lib64:/usr/libexec/gcc:/usr/sbin:/usr/share:/var/lib:/opt/microsoft:/opt/mssql-extensibility:/opt/mssql/mlservices:/opt/mssql/lib/zulu-jre-11:/opt/mssql-tools
2024-08-06 13:15:43 2024/08/06 18:15:43 Drop permitted effective capabilities.
2024-08-06 13:15:43 2024/08/06 18:15:43 [launchpadd] INFO: Polybase remote hadoop bridge disabled
2024-08-06 13:15:43 2024/08/06 18:15:43 [launchpadd] INFO: Launchpadd is connecting to mssql on localhost:1431
2024-08-06 13:15:43 2024/08/06 18:15:43 [launchpadd] WARNING: Failed to connect to SQL because: dial tcp [::1]:1431: connect: connection refused, will reattempt connection.
2024-08-06 13:15:44 2024/08/06 18:15:44 [launchpadd] WARNING: Failed to connect to SQL because: dial tcp [::1]:1431: connect: connection refused, will reattempt connection.
2024-08-06 13:15:45 2024/08/06 18:15:45 [launchpadd] WARNING: Failed to connect to SQL because: dial tcp [::1]:1431: connect: connection refused, will reattempt connection.
2024-08-06 13:15:46 2024/08/06 18:15:46 [launchpadd] WARNING: Failed to connect to SQL because: dial tcp [::1]:1431: connect: connection refused, will reattempt connection.
2024-08-06 13:15:47 2024/08/06 18:15:47 [launchpadd] WARNING: Failed to connect to SQL because: dial tcp [::1]:1431: connect: connection refused, will reattempt connection.
2024-08-06 13:15:49 This program has encountered a fatal error and cannot continue running at Tue Aug  6 18:15:49 2024

@caseyhelbling
Copy link

How does one Downgrade to 4.32.0?

@zhedar
Copy link

zhedar commented Aug 6, 2024

@caseyhelbling
Simply delete the Docker.app from Applications folder and install 4.32.0 after downloading from the releases section: https://docs.docker.com/desktop/release-notes/

@andrea-reale andrea-reale mentioned this issue Aug 8, 2024
Open
@andrea-reale
Copy link

Hi there,

the source of the issue is a known bugin mssql-server-docker that causes mssql-server fail with the symptoms you are describing when running on Linux kernels 6.7 and later.

Docker Desktop 4.33 ships with Linux Kernel 6.10, while Docker Desktop 4.32 is on kernel 6.6.

Microsoft has fixed the bug on mcr.microsoft.com/mssql/server starting from tag 2022-CU11-ubuntu-22.04 (and 2022-latest), but the fix appears not to have been backported to 2017-* tags.

Workarounds are to either upgrade your mssql/server image to 2022-latest or downgrade Docker Desktop to 4.32 (in order to use an older linux kernel).

The bug also affects azure-sql-edge images, which run MSSQL. Unfortunately, the images currently published by Microsoft do not seem to include the fix to support kernels 6.7 and later.

In this case, the only possible workaround until Microsoft updates the images is to downgrade Docker Desktop to 4.32.

I have opened an issue with Microsoft to make them aware of the problem. Please, feel free to follow that issue to track progress.

@neonima neonima added area/kernel Linux kernel bug and removed status/triage labels Aug 8, 2024
@nuarhu
Copy link

nuarhu commented Aug 13, 2024 •
edited
Loading

azure-sql-edge does no longer support Apple Silicon, see https://techcommunity.microsoft.com/t5/sql-server-blog/azure-sql-edge-update-september-2023/ba-p/3930827

Good news: Docker for Mac includes virtualization for AMD images (under Settings > General), so it's possible to use the image mcr.microsoft.com/mssql/server:2022-latest which includes the fix for the newer linux kernels.

See also https://www.justinjbird.me/blog/2024/running-sql-server-on-macos-using-docker/

@uinfante
Copy link

uinfante commented Aug 23, 2024 •
edited
Loading

Same issue and logs here using SQL Edge image from an Apple M1 chip.
Downgrading to 4.32.0 worked, hope this will get fixed on the next release

Update: Moved to mcr.microsoft.com/mssql/server:2022-latest. Some config is needed when running on Docker Compose:

user: root  
platform: linux/amd64  # Required since the base image uses a different platform (Docker Virtualization)

@ibiwan
Copy link

ibiwan commented Sep 13, 2024

I was experiencing this issue exactly as described, and was able to get moving again (with DD 4.34.2) with the latest cumulative update from MS, on SQL Server 2019:
https://learn.microsoft.com/en-us/troubleshoot/sql/releases/linux/release-history-2019#CU28
FROM mcr.microsoft.com/mssql/server:2019-CU28-ubuntu-20.04

@parvini82
Copy link

I encountered the same issue with Docker on my M1 Pro, but downgrading to Docker 4.32 resolved the problem for me.

@mzieniukbw mzieniukbw mentioned this issue Sep 17, 2024
Merged
@withinfocus withinfocus mentioned this issue Sep 17, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andrea-reale andrea-reale

Labels
area/kernel Linux kernel bug version/4.33.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests