Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--dynamic-space-size handling on SBCL is broken #36

Open
rpgoldman opened this issue Mar 2, 2023 · 4 comments · May be fixed by #39
Open

--dynamic-space-size handling on SBCL is broken #36

rpgoldman opened this issue Mar 2, 2023 · 4 comments · May be fixed by #39

Comments

@rpgoldman
Copy link

I was having trouble with an application I built with this command line:

buildapp --dynamic-space-size 100000 --output strike-force-one --load game-player.lisp --entry 'cl-user::main' --dumpfile-copy 'strike-force-one-dumpfile.lisp' --logfile 'build-game-player.log'

When I ran the resulting application, it ran out of memory quickly. But when I ran sbcl with --dynamic-space-size 100000 and passed it a script that did the equivalent, it did not run out of memory.

Tracking it down, I found that run-program for SBCL, invoked in main was not getting the --dynamic-space-size 100000 argument:

(RUN-PROGRAM "sbcl"
             '("--noinform" "--disable-debugger" "--no-userinit" "--no-sysinit"
               "--load"
               "/home/rpg/projects/scepter/shop-mcts/strike-force-one/dumper-2SKVI5f7.lisp"))

Further debugging shows that buildapp does not see the --dynamic-space-size argument:

Constructing a dumper from (--output strike-force-one --load game-player.lisp
                            --entry cl-user::main --dumpfile-copy
                            strike-force-one-dumpfile.lisp --logfile
                            build-game-player.log)
After processing the command line arguments, we have the following dumper object:
#<BUILDAPP::DUMPER strike-force-one cl-user::main>
  [standard-object]

Slots with :INSTANCE allocation:
  PACKAGE                        = #:DUMPER124
  ACTIONS                        = ((:LOAD "game-player.lisp"))
  ENTRY                          = cl-user::main
  DISPATCHED-ENTRIES             = NIL
  ASDF-DIRECTIVES                = NIL
  LOAD-PATHS                     = NIL
  SBCL                           = "sbcl"
  CCL                            = "ccl"
  OUTPUT                         = "strike-force-one"
  LOGFILE                        = "build-game-player.log"
  DUMPFILE-COPY                  = "strike-force-one-dumpfile.lisp"
  CORE-ONLY                      = NIL
  COMPRESS-CORE                  = NIL
  DYNAMIC-SPACE-SIZE             = NIL

The "constructing a dumper" message above formats the arguments that are passed to command-line-dumper.

I conjecture that SBCL is stripping out this argument before the buildapp executable can see it.

I will attempt to verify this.
@rpgoldman
Copy link
Author

OK, I have verified the problem and here's what I did:

I replaced the command line parsing clause as follows:

          (:space-size ;; :dynamic-space-size
           (format t "Trying to parse dynamic-space-size from \"~a\" giving ~d~%"
                   value (parse-integer value))
           (setf (dynamic-space-size plan) (parse-integer value)))

and changed my command line to

buildapp --space-size 100000 --output strike-force-one --load game-player.lisp --entry 'cl-user::main' --dumpfile-copy 'strike-force-one-dumpfile.lisp' --logfile 'build-game-player.log'

and that gets the argument passed through:

After processing the command line arguments, we have the following dumper object:
#<BUILDAPP::DUMPER strike-force-one cl-user::main>
  [standard-object]

Slots with :INSTANCE allocation:
  PACKAGE                        = #:DUMPER124
  ACTIONS                        = ((:LOAD "game-player.lisp"))
  ENTRY                          = cl-user::main
  DISPATCHED-ENTRIES             = NIL
  ASDF-DIRECTIVES                = NIL
  LOAD-PATHS                     = NIL
  SBCL                           = "sbcl"
  CCL                            = "ccl"
  OUTPUT                         = "strike-force-one"
  LOGFILE                        = "build-game-player.log"
  DUMPFILE-COPY                  = "strike-force-one-dumpfile.lisp"
  CORE-ONLY                      = NIL
  COMPRESS-CORE                  = NIL
  DYNAMIC-SPACE-SIZE             = 100000
About to recursively start SBCL, dynamic-space-size is 100000
running this command to try to build the application:

(RUN-PROGRAM "sbcl"
             '("--dynamic-space-size" "100000" "--noinform"
               "--disable-debugger" "--no-userinit" "--no-sysinit" "--load"
               "/home/rpg/projects/scepter/shop-mcts/strike-force-one/dumper-2SKVI5f7.lisp"))

I'm not sure how to fix this problem. Renaming the argument as I did is inelegant but works. It would be nicer if we could keep SBCL from snarfing up that argument.

I suppose it's possible that ensuring that argument appears at the end would cause SBCL to not recognize it as a toplevel argument and pass it through. But that seems like a very brittle fix.

@PaulLockett
Copy link

Would love to see a resolution on this is causing an issue for dependents like pgloader. see here dimitri/pgloader#962

@rpgoldman rpgoldman linked a pull request Jan 11, 2024 that will close this issue
Open
@rpgoldman
Copy link
Author

@PaulLockett -- I have made a pull request for my proposed fix, which you could use for the present (you could just clone my repo instead of this one).

@PaulLockett
Copy link

@PaulLockett -- I have made a pull request for my proposed fix, which you could use for the present (you could just clone my repo instead of this one).

@rpgoldman hopefully your solution gets merged.

I ended up just building SBCL locally from source and setting the default to a higher value.

@rpgoldman rpgoldman mentioned this issue Jan 23, 2024
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Replace "dynamic-space-size" argument by "space-size". rpgoldman/buildapp
2 participants
@rpgoldman @PaulLockett