multi-line videos

[aspiers]

It would be cool to allow multiple systems of music in the same video frame, where the cursor line would only cover one system at any given moment in the video.

[jstma]

@aspiers I know you don't have much time to spend on this, but hopefully you can look this over and let me know what you think.

The existing solutions to multiple systems of music in the same video frame for lilypond are lacking. They do clever things like adding colour to noteheads, but the way the 'page' of music flips to change content is very distracting.

The best way for a horizontal scroll to work would be to leave the music stationary. Have the cursor 'paint' new music behind it as it travels along and have that music become more opaque as the time comes to play it. That would be pretty neat, but not something I want to work on.

I'd like to make lilypond videos that look like this:

Drumeo
https://github.com/aspiers/ly2video/assets/87273793/8745efba-7c3a-4f02-975f-2808c3ff3bc1

and

Drumistry
https://github.com/aspiers/ly2video/assets/87273793/7313c5d4-fddb-41a6-a9a4-f8bf8e465883

I didn't include the audio, but both examples are syncrhonised to music. In the Drumeo case, I think they sync their beatmap to a youtube video and that video plays while the sheet music scrolls.

The Drumeo style is great for larger screens and perhaps tablets in landscape mode. If the page width was reduced, it could probably work well for tablets in portrait mode too. I don't care about the note head colour. It doesn't add anything useful.

The Drumistry style rules for small screens. I just rest my phone on my music stand in landscape mode and it's better than juggling pieces paper.

Both of the above options are better than the horizontal scrolling you find in ly2video as it allows reading ahead quite easily. The Drumistry style is probably done by hand, snipping out parts of a pdf transcription (ie. screen grabs) and then dragging them into video editing software. With ly2video it could be quite automated once a beatmap is created.

The Drumistry style does not need a cursor. I just create a small page and lilypond does a good job of fitting music to those tiny pages. I get a number of png files as a result. So the work involved here is to stitch them together into a video with proper fading. I'm not sure what the best approach is for stitching the video together or how a beatmap could be used.

The Drumeo style can't be done without ly2video. But a similar technique would be to make a page that is really really long and lilypond should generate a single png. Then it's just a matter of drawing the cursor in the right place just as horizontal scrolling does. The tricky bit here though is to move the background page (or move the viewpoint) so that the next line of music moves to the top of the frame. The same beatmap should be able to make horizontal or vertical scrolling.

I'm going to give it a go.

[jstma]

There are a number of challenges that I didn't see coming. I'll post my progress here.

First is getting lilypond to play nice. I don't want to modify the music. I just want to create a new file, include the music.ly, and modify the behaviour. I believe this is the approach ly2video uses.

I set the paper size to be approx half the width of a letter size sheet. And adjusted the height to suit. This might be difficult to generalise. I suppose they could be passed in as parameters and then trial and error used on some intermediate output.

I also removed indentations and page numbers. One issue I had was that the beams would force some staves down more than others and this would cause jittering as I moved through the pictures in a photo viewer. I fixed that by moving the staves down more than any of the beams did. This is going to be a problem for a generalised solution.

#(set! paper-alist (cons '("one-line" . (cons (* 90 mm) (* 35 mm))) paper-alist))
\paper {
	#(set-paper-size "one-line")
	print-page-number = ##f
	% remove indentation
	indent = 0.0
	short-indent = 0.0

	% This hack keeps the staves in the same place.  Otherwise the beams move the staves down.
	top-system-spacing.basic-distance = #7
}

I turned off the time signature and clefs to make room for more notes. I removed the bar numbers too.

I needed to turn off repeats. I didn't want to unroll the music and I found the Repeat_acknowledge_engraver. Removing it seems to have removed the repeats, but I'm not entirely sure it did the right thing yet. Is there a better way to do it?

\layout {
	\context {
		\Score
		\omit TimeSignature
		\omit Clef
		\omit BarNumber
		\remove Repeat_acknowledge_engraver
	}
}

Lastly, I removed all header material that was defined in the original music.ly file.

\header {
	print-all-headers = ##f
	title = ##f
	composer = ##f
	copyright = ##f
	tagline = ##f
}

This produces png images like:

Then I moved on to creating some composite images with Pillow. I figure I should learn how this works before I go digging into ly2video code.

The first problem I ran into is sorting the list of lilypond images. See lilypond outputs files like image1.png and image10.png which don't sort properly without a 'natural sort' function that doesn't exist in vanilla python. There is a natsort library that can be used, but it means another dependency. I just manually renamed the first 9 files and carried on.

The next problem is transparency. If the main background is not white, then the pages of png music will show up white on top of the background. For my use case, this is fine, but it lacks sophistication. The python code I've got at the moment looks like this:

# Create our background canvas.  It will have a background and two
# regions for the sheet music lines.

# Todo: Make the size configurable and optionally use a background image
canvas = Image.new('RGBA', (1280,720), color='grey')

# Accounting for some margins, make each lilypond page 1240 x 320.
# Create two regions:
#   each region will be 1240 x 320
#   with 20px horizontal padding left and right
#   with 25px of vertical padding top and bottom
#   with 30px of vertical padding between regions
regionA = (20, 25)
regionB = (20, 375)
regionSize = (1240, 320)

opafull = 255
opasemi = 100

# First frame
k = 0
imageA = Image.open(images[k]).convert('RGBA').resize(regionSize)
k += 1
imageA.putalpha(opafull)
canvas.paste(imageA, regionA)

imageB = Image.open(images[k]).convert('RGBA').resize(regionSize)
k += 1
imageB.putalpha(opasemi)
canvas.paste(imageB, regionB)

canvas.show()  # regionA, 1.0, regionB, 0.5

# Next frame
imageA = Image.open(images[k]).convert('RGBA').resize(regionSize)
k += 1
imageA.putalpha(opasemi)
canvas.paste(imageA, regionA)

imageB.putalpha(opafull)
canvas.paste(imageB, regionB)

canvas.show()  # regionA, 0.5, regionB, 1.0

# Next frame
imageA.putalpha(opafull)
canvas.paste(imageA, regionA)

imageB = Image.open(images[k]).convert('RGBA').resize(regionSize)
k += 1
imageB.putalpha(opasemi)
canvas.paste(imageB, regionB)

canvas.show()  # regionA, 1.0, regionB, 0.5

Which produces

The images produced are low res. It's necessary to add -dresolution 300 to the lilypond command to get decent images after scaling to the video frame.

You'll have to excuse my very c-ish python code. I don't use python day to day.

So the basic idea works, at least for the sample of drum notation that I used.

Now I need timing events to drive the transitions. I imagine they can be a bit sloppy because there's no cursor or notehead animation. I don't know how to get this from lilypond because dump-spacetime-info.ly doesn't produce page numbers (and doesn't have to in it's original form). I don't know how many bars there are per page or I could use a modified beatmap that just has the downbeat of every measure. Suggestions on how to tackle it would be appreciated.

There will be multiple video frames of the same image. The old and new regions (ie. music) will disappear and reappear abruptly. To make the transitions less abrupt, the opacity change needs to step on it's way out and on it's way in. That's another sophistication that can wait until later.

Once the number of frames is known, they can be written to a container. Something like a 'motion png'. With 60 fps or something. A container that can easily be muxed with audio and turned into an mp4.

Also needed will be a separate 'title page' with the drum key, song title, etc. And a count in of some form. Stuff ly2video can already do IIRC.

[aspiers]

Thanks for the detailed write-up! You're right that unfortunately I have precious little time to devote to this. However a couple of things come to mind:

First is getting lilypond to play nice. I don't want to modify the music. I just want to create a new file, include the music.ly, and modify the behaviour. I believe this is the approach ly2video uses.

Yes that's right. Definitely take a look at the existing approach (in particular TROUBLE-SHOOTING.md) before going further, otherwise you might risk reinventing certain wheels.

I also removed indentations and page numbers.

I'm pretty sure writePaperHeader() already does this, or at least some of it.

I set the paper size to be approx half the width of a letter size sheet. And adjusted the height to suit. This might be difficult to generalise. I suppose they could be passed in as parameters and then trial and error used on some intermediate output.

I think the width only needs to be specified in order to determine how much to fit on one line, but the height can probably be infinite, or at least huge. This may make more sense after the explanations I give below.

One issue I had was that the beams would force some staves down more than others and this would cause jittering as I moved through the pictures in a photo viewer. I fixed that by moving the staves down more than any of the beams did. This is going to be a problem for a generalised solution.

With the cropping approaches I describe above, I don't see why that would be a problem.

I turned off the time signature and clefs to make room for more notes.

That could be an option but it wouldn't be appropriate as a general solution, especially outside drum music.

I removed the bar numbers too.

I needed to turn off repeats. I didn't want to unroll the music and I found the Repeat_acknowledge_engraver. Removing it seems to have removed the repeats, but I'm not entirely sure it did the right thing yet. Is there a better way to do it?

I'm not sure but sounds reasonable. Best ask on the Lilypond mailing list about these things.

Lastly, I removed all header material that was defined in the original music.ly file.

ly2video already does this in sanitiseLy().

This produces png images like:

You pasted one image per line here, which seems to imply that you are maybe using a separate .ly file for each line of music? I would definitely recommend to avoid that as it will become horrendously slow if you need to run Lilypond once for each line! Instead, I suggest getting lilypond to generate as few pages as possible, and then cropping a single image of a page with different offsets to generate all the video frames you need.

In fact, this is how ly2video already works, so if you do it a different way, I'm not sure it's feasible to merge that new approach into ly2video.

For the Drumeo style of video, the only changes you need to make are:

• to get rid of page-breaking = #ly:one-line-breaking
• to change the cropping algorithm so that each video frame contains more than one line of music (this could get a bit complex though)

For the Drumistry style, it's similar but you'd need to crop each line of music separately and then stitch them together to make video frames containing two lines at once.

Worth noting that ly2video already auto-detects the croppable area, but this only works by assuming the image has a single system. For multiline cropping it would have to be changed to start at the y position for the current grob, and then search upwards and downwards for the top and bottom of the crop rectangle respectively.

Then I moved on to creating some composite images with Pillow. I figure I should learn how this works before I go digging into ly2video code.

The first problem I ran into is sorting the list of lilypond images. See lilypond outputs files like image1.png and image10.png which don't sort properly without a 'natural sort' function that doesn't exist in vanilla python. There is a natsort library that can be used, but it means another dependency. I just manually renamed the first 9 files and carried on.

This should be a non-issue if you follow my recommended approach of only generating a single image from Lilypond.

Now I need timing events to drive the transitions. I imagine they can be a bit sloppy because there's no cursor or notehead animation. I don't know how to get this from lilypond because dump-spacetime-info.ly doesn't produce page numbers (and doesn't have to in it's original form).

As per above I think you should aim for generating a single huge page. However if that's not possible, the dumper could probably be modified to include page numbers pretty easily. People on the Lilypond mailing list could help with that.

I don't know how many bars there are per page or I could use a modified beatmap that just has the downbeat of every measure. Suggestions on how to tackle it would be appreciated.

Ideally there's only one page. But anyway, timing events come from the MIDI file not from bar numbers and page numbers (as explained in TROUBLE-SHOOTING.md, so I'm not sure why you are worrying about this.

There will be multiple video frames of the same image.

ly2video already generates frames for the entire video from the same image, so this is a given.

Once the number of frames is known, they can be written to a container. Something like a 'motion png'. With 60 fps or something. A container that can easily be muxed with audio and turned into an mp4.

It sounds like you are describing what already happens. ly2video generates each frame as a .png:

ly2video/ly2video/video.py

Line 523 in 41364ad

self.__fileName = "%s%09.4f.png" % (self.__fileNamePrefix,0.0)

and then stitches them together into a video:

ly2video/ly2video/cli.py

Lines 1160 to 1176 in 41364ad

	def generateNotesVideo(ffmpeg, fps, quality, frames, wavPath):
	progress("Generating video with animated notation\n")
	notesPath = tmpPath("notes.mpg")
	cmd = [
	ffmpeg,
	"-nostdin",
	"-f", "image2pipe",
	"-r", str(fps),
	"-i", "-",
	"-i", wavPath,
	"-q:v", quality,
	"-f", "avi",
	notesPath
	]
	safeRunInput(cmd, inputs=(imageToBytes(frame) for frame in frames), exitcode=15)
	output_divider_line()
	return notesPath

Also needed will be a separate 'title page' with the drum key, song title, etc. And a count in of some form. Stuff ly2video can already do IIRC.

Correct.

[jstma]

Thanks for the detailed write-up! You're right that unfortunately I have precious little time to devote to this.

I really appreciate the response.

I also removed indentations and page numbers.

I'm pretty sure writePaperHeader() already does this, or at least some of it.

For this particular format, I need to be more aggressive in removal, but thanks for pointing out where that code is.

I set the paper size to be approx half the width of a letter size sheet. And adjusted the height to suit. This might be difficult to generalise. I suppose they could be passed in as parameters and then trial and error used on some intermediate output.

I think the width only needs to be specified in order to determine how much to fit on one line, but the height can probably be infinite, or at least huge. This may make more sense after the explanations I give below.

Perhaps.

One issue I had was that the beams would force some staves down more than others and this would cause jittering as I moved through the pictures in a photo viewer. I fixed that by moving the staves down more than any of the beams did. This is going to be a problem for a generalised solution.

With the cropping approaches I describe above, I don't see why that would be a problem.

Except that there's no need to crop. I'll explain below.

I turned off the time signature and clefs to make room for more notes.

That could be an option but it wouldn't be appropriate as a general solution, especially outside drum music.

Agree 100%.

I removed the bar numbers too.
I needed to turn off repeats. I didn't want to unroll the music and I found the Repeat_acknowledge_engraver. Removing it seems to have removed the repeats, but I'm not entirely sure it did the right thing yet. Is there a better way to do it?

I'm not sure but sounds reasonable. Best ask on the Lilypond mailing list about these things.

Removing the Repeat_acknowledge_engraver is wrong. And after searching I found a thread that explains the only way to do this is to unroll the music in the score. And that's what happens in sanitised.ly. Though I haven't yet figured out how santised.ly is produced.

You pasted one image per line here, which seems to imply that you are maybe using a separate .ly file for each line of music? I would definitely recommend to avoid that as it will become horrendously slow if you need to run Lilypond once for each line! Instead, I suggest getting lilypond to generate as few pages as possible, and then cropping a single image of a page with different offsets to generate all the video frames you need.

Lilypond will generate a png for every page it produces. I've set the page to be very small and lilypond generates a png for each small page as a result. So I get multiple snippets of music all framed nicely (no cropping required) as long as the page size is correct. I only run the lilypond command once:

$ lilypond --png video.ly

Where video.ly has the proper overrides that will eventually find their way into ly2video. For the piece of music I'm using as an example, there are 46 pngs produced. Each png is a correctly sized, cropped, piece of music ready to scale and place in the video.

In fact, this is how ly2video already works, so if you do it a different way, I'm not sure it's feasible to merge that new approach into ly2video.

I'm thinking a good deal of code can be reused. We'll see how well it integrates into ly2video.

For the Drumeo style of video, the only changes you need to make are:

* to get rid of `page-breaking = #ly:one-line-breaking`

* to change the cropping algorithm so that each video frame contains more than one line of music (this could get a bit complex though)

But then you lose the formatting that lilypond already produces. If it's a single system, then lilypond isn't laying them out and you'll get the equivalent of ragged right with cropping. I think.

What I did for this style was just make a really really long page that could hold all the staves. Then lilypond produces one giant png with all the staves in a single image. The width of that png can be scaled to the video width and then it shouldn't be too hard to 'grab' just a portion of that image for the viewable height.

That's the easy way to implement the scrolling, but I haven't looked into what will be required to draw the cursor.

For the Drumistry style, it's similar but you'd need to crop each line of music separately and then stitch them together to make video frames containing two lines at once.

Same argument here. Lilypond has already spaced out the systems so that they line up at the end of the page. When I take care of the indents/margins and let lilypond create the pages, all the staves look the same as I flip through them and the outer bar lines stay in the same place.

With lilypond handling the width of the system, you can reduce or enlarge the page width to accommodate the video you are making and get a nicely formatted set of staves.

Then I moved on to creating some composite images with Pillow. I figure I should learn how this works before I go digging into ly2video code.
The first problem I ran into is sorting the list of lilypond images. See lilypond outputs files like image1.png and image10.png which don't sort properly without a 'natural sort' function that doesn't exist in vanilla python. There is a natsort library that can be used, but it means another dependency. I just manually renamed the first 9 files and carried on.

This should be a non-issue if you follow my recommended approach of only generating a single image from Lilypond.

Seems to be an easy problem to solve. I'm not convinced the single system can result in the display I want.

Now I need timing events to drive the transitions. I imagine they can be a bit sloppy because there's no cursor or notehead animation. I don't know how to get this from lilypond because dump-spacetime-info.ly doesn't produce page numbers (and doesn't have to in it's original form).

As per above I think you should aim for generating a single huge page. However if that's not possible, the dumper could probably be modified to include page numbers pretty easily. People on the Lilypond mailing list could help with that.

Right, I'll try to understand that code and ask the mailing list for help.

I don't know how many bars there are per page or I could use a modified beatmap that just has the downbeat of every measure. Suggestions on how to tackle it would be appreciated.

Ideally there's only one page. But anyway, timing events come from the MIDI file not from bar numbers and page numbers (as explained in TROUBLE-SHOOTING.md, so I'm not sure why you are worrying about this.

The MIDI file only contains note events. How are the rests handled? I guess I need to look at the relevant code and understand the whole approach.

For the Drumeo style, how will the cursor know when to move down to the next line, if it doesn't know where the bar lines are? That's what I'm worrying about. Ideally the cursor disappears at one barline and reappears at another.

Once the number of frames is known, they can be written to a container. Something like a 'motion png'. With 60 fps or something. A container that can easily be muxed with audio and turned into an mp4.

It sounds like you are describing what already happens. ly2video generates each frame as a .png:
and then stitches them together into a video:

Not quite. There's no need to create a png file for every frame. They could just be stored in a video file container as they are created. Effectively just a silent video created without ffmpeg. ffmpeg would still be needed to mux the audio and video files.

It's probably not worth the effort to do it the way I just described. Having a silent video would be nice to mux with say an audio file of the recorded song and also with just a metronome. It's not a big deal to run ly2video twice though.

Also needed will be a separate 'title page' with the drum key, song title, etc. And a count in of some form. Stuff ly2video can already do IIRC.

Correct.

Perhaps it's time to close this issue:
#45

[jstma]

I had a look at how santised.ly is created. I was thinking there was a lilypond preprocess command, but that's not the case.

Instead the ly file is opened and then parsed by ly2video to create sanitised.ly. Using regular expressions, it modifies what it read in from the lilypond file and writes that to santised.ly. It looks for the score block and adds an unfoldRepeats, which is the only way to do that.

I was looking for a way to add \paper and \layout blocks to override certain settings the way I wanted them at the end of the lilypond file. So my video.ly starts with this:

\include "./TickTickBoom.ly"

And below that are /header and /paper blocks.

TickTickBoom.ly comes from here:

https://bitbucket.org/sinbad/drum-music/src/master/

It's not mine, it's old, and it doesn't contain a score block. That means ly2video won't like it. convert-ly does not wrap the DrumStaff in a score block. I don't know how many scores are like this. Lilypond processes it just fine though and knows how to add the score block.

The usage in the readme says:

You must ensure that your .ly input file contains both \layout { } and \midi commands, which ensure that valid .midi and .png files are generated when it is run through lilypond --png.

to have a midi block you need to have it set up with a score block, that makes score blocks implicitly required:

\score {
  … music …
  \layout { }
  \midi { }
}

In my view of the world, it would be good to put the music into a variable in it's own file. It's how I organise my own stuff. That eventually looks like:

\include TickTickBoomWithVariable.ly

\score {
	\unfoldRepeats { \theMusic }

	\layout {
		\context {
			\Score
			\omit TimeSignature
			\omit Clef
			\omit BarNumber
		}
	}

	\midi { }
}

#(set! paper-alist (cons '("one-line" . (cons (* 90 mm) (* 26 mm))) paper-alist))
\paper {
	#(set-paper-size "one-line")
	print-page-number = ##f
	% remove indentation
	indent = 0.0
	short-indent = 0.0
	top-system-spacing.basic-distance = #7
}

\header {
	print-all-headers = ##f
	title = ##f
	composer = ##f
	copyright = ##f
	tagline = ##f
}

I think ly2video will process that properly, even if it sanitises. Maybe. Though the page size might interfere with generating a single staff png. The easiest thing to get a horizontal scrolling video is:

\include TickTickBoomWithVariable.ly

\score {
	\theMusic
	\layout {
	}
	\midi {
	}
}

Sanitising is not required with the first example. The easiest thing to do might be to add a comment:

% NOSANITISE

And parse for that string. Maybe make it the first line and the sanitiser could abort any regex. Maybe even the scrolling mode could be parsed this way. Though I don't know if it's advantageous to do so.

At this point I'm just blathering on about what I've learned tonight.

[jstma]

As I keep picking at this:

\include TickTickBoomWithVariable.ly

Also includes the header with the title. ly2video won't see it. This is how I have some of my own files organised. I can't be alone here.

A simple solution would be to copy/paste the header from TickTickBoomWithVariable.ly to video.ly. Then ly2video could pick up the right details before it scrubs them, or they could be entered as comments at the top of the file. ly2video would need to learn how to read them there. There's probably no advantage to that. It probably wouldn't be too hard for ly2video to scan the includes for title and author details too.

I'm creating a separate file for the video score because my printed score might be way more complicated. It makes sense to put the right content in the video score instead of having ly2video santise a pdf score. In that case, I could create templates like: video-horizontal-scroll.ly, video-vertical-scroll.ly, and video-paged.ly. This would make it easy to include the music from elsewhere and ly2video could learn to work with them as additional behaviour. And ly2video could create those templates (suitably self documented) for the user whenever required.

Any layout tweaks that become common to both the pdf score and the scrolling score would have to be put in a common include. I don't know how onerous that becomes in return for greater flexibility.

[jstma]

@aspiers I hate to pester, but I need your input. I don't want to waste time making changes that won't be accepted.

I've been asking on the mailing list about certain things and I will summarise what I've learned here.

I'd like to get rid of the santiseLy() function. I feel that it's not the right way to go about things.

The way ly2video inserts \unfoldRepeats is technically incorrect as was pointed out to me:

ly2video or not, you need to unfold repeats as soon as you want correct MIDI output. I agree that requiring to do it explicitly is not very user-friendly, but auto-adding \unfoldRepeats sounds like solving the problem at the wrong level. Also, what ly2video does is not always correct since \score { \midi { ... } \music } is valid but \score { \unfoldRepeats \midi { ... } \music } is not — it would need to be \score { \midi { ... } \unfoldRepeats \music }.

There is a way to unroll repeats from an included file.

#(set! toplevel-music-functions (cons unfoldRepeats toplevel-music-functions))

So the major reason to scrape/rewrite the user's file is probably gone.

With ly:one-line-breaking breaks are ignored.

With ly:one-page-breaking, there won't be page breaks, but the manual line breaks will be preserved which is probably what the user wants anyway. The idea would be to scroll the 'pdf like' score.

The issue I have is with a reduced page width where the user's custom \breaks will be a problem. For this case:

break = {}
pageBreak = {}

added to dump-spacetime.ly would solve that.

In scores where the \header block is \include'd from the main song.ly, ly2video can't parse the \header directly. There is a way to do it though that's more robust. lilypond has a --header option that will output fields to files.

lilypond --header title --header author song.ly

Will produce files called song.title and song.author.

The only missing piece is:

        # ignore these commands
        if line.find("#(set-global-staff-size") != -1 or \
           line.find("\\bookOutputName") != -1:
            pass

The staff size can maybe be addressed with:

#(set-global-staff-size 20) % restore to default size

added to the end of the input file. I'm sure there's an actual symbol for that default staff size, but I haven't found it. Actually it's not even necessary because writePaperHeader() already does it.

 fFile.write("#(set-global-staff-size %d)\n" % GLOBAL_STAFF_SIZE)

\bookOutputName is troublesome though. I don't know what it means for ly2video to process a book. It was present in the initial import of this repository so I don't know when or why it was added.

If the functionality of santiseLy() can be replaced by inline code that disables or overrides certain behaviour, I think it's worthwhile to do.

The idea would be to concatenate three files: dump-spacetime-improved.ly song.ly video-type.ly. And eliminate the santiseLy() function. How do you feel about eliminating santiseLy()?

[jstma]

For files that contain \book and \bookOutputName, how should ly2video proceed? Here's an example:

\header {
  title = "File title"
  midititle = "midi title"
}
music = \relative c' {
  c1
}
\book {
  \bookOutputName "pdf_file_only"
  \score {
    \new Staff \music
  \layout {}
  }
}
\book {
  \bookOutputName "midi_file_only"
  \score {
    \new Staff \music
  \midi {}
  }
}

[aspiers]

I'm afraid I'm getting quite overwhelmed by all these comments - with my very limited free time, I'm going to struggle to read them all.

I definitely don't mean to discourage you from being verbose, since I think that's very good practice and also allows other people to follow along and maybe even help you out if we are lucky. But if you have specific questions for me, please could you isolate those into a concise list? Or actually even better: submit a short issue for each question (e.g. with a Q: prefix in the title) and the question label applied? That way I can clearly see how to unblock you without having to follow all the details of what you're working on.

[jstma]

I get what you're saying which is why I only @aspiers for the one I'd like you to read ;)

In this case, the question is simply

"Do you oppose removing santiseLy() if the functionality can be done in another way?"

Those other ways are listed in the previous comment, if you need to see the reasoning:

#41 (comment)

[aspiers]

I see. I'm definitely not strongly wedded to keeping hacky code which attempts to parse and munge Lilypond code through regular expressions; in fact, I'm already convinced that's less than ideal.

I had previously assumed that the only way to make this robust would be via some more complete Lilypond parser which understands Lilypond grammar and syntax. But if you think you can reimplement all the existing functionality (or some reasonable approximation thereof) via the tricks you describe (e.g. concatenating multiple .ly files together, and using CLI options to lilypond) then I'd say go for it. My main concern would be to avoid dropping existing functionality which other people may be relying on.

[jstma]

Excellent, I shall continue. No need to read this one.

There may be another way to get the header data from lilypond. This is probably not working code from the mailing list archive
https://lists.gnu.org/archive/html/lilypond-user/2019-10/msg00104.html

get-layout-header = #(define-scheme-function (score)
                       (ly:score?)
                       (let* ((output-defs (ly:score-output-defs score))
                              (header (ly:module->alist (ly:score-header score)))
                              (title (assoc-get 'title header))
                              (composer (assoc-get 'composer header))
                              ;; Look for sym in all output definitions
                              ;; is there a way to determine of which type an output definition is?
                              (get-sym (lambda (sym)
                                         (car (filter (lambda (el)
                                                        (not (null? el)))
                                                      (map (lambda (output-def)
                                                             (ly:output-def-lookup output-def sym))
                                                        output-defs)))))
                              (indent (get-sym 'indent)))
                         (display indent)
                         (newline)
                         (display title)
                         (newline)
                         (display composer)))

To determine which type an output definition is, this can be used.

(ly:output-def-lookup output-def 'is-layout #f)

The advantage is we can send this to stdout where ly2video can parse it rather than opening and reading files. However, is it worth the effort of debugging that scheme code when opening a file is not difficult at all...