beatmaps

[jstma]

The first issue with beatmaps is that the two scripts have not been updated to python3. They generate an error when xrange is used. The midi-rubato script also has a type error. I managed to fix those things.

The second issue is the use of Transcribe! which I am not using. I don't know what it's output looks like or how to translate for what I am using.

There is an open source editor for certain rhythm games (Rocksmith being one) that I've used in the past. It's called Editor on Fire and binaries are available for windows and mac. Editor on Fire, or EoF, can be compiled for linux which is how I use it.

Creating a beatmap is not a 'press a key' affair with EoF. Instead you get a tempo map grid that you line up to the music. You set the tempo, then drag the tempo map into place where the notes begin. This is what it looks like for the start of the song Tick Tick Boom by The Hives.

Then as the tempo deviates you can drag the tempo map around. Then you can play back the tempo map as metronome clicks with the audio. This is important for rhythm games, but probably not so much for these scrolling videos. But I prefer this approach. And because I am mainly transcribing songs for drums, it's possible the songs already have a tempo map available. This particular song did, but it wasn't good so I couldn't use it. C’est la vie.

You'll notice that the tempo grid doesn't begin until much later in the song. This is because the transcription I have begins when the real music begins. So having extra measures before that point would be, well, pointless.

I had to create an eof2beatmap script based on the xsc2beatmap script as best as I could manage. EoF produces an xml file only when there are notes so I had to place at least one. Inside that xml file is a stanza for all the beats with timecodes of where they are in the audio file. They look like this:

  <ebeats count="447">
    <ebeat time="10.060" measure="1" />
    <ebeat time="10.502" />
    <ebeat time="10.945" />
    <ebeat time="11.387" />
    <ebeat time="11.830" measure="2" />
    <ebeat time="12.272" />
    <ebeat time="12.715" />
    <ebeat time="13.157" />

I used most of xsc2beatmap replacing the read_beats function with my own:

import xml.etree.ElementTree as ET

def read_beats(filename):
    mytree = ET.parse(filename)
    myroot = mytree.getroot()

    beat = 0
    measure = 0
    section = 0
    start_time = None

    label = "-"
    beats = []

    for x in myroot:
        if (x.tag == 'ebeats'):
            ebeats = x

    for x in ebeats:
        y = x.attrib
        if 'measure' in y.keys():
            m = y['measure']
            beat = 1
        else:
            beat += 1

        secs = float(y['time'])
        if start_time is None:
            start_time = secs

        beats.append((label, section, measure, beat, secs))

    return start_time, beats

and the resulting beatmap file looks like this:

-       0   0  1   0:00:10.060000 1 1  135.74660633
-       0   0  2   0:00:10.502000 1 1  135.44018059
-       0   0  3   0:00:10.945000 1 1  135.74660633
-       0   0  4   0:00:11.387000 1 1  135.44018059
-       0   0  1   0:00:11.830000 1 1  135.74660633
-       0   0  2   0:00:12.272000 1 1  135.44018059
-       0   0  3   0:00:12.715000 1 1  135.74660633
-       0   0  4   0:00:13.157000 1 1  135.44018059
-       0   0  1   0:00:13.600000 1 1  135.74660633
-       0   0  2   0:00:14.042000 1 1  135.44018059

I created a video with the beatmap and then used ffmpeg to insert the song audio with:

ffmpeg -i TickTickBoom.avi -i TickTickBoom.m4a -map 0:v:0 -map 1:a:0 TickTickBoom.mp4

Here's a snippet:

snip.mp4

The sheet music doesn't begin until approximately 12 seconds into the song, but scrolling starts immediately. If ly2video doesn't know about the beatmap until midi-rubato is run, there's no way to add silent frames to the start of the video. I'm not sure how to address this.

I don't know if I need to handle the anacrusis beats separately. In lilypond, the pickup notes are entered as a full bar with rests at the beginning:

\drummode {
		<<
			{ s2. cymca4 }
			\\
			{ r2 r8 bd8 sn4 }
		>>
}

So from the perspective of the score, beat 1 begins on the rest. If my tempo map begins on that same beat, I shouldn't need to care about special handling of the pickup beats. However, ly2video doesn't start the scrolling line on the rest. And IMHO it should. I don't know how to fix that either.

I know it's been a long time since you've touched the midi-rubato stuff @aspiers, but if you have any ideas on how to fix this I'm happy to explore them.