Sonora Can't Read A Large Library File

[nimapurkarimi]

my music library is 700 gb and Library.sndb file is 1.12 gb in size ,sonora says "The file "library" couldn't be opened." at the startup,and the app crashes instantly after scrolling down the artist list.

[enrique-ramirez]

Similar problem, not really sure if this is related or not (though I think it is).

My music library is around 200gb, 20k music files long. Sonora opens fine, and I can playback music pretty easily IF I double click on the "Albums" option on the sidebar. Mixes seem to work fine as well.

Problem arrives when I click on either Compilations or any artist name and playback that list. Things are all good and well; music plays and nothing is wrong until I attempt to scroll the sidebar, then Sonora crashes.

Funny enough, I can browse the Albums section's thumbnails and play any given album, and scroll around without problem. It seems to be a problem ONLY when selecting something else on the sidebar, playing any file and then scrolling (does not happen without playback).

Hope that makes sense. If not, I can do a quick video or a step by step guide on how to reproduce this issue.

[miguel-negrao]

I get the same problem, my library has 500GB

[chrbl]

I'm also having the same problem, and I thought it was because I updated to iTunes 11. I downgraded back properly to 10.7 and replaced all the necessary files, but it's still there. My library size is 100GB.

I deleted the Sonora library file and relaunched the app, and the songs seemed to load fine from iTunes until it gets around 75% through; the error message "The file "library" couldn't be opened." keeps popping up repeatedly until I force quit the app.

[macchina50]

My library is 310GB and has 21K songs.
I am getting the same types of crashes as enrique-ramirez with the Albums and Compilations.

Right now, however, my Library.sndb file is 256.6MB and Sonora IS working. When it broke, the library file was just above 260MB. I got it working again by building the library in Sonora 1.0 and then switching back to this version.

I think I am right on the edge, because I saw the error again today when I was cleaning up my library (it went away when I restarted the app.)

I wonder if the problem is related to the amount of total albums rather than overall size of a library. I have just under 2,000 separate albums.

[Zeriuno]

Just downloaded Sonora 2 beta and it looks like this bug is still there. Plenty of "The file "library" couldn't be opened." popup. (My library in an external drive is 598 GB).

[ebarnard]

Unfortunately I don't have a library large enough to reproduce this. However, this is probably due to album artwork being stored along with metadata in a flat sqlite database. Even if this is not the case there are a number of other problems associated with storing artwork in this way - or more accurately having sqlite databases of this size. It would be better to use a folder hierarchy and a referencing id in the database. indragiek, I'd be happy to implement this.

[indragiek]

That implementation would be much appreciated! My original reasoning for keeping the artwork in the sqlite db is that the image data once compressed was fairly small (~10-20KB per album cover) but this doesn't seem to scale too well to larger data sets.

[ebarnard]

Fair enough, although quite a few of my images are ~500k, I guess I just have high res artwork. I've written a first implementation which is available at https://github.com/edwardbarnard/Sonora in branch artwork-store. Zeriuno and the others in this thread, it would be great to know if this fixes your problem. indragiek, I was going to submit a pull request once I've improved caching a bit but if it fixes this issue then it would probably be best to do one sooner.

[macchina50]

Ed, thanks so much for working on this. I'm not sure if I am the best person to test this, because as I mentioned here #48 (comment) I have very little idea what I'm doing. I'm just figuring this out as I go. (For example, I had all sorts of compiling issues, but downgrading my copy of Xcode fixed this—and then I discovered I hadn't properly isolated your branch as my local repository—but I think I am getting the hang of it.)

In any event, I tried your implementation and still had the error. (There is still the chance I messed up somewhere obvious.)

I really do think you're onto something though. So everyone can better understand the problem, I'll try to explain my experience in greater detail. Hopefully this makes sense:

(1)I have 1,984 "albums." Some are large and some consist of a single track. Generally, when I import a new album into Sonora, I get an error that the library cannot be opened. This doesn't always crash the app, but it prevents me from accessing and browsing the "Albums" panel under "Collections." However, I can select individual artists. If I browse to an artist and delete an album (some times this requires deleting several albums at once), Sonora will snap into the once inaccessible full Album pane. After I end a session and restart Sonora, the deleted albums will be imported again from iTunes, and I will get the library read error once again.

(2)If I don't import any albums, Sonora is still somewhat unstable at 1,984 albums. The functions work up to a point. But I get intermittent problems like not being able to search and navigate to an album or artists. Sonora also breaks sometimes when I scroll through the Artist column or the main album page. And sometimes the album sorting doesn't match the selected sort button, switching back and forth between sort by popularity and age fixes this though.

(3) So if I want to add a new album to my Sonora library, I have to delete an album. I delete it from Sonora and iTunes. Then I add the new album to iTunes and import to Sonora. This procedure almost always works.

When I tried your implementation, I thought the problem was fixed, because I could initially import several albums with no problems. However, I imported a few more and I got the error again. I then deleted these albums in the way described in (3). I ran Sonora again and found the error had fully returned to (1).

[Another, and possibly related issue (maybe I ought to report it separately), occurs when I delete all albums for an Artist from iTunes but do not delete those albums from Sonora: When I load Sonora, the Albums deleted in iTunes will be gone, but the Artist will still appear in the search function. That is, the Artist will not be in the sidebar, but if I search the name of the Artist, I will get a result that says for example "Phil Collins 0 Albums" with no thumbnail. However, selecting that result does nothing - it's just an empty placeholder.]

In case it matters, I am using Mountain Lion.

[ebarnard]

Thanks for the info. It looks like this might be a bit more complicated than first thought. I haven't got internet access at the moment so I'll try and have a more helpful reply at the start of next week.