Library scan: log summary and show popup #13427

ronso0 · 2024-07-01T11:51:00Z

This emphasizes the logged scan summary and allows to show a respective popup, both for automatic scan at startup and manual scan.

Fixes #10720

For me, this is a step towards being able to manually find successors for moved missing tracks, may also be helpful for the New Tracks filter/feature.

info [LibraryScanner 1] ----------------------------------------------
info [LibraryScanner 1] Library scan finished after 963 ms
info [LibraryScanner 1]  1235 unchanged directories
info [LibraryScanner 1]  197 changed/added directories
info [LibraryScanner 1]  0 tracks verified from changed/added directories
info [LibraryScanner 1]  0 new tracks
info [LibraryScanner 1]  0 moved tracks
info [LibraryScanner 1]  0 missing tracks
info [LibraryScanner 1] ----------------------------------------------

if nothing changed it displays only duration and ""No changes were detected."

TODO
~~Find an alternative for int LibraryScanner::cleanUpScan() to get the number of relocated tracks.~~ ✔️

ronso0 · 2024-07-03T22:21:58Z

Find an alternative for int LibraryScanner::cleanUpScan() to get the number of relocated tracks.

Simply storing that in an LibraryScanner int member seems fine to me (reset when scan is started, no need to bother scannerGlobal with this).
Wdyt?

JoergAtGithub · 2024-07-04T07:24:15Z

show a respective popup, both for automatic scan at startup and manual scan

It's a nice addition to get a feedback, when I triggered the scan amnually. But I would no like to get this pop-up, when I start Mixxx.

ronso0 · 2024-07-04T08:01:45Z

Funny thing is, I moved the connection to CoreServices so we can always get the popup.

Why do you consider the info popup disturbing / not helpful for the autorun scan during startup?

ronso0 · 2024-07-18T17:14:17Z

src/library/dao/trackdao.cpp

-    } // namespace
+    return matchLength;
+}
+} // namespace


This is only fixing indentation, pre-commit

ronso0 · 2024-07-24T12:14:26Z

friendly ping @JoergAtGithub

Why do you consider the info popup disturbing / not helpful for the autorun scan during startup?

We could make the report optional when scanning during startup, but I'd rather avoid that sort of complification / extra pref option.

JoergAtGithub · 2024-07-24T17:46:59Z

In my opinion a dialog should only appear, if:

it's the response to a users action (like clicking on rescan library here)
an error occurs

but on startup it is always anoying to click dialogs away. Normal information should be visualize unobtrusive in the UI, in a way that does not require user action to continue.

ronso0 · 2024-07-25T16:10:06Z

The last commit disables the report/summary dialog for the auto scan run during startup.

ronso0 · 2024-07-25T16:11:03Z

I'm a bit puzzled why the "changed/added" directory count is not 0 repeatedly even if the dirs haven't been modified.
(no regression btw)

ronso0 · 2024-07-29T09:37:36Z

I'm a bit puzzled why the "changed/added" directory count is not 0 repeatedly even if the dirs haven't been modified. (no regression btw)

Turns out it's just the confusing naming scanned vs. changed/added.
"scanned directories" are the directories where no previous hash exists (??) or prev hash != new hash (even though there was absolutely nothing even touching the dirs in between the scans 🤷 )

I adjusted the dialog and the logged lines. All good now!

JoergAtGithub · 2024-07-29T18:12:18Z

Is it the intended behavior, that I get negative values, if I: Delete a file -> Rescan -> Restore the file -> Rescan

It would make sense to print the scan results into the log output at startup, if Mixxx is started in developer mode.

ronso0 · 2024-07-29T23:09:26Z

Is it the intended behavior, that I get negative values, if I: Delete a file -> Rescan -> Restore the file -> Rescan

Thanks for your thourough testing!
This is not expected, but the numerical result of the TrackDAO/Scanner methods we had at our disposal:

  previously existing track locations
+ new tracks (= added tracks - relocated tracks)
- existing track locations

In order to reliably report rediscovered tracks I'll add a method to TrackDAO to get missing track locations.

ronso0 · 2024-07-29T23:55:14Z

The report now looks like this

We could calculate new missing tracks but I don't think that's worth it.

ronso0 · 2024-07-29T23:57:46Z

It would make sense to print the scan results into the log output at startup, if Mixxx is started in developer mode.

The report is always logged, just the dialog is now suppressed for the scan during startup.

daschuer · 2024-07-30T13:30:31Z

I have not tested this yet and I have scanning during start up disabled so I probably not fully qualified to comment. But here some thoughts:

I think we have a scanning dialog that pops up when the scan takes longer. There is a risk that mixxx is locked when using the library during scanning. So I would argue the user is disturbed anyway by the startup scan.

On the other hand I can confirm that the scan during startup is annoying and a "useless" dialog would be more annoying.

How about showing the dialog only if new tracks are found or some are missing in the startup case?
For my use case it would be also nice to put new tracks in an "Incoming" crate.

What do you think?

JoergAtGithub · 2024-07-30T18:03:50Z

I think the currently implemented approach with printing always to the log, but showing the dialog only when the user manually triggers the library scan is fine.

ronso0 · 2024-07-30T20:53:12Z

I think we have a scanning dialog that pops up when the scan takes longer. There is a risk that mixxx is locked when using the library during scanning. So I would argue the user is disturbed anyway by the startup scan.

Exactly, the scan progress dialog pops up if the scan takes longer than 2s.
(fwiw during testing with auto-scan on startup I see the -empty- dialog artefact which never disappears, have to close it #12283)
Yes, the scan may be disturbing if one has new tracks on each start (many of them or a slow computer). And the scan during start with a fresh profile (new music dirs with lots of files) can lead to #11738 (which is why I have a delayed scan fix stashed).

ronso0 · 2024-07-30T20:53:51Z

How about showing the dialog only if new tracks are found or some are missing in the startup case?

Yes, that wold make sense. new tracks, new missing tracks, track rediscovered.

ronso0 · 2024-07-30T20:57:20Z

For my use case it would be also nice to put new tracks in an "Incoming" crate.

The "New tracks" feature / crate would make sense. How / when tracks are removed needs to be discussed. Automatically when a rating is set? And/or when they're sorted into playlists or crates?

I appreciate the idea, though it's unrelated to this UX fix/feature. I remember we briefly discussed this recently, is there already an Issue for tracking it?

ronso0 · 2024-07-30T21:12:01Z

I think the report can be optimized. IMO this is sufficient:

Scan took 1181 ms.

1 new tracks found
0 moved tracks detected
1 tracks are missing
0 tracks have been rediscovered

16066 tracks in total

ToDo

count new missing tracks (change discovered during this scan)

daschuer · 2024-07-31T09:33:07Z

The "New tracks" feature ...

My current use case would work perfectly if the tracks stick in the "Incoming" crate, just like in my email in box.

Normally I sort them from there and than weeks later I delete them they are no longer considered as "New".

daschuer · 2024-07-31T09:35:21Z

We have this one:
#13403

I will file new one for the scanner part.

daschuer · 2024-07-31T09:40:37Z

#13526

ronso0 · 2024-07-31T14:10:20Z

The report dialog now only displays relevant information:
new / moved / new missing / rediscovered tracks only if there are any.

I'll work on a fixup to show the report dialog during startup only if changes have been detected.

…nges

ronso0 · 2024-08-29T15:37:56Z

I'll work on a fixup to show the report dialog during startup only if changes have been detected.

done.

… tracks

daschuer

Some comments. Not yet tested.

daschuer · 2024-10-30T04:44:31Z

src/library/dao/trackdao.cpp

+    while (query.next()) {
+        locations.insert(query.value(locationColumn).toString());
+    }
+    return locations;


you may put this lower part into a common function taking a query, and returning the QSet.

thanks, done.

daschuer · 2024-10-30T05:08:49Z

src/library/scanner/libraryscanner.cpp

+            numRediscoveredTracks++;
+        }
+    }
+    const auto missingTracks = m_trackDao.getAllMissingTrackLocations();


I have no idea how this performs and whether optimization is beneficial:
An idea for optimizion would be to move the compare loop into getAllMissingTrackLocations and get around the temporary QSet(). This will require a lot more compares though.

…g tracks

github-actions bot added library scanner labels Jul 1, 2024

ronso0 changed the title ~~Lib scan summary etc~~ Library scan: log summary and show popup Jul 3, 2024

Library: improve logged scan summary, also log new and missing tracks

da82030

ronso0 force-pushed the lib-scan-summary-etc branch from 35598a0 to 289c57f Compare July 3, 2024 22:21

ronso0 marked this pull request as ready for review July 4, 2024 08:01

ronso0 commented Jul 18, 2024

View reviewed changes

src/library/dao/trackdao.cpp

} // namespace

return matchLength;

}

} // namespace

Copy link

Member Author

ronso0 Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only fixing indentation, pre-commit

ronso0 added 2 commits July 29, 2024 11:34

Library: show scan summary popup

db2e6fa

Library scan: don't show summary when run during startup

1cd81d7

ronso0 force-pushed the lib-scan-summary-etc branch from 41f64ac to 1cd81d7 Compare July 29, 2024 09:34

TrackDAO: add getAllMissingTrackLocations()

b196c4e

ronso0 force-pushed the lib-scan-summary-etc branch from 2c254d5 to 1cd547c Compare July 30, 2024 21:01

Library scan: report rediscovered tracks, fix report of missing tracks

f8d7fd9

ronso0 force-pushed the lib-scan-summary-etc branch from 1cd547c to f8d7fd9 Compare July 31, 2024 14:09

Library scan: don't show report dialog when auto-scan detected no cha…

c2554f2

…nges

ronso0 force-pushed the lib-scan-summary-etc branch from 554c965 to c2554f2 Compare August 29, 2024 15:36

fixup! Library: improve logged scan summary, also log new and missing…

5141b18

… tracks

ronso0 force-pushed the lib-scan-summary-etc branch from 9e45b1c to 5141b18 Compare October 29, 2024 21:24

daschuer reviewed Oct 30, 2024

View reviewed changes

ronso0 added 2 commits November 4, 2024 10:20

TrackDAO: add helper collectTrackLocations(), use FwdSqlQuery

cc7d681

TrackDAO/PlaylistDAO: use table columns constants

9e90d9a

ronso0 force-pushed the lib-scan-summary-etc branch from 0579a2b to ba77efa Compare November 5, 2024 15:04

fixup! Library scan: report rediscovered tracks, fix report of missin…

47eb4f5

…g tracks

ronso0 force-pushed the lib-scan-summary-etc branch from ba77efa to 47eb4f5 Compare November 7, 2024 23:27

Merge remote-tracking branch 'mixxx/main' into lib-scan-summary-etc

59520b7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Library scan: log summary and show popup #13427

Library scan: log summary and show popup #13427

ronso0 commented Jul 1, 2024 •

edited

Loading

ronso0 commented Jul 3, 2024

JoergAtGithub commented Jul 4, 2024

ronso0 commented Jul 4, 2024

ronso0 Jul 18, 2024

ronso0 commented Jul 24, 2024

JoergAtGithub commented Jul 24, 2024

ronso0 commented Jul 25, 2024

ronso0 commented Jul 25, 2024

ronso0 commented Jul 29, 2024

JoergAtGithub commented Jul 29, 2024

ronso0 commented Jul 29, 2024

ronso0 commented Jul 29, 2024

ronso0 commented Jul 29, 2024

daschuer commented Jul 30, 2024

JoergAtGithub commented Jul 30, 2024

ronso0 commented Jul 30, 2024

ronso0 commented Jul 30, 2024

ronso0 commented Jul 30, 2024

ronso0 commented Jul 30, 2024 •

edited

Loading

daschuer commented Jul 31, 2024

daschuer commented Jul 31, 2024

daschuer commented Jul 31, 2024

ronso0 commented Jul 31, 2024

ronso0 commented Aug 29, 2024

daschuer left a comment

daschuer Oct 30, 2024

ronso0 Nov 4, 2024

daschuer Oct 30, 2024

Library scan: log summary and show popup #13427

Are you sure you want to change the base?

Library scan: log summary and show popup #13427

Conversation

ronso0 commented Jul 1, 2024 • edited Loading

ronso0 commented Jul 3, 2024

JoergAtGithub commented Jul 4, 2024

ronso0 commented Jul 4, 2024

ronso0 Jul 18, 2024

Choose a reason for hiding this comment

ronso0 commented Jul 24, 2024

JoergAtGithub commented Jul 24, 2024

ronso0 commented Jul 25, 2024

ronso0 commented Jul 25, 2024

ronso0 commented Jul 29, 2024

JoergAtGithub commented Jul 29, 2024

ronso0 commented Jul 29, 2024

ronso0 commented Jul 29, 2024

ronso0 commented Jul 29, 2024

daschuer commented Jul 30, 2024

JoergAtGithub commented Jul 30, 2024

ronso0 commented Jul 30, 2024

ronso0 commented Jul 30, 2024

ronso0 commented Jul 30, 2024

ronso0 commented Jul 30, 2024 • edited Loading

daschuer commented Jul 31, 2024

daschuer commented Jul 31, 2024

daschuer commented Jul 31, 2024

ronso0 commented Jul 31, 2024

ronso0 commented Aug 29, 2024

daschuer left a comment

Choose a reason for hiding this comment

daschuer Oct 30, 2024

Choose a reason for hiding this comment

ronso0 Nov 4, 2024

Choose a reason for hiding this comment

daschuer Oct 30, 2024

Choose a reason for hiding this comment

ronso0 commented Jul 1, 2024 •

edited

Loading

ronso0 commented Jul 30, 2024 •

edited

Loading