navidrome/utils/str/str_test.go
Deluan Quintão c795bcfcf7
feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709)
* fix(server): more race conditions when updating artist/album from external sources

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scanner): add .gitignore syntax to .ndignore. Resolves #1394

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): null

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): pass configfile option to child process

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): resume interrupted fullScans

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): remove old scanner code

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): rename old metadata package

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): move old metadata package

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: tests

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): update Go to 1.23.4

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: logs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(test):

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: log level

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: remove log message

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add config for scanner watcher

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: children playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: replace `interface{}` with `any`

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: smart playlists with genres

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: allow any tags in smart playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: artist names in playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: smart playlist's sort by tags

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add moods to child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add moods to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic): use generic JSONArray for OS arrays

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic): use https in test

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add releaseTypes to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add recordLabels to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic): rename JSONArray to Array

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add artists to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add artists to Child

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): do not pre-populate smart playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): implement a simplified version of ArtistID3.

See https://github.com/opensubsonic/open-subsonic-api/discussions/120

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add artists to album child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add contributors to mediafile Child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add albumArtists to mediafile Child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add displayArtist and displayAlbumArtist

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add displayComposer to Child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add roles to ArtistID3

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): use " • " separator for displayComposer

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor:

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic):

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): respect `PreferSortTags` config option

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic):

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: optimize purging non-unused tags

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: don't run 'refresh artist stats' concurrently with other transactions

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor:

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: log message

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add Scanner.ScanOnStartup config option, default true

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: better json parsing error msg when importing NSPs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't update album's imported_time when updating external_metadata

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: handle interrupted scans and full scans after migrations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: run `analyze` when migration requires a full rescan

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: run `PRAGMA optimize` at the end of the scan

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't update artist's updated_at when updating external_metadata

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: handle multiple artists and roles in smart playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): dim missing tracks

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album missing logic

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: error encoding in gob

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: separate warnings from errors

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: mark albums as missing if they were contained in a deleted folder

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: add participant names to media_file and album tables

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: use participations in criteria, instead of m2m relationship

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename participations to participants

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add moods to album child

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: albumartist role case

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scanner): run scanner as an external process by default

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): show albumArtist names

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): dim out missing albums

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: flaky test

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): scrobble buffer mapping. fix #3583

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: more participations renaming

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: listenbrainz scrobbling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: send release_group_mbid to listenbrainz

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): implement OpenSubsonic explicitStatus field (#3597)

* feat: implement OpenSubsonic explicitStatus field

* fix(subsonic): fix failing snapshot tests

* refactor: create helper for setting explicitStatus

* fix: store smaller values for explicit-status on database

* test: ToAlbum explicitStatus

* refactor: rename explicitStatus helper function

---------

Co-authored-by: Deluan Quintão <deluan@navidrome.org>

* fix: handle album and track tags in the DB based on the mappings.yaml file

Signed-off-by: Deluan <deluan@navidrome.org>

* save similar artists as JSONB

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: getAlbumList byGenre

Signed-off-by: Deluan <deluan@navidrome.org>

* detect changes in PID configuration

Signed-off-by: Deluan <deluan@navidrome.org>

* set default album PID to legacy_pid

Signed-off-by: Deluan <deluan@navidrome.org>

* fix tests

Signed-off-by: Deluan <deluan@navidrome.org>

* fix SIGSEGV

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't lose album stars/ratings when migrating

Signed-off-by: Deluan <deluan@navidrome.org>

* store full PID conf in properties

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: keep album annotations when changing PID.Album config

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: reassign album annotations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: use (display) albumArtist and add links to each artist

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: not showing albums by albumartist

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: error msgs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: hide PID from Native API

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album cover art resolution

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: trim participant names

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: reduce watcher log spam

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: panic when initializing the watcher

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: various artists

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't store empty lyrics in the DB

Signed-off-by: Deluan <deluan@navidrome.org>

* remove unused methods

Signed-off-by: Deluan <deluan@navidrome.org>

* drop full_text indexes, as they are not being used by SQLite

Signed-off-by: Deluan <deluan@navidrome.org>

* keep album created_at when upgrading

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): null pointer

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album artwork cache

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't expose missing files in Subsonic API

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: searchable interface

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: filter out missing items from subsonic search

* fix: filter out missing items from playlists

* fix: filter out missing items from shares

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): add filter by artist role

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): only return albumartists in getIndexes and getArtists endpoints

Signed-off-by: Deluan <deluan@navidrome.org>

* sort roles alphabetically

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: artist playcounts

Signed-off-by: Deluan <deluan@navidrome.org>

* change default Album PID conf

Signed-off-by: Deluan <deluan@navidrome.org>

* fix albumartist link when it does not match any albumartists values

Signed-off-by: Deluan <deluan@navidrome.org>

* fix `Ignoring filter not whitelisted` (role) message

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: trim any names/titles being imported

Signed-off-by: Deluan <deluan@navidrome.org>

* remove unused genre code

Signed-off-by: Deluan <deluan@navidrome.org>

* serialize calls to Last.fm's getArtist

Signed-off-by: Deluan <deluan@navidrome.org>

xxx

Signed-off-by: Deluan <deluan@navidrome.org>

* add counters to genres

Signed-off-by: Deluan <deluan@navidrome.org>

* nit: fix migration `notice` message

Signed-off-by: Deluan <deluan@navidrome.org>

* optimize similar artists query

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: last.fm.getInfo when mbid does not exist

Signed-off-by: Deluan <deluan@navidrome.org>

* ui only show missing items for admins

Signed-off-by: Deluan <deluan@navidrome.org>

* don't allow interaction with missing items

Signed-off-by: Deluan <deluan@navidrome.org>

* Add Missing Files view (WIP)

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: merged tag_counts into tag table

Signed-off-by: Deluan <deluan@navidrome.org>

* add option to completely disable automatic scanner

Signed-off-by: Deluan <deluan@navidrome.org>

* add delete missing files functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: playlists not showing for regular users

Signed-off-by: Deluan <deluan@navidrome.org>

* reduce updateLastAccess frequency to once every minute

Signed-off-by: Deluan <deluan@navidrome.org>

* reduce update player frequency to once every minute

Signed-off-by: Deluan <deluan@navidrome.org>

* add timeout when updating player

Signed-off-by: Deluan <deluan@navidrome.org>

* remove dead code

Signed-off-by: Deluan <deluan@navidrome.org>

* fix duplicated roles in stats

Signed-off-by: Deluan <deluan@navidrome.org>

* add `; ` to artist splitters

Signed-off-by: Deluan <deluan@navidrome.org>

* fix stats query

Signed-off-by: Deluan <deluan@navidrome.org>

* more logs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* add record label filter

Signed-off-by: Deluan <deluan@navidrome.org>

* add release type filter

Signed-off-by: Deluan <deluan@navidrome.org>

* fix purgeUnused tags

Signed-off-by: Deluan <deluan@navidrome.org>

* add grouping filter to albums

Signed-off-by: Deluan <deluan@navidrome.org>

* allow any album tags to be used in as filters in the API

Signed-off-by: Deluan <deluan@navidrome.org>

* remove empty tags from album info

Signed-off-by: Deluan <deluan@navidrome.org>

* comments in the migration

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: Cannot read properties of undefined

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: listenbrainz scrobbling (#3640)

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: remove duplicated tag values

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't ignore the taglib folder!

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: show track subtitle tag

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: show artists stats based on selected role

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: inspect

Signed-off-by: Deluan <deluan@navidrome.org>

* add media type to album info/filters

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: change format of subtitle in the UI

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: subtitle in Subsonic API and search

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: subtitle in UI's player

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: split strings should be case-insensitive

Signed-off-by: Deluan <deluan@navidrome.org>

* disable ScanSchedule

Signed-off-by: Deluan <deluan@navidrome.org>

* increase default sessiontimeout

Signed-off-by: Deluan <deluan@navidrome.org>

* add sqlite command line tool to docker image

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: resources override

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album PID conf

Signed-off-by: Deluan <deluan@navidrome.org>

* change migration to mark current artists as albumArtists

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): Allow filtering on multiple genres (#3679)

* feat(ui): Allow filtering on multiple genres

Signed-off-by: Henrik Nordvik <henrikno@gmail.com>
Signed-off-by: Deluan <deluan@navidrome.org>

* add multi-genre filter in Album list

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Henrik Nordvik <henrikno@gmail.com>
Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Henrik Nordvik <henrikno@gmail.com>

* add more multi-valued tag filters to Album and Song views

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): unselect missing files after removing

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): song filter

Signed-off-by: Deluan <deluan@navidrome.org>

* fix sharing tracks. fix #3687

Signed-off-by: Deluan <deluan@navidrome.org>

* use rowids when using search for sync (ex: Symfonium)

Signed-off-by: Deluan <deluan@navidrome.org>

* fix "Report Real Paths" option for subsonic clients

Signed-off-by: Deluan <deluan@navidrome.org>

* fix "Report Real Paths" option for subsonic clients for search

Signed-off-by: Deluan <deluan@navidrome.org>

* add libraryPath to Native API /songs endpoint

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add album version

Signed-off-by: Deluan <deluan@navidrome.org>

* made all tags lowercase as they are case-insensitive anyways.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): Show full paths, extended properties for album/song (#3691)

* feat(ui): Show full paths, extended properties for album/song

- uses library path + os separator + path
- show participants (album/song) and tags (song)
- make album/participant clickable in show info

* add source to path

* fix pathSeparator in UI

Signed-off-by: Deluan <deluan@navidrome.org>

* fix local artist artwork (#3695)

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: parse vorbis performers

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: clean function into smaller functions

Signed-off-by: Deluan <deluan@navidrome.org>

* fix translations for en and pt

Signed-off-by: Deluan <deluan@navidrome.org>

* add trace log to show annotations reassignment

Signed-off-by: Deluan <deluan@navidrome.org>

* add trace log to show annotations reassignment

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: allow performers without instrument/subrole

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: metadata clean function again

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: optimize split function

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: split function is now a method of TagConf

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: humanize Artist total size

Signed-off-by: Deluan <deluan@navidrome.org>

* add album version to album details

Signed-off-by: Deluan <deluan@navidrome.org>

* don't display album-level tags in SongInfo

Signed-off-by: Deluan <deluan@navidrome.org>

* fix genre clicking in Album Page

Signed-off-by: Deluan <deluan@navidrome.org>

* don't use mbids in Last.fm api calls.

From 1337574018:

With MBID:
```
GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&mbid=a41ac10f-0a56-4672-9161-b83f9b223559&method=artist.getInfo

{
artist: {
name: "Bee Gees",
mbid: "bf0f7e29-dfe1-416c-b5c6-f9ebc19ea810",
url: "https://www.last.fm/music/Bee+Gees",
}
```

Without MBID:
```
GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&method=artist.getInfo

{
artist: {
name: "Van Morrison",
mbid: "a41ac10f-0a56-4672-9161-b83f9b223559",
url: "https://www.last.fm/music/Van+Morrison",
}
```

Signed-off-by: Deluan <deluan@navidrome.org>

* better logging for when the artist folder is not found

Signed-off-by: Deluan <deluan@navidrome.org>

* fix various issues with artist image resolution

Signed-off-by: Deluan <deluan@navidrome.org>

* hide "Additional Tags" header if there are none.

Signed-off-by: Deluan <deluan@navidrome.org>

* simplify tag rendering

Signed-off-by: Deluan <deluan@navidrome.org>

* enhance logging for artist folder detection

Signed-off-by: Deluan <deluan@navidrome.org>

* make folderID consistent for relative and absolute folderPaths

Signed-off-by: Deluan <deluan@navidrome.org>

* handle more folder paths scenarios

Signed-off-by: Deluan <deluan@navidrome.org>

* filter out other roles when SubsonicArtistParticipations = true

Signed-off-by: Deluan <deluan@navidrome.org>

* fix "Cannot read properties of undefined"

Signed-off-by: Deluan <deluan@navidrome.org>

* fix lyrics and comments being truncated (#3701)

* fix lyrics and comments being truncated

* specifically test for lyrics and comment length

* reorder assertions

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>

* fix(server): Expose library_path for playlist (#3705)

Allows showing absolute path for UI, and makes "report real path" work for playlists (Subsonic)

* fix BFR on Windows (#3704)

* fix potential reflected cross-site scripting vulnerability

Signed-off-by: Deluan <deluan@navidrome.org>

* hack to make it work on Windows

* ignore windows executables

* try fixing the pipeline

Signed-off-by: Deluan <deluan@navidrome.org>

* allow MusicFolder in other drives

* move windows local drive logic to local storage implementation

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* increase pagination sizes for missing files

Signed-off-by: Deluan <deluan@navidrome.org>

* reduce level of "already scanning" watcher log message

Signed-off-by: Deluan <deluan@navidrome.org>

* only count folders with audio files in it

See https://github.com/navidrome/navidrome/discussions/3676#discussioncomment-11990930

Signed-off-by: Deluan <deluan@navidrome.org>

* add album version and catalog number to search

Signed-off-by: Deluan <deluan@navidrome.org>

* add `organization` alias for `recordlabel`

Signed-off-by: Deluan <deluan@navidrome.org>

* remove mbid from Last.fm agent

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: support inspect in ui (#3726)

* inspect in ui

* address round 1

* add catalogNum to AlbumInfo

Signed-off-by: Deluan <deluan@navidrome.org>

* remove dependency on metadata_old (deprecated) package

Signed-off-by: Deluan <deluan@navidrome.org>

* add `RawTags` to model

Signed-off-by: Deluan <deluan@navidrome.org>

* support parsing MBIDs for roles (from the https://github.com/kgarner7/picard-all-mbids plugin) (#3698)


* parse standard roles, vorbis/m4a work for now

* fix djmixer

* working roles, use DJ-mix

* add performers to file

* map mbids

* add a few more tests

* add test

Signed-off-by: Deluan <deluan@navidrome.org>

* try to simplify the performers logic

Signed-off-by: Deluan <deluan@navidrome.org>

* stylistic changes

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>

* remove param mutation

Signed-off-by: Deluan <deluan@navidrome.org>

* run automated SQLite optimizations

Signed-off-by: Deluan <deluan@navidrome.org>

* fix playlists import/export on Windows

* fix import playlists

* fix export playlists

* better handling of Windows volumes

Signed-off-by: Deluan <deluan@navidrome.org>

* handle more album ID reassignments

Signed-off-by: Deluan <deluan@navidrome.org>

* allow adding/overriding tags in the config file

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): Fix playlist track id, handle missing tracks better (#3734)

- Use `mediaFileId` instead of `id` for playlist tracks
- Only fetch if the file is not missing
- If extractor fails to get the file, also error (rather than panic)

* optimize DB after each scan.

Signed-off-by: Deluan <deluan@navidrome.org>

* remove sortable from AlbumSongs columns

Signed-off-by: Deluan <deluan@navidrome.org>

* simplify query to get missing tracks

Signed-off-by: Deluan <deluan@navidrome.org>

* mark Scanner.Extractor as deprecated

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Signed-off-by: Henrik Nordvik <henrikno@gmail.com>
Co-authored-by: Caio Cotts <caio@cotts.com.br>
Co-authored-by: Henrik Nordvik <henrikno@gmail.com>
Co-authored-by: Kendall Garner <17521368+kgarner7@users.noreply.github.com>
2025-02-19 20:35:17 -05:00

148 lines
12 KiB
Go

package str_test
import (
"github.com/navidrome/navidrome/utils/str"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("String Utils", func() {
Describe("Clear", func() {
DescribeTable("replaces some Unicode chars with their equivalent ASCII",
func(input, expected string) {
Expect(str.Clear(input)).To(Equal(expected))
},
Entry("k-os", "k–os", "k-os"),
Entry("k‐os", "k‐os", "k-os"),
Entry(`"Weird" Al Yankovic`, "“Weird” Al Yankovic", `"Weird" Al Yankovic`),
Entry("Single quotes", "‘Single’ quotes", "'Single' quotes"),
)
})
Describe("LongestCommonPrefix", func() {
It("finds the longest common prefix", func() {
Expect(str.LongestCommonPrefix(testPaths)).To(Equal("/Music/iTunes 1/iTunes Media/Music/"))
})
It("does NOT handle partial prefixes", func() {
albums := []string{
"/artist/albumOne",
"/artist/albumTwo",
}
Expect(str.LongestCommonPrefix(albums)).To(Equal("/artist/album"))
})
})
})
var testPaths = []string{
"/Music/iTunes 1/iTunes Media/Music/ABBA/Gold_ Greatest Hits/Dancing Queen.m4a",
"/Music/iTunes 1/iTunes Media/Music/ABBA/Gold_ Greatest Hits/Mamma Mia.m4a",
"/Music/iTunes 1/iTunes Media/Music/Art Blakey/A Night At Birdland, Vol. 1/01 Annoucement By Pee Wee Marquette.m4a",
"/Music/iTunes 1/iTunes Media/Music/Art Blakey/A Night At Birdland, Vol. 1/02 Split Kick.m4a",
"/Music/iTunes 1/iTunes Media/Music/As Frenéticas/As Frenéticas/Perigosa.m4a",
"/Music/iTunes 1/iTunes Media/Music/Bachman-Turner Overdrive/Gold/Down Down.m4a",
"/Music/iTunes 1/iTunes Media/Music/Bachman-Turner Overdrive/Gold/Hey You.m4a",
"/Music/iTunes 1/iTunes Media/Music/Bachman-Turner Overdrive/Gold/Hold Back The Water.m4a",
"/Music/iTunes 1/iTunes Media/Music/Belle And Sebastian/Write About Love/01 I Didn't See It Coming.m4a",
"/Music/iTunes 1/iTunes Media/Music/Belle And Sebastian/Write About Love/02 Come On Sister.m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/Elephunk/03 Let's Get Retarded.m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/Elephunk/04 Hey Mama.m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/Monkey Business/10 They Don't Want Music (Feat. James Brown).m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/The E.N.D/1-01 Boom Boom Pow.m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/Timeless/01 Mas Que Nada.m4a",
"/Music/iTunes 1/iTunes Media/Music/Blondie/Heart Of Glass/Heart Of Glass.m4a",
"/Music/iTunes 1/iTunes Media/Music/Bob Dylan/Nashville Skyline/06 Lay Lady Lay.m4a",
"/Music/iTunes 1/iTunes Media/Music/Botany/Feeling Today - EP/03 Waterparker.m4a",
"/Music/iTunes 1/iTunes Media/Music/Céu/CéU/06 10 Contados.m4a",
"/Music/iTunes 1/iTunes Media/Music/Chance/Six Through Ten/03 Forgive+Forget.m4a",
"/Music/iTunes 1/iTunes Media/Music/Clive Tanaka Y Su Orquesta/Jet Set Siempre 1°/03 Neu Chicago (Side A) [For Dance].m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Absolute Rock Classics/1-02 Smoke on the water.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Almost Famous Soundtrack/10 Simple Man.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Audio News - Rock'n' Roll Forever/01 Rock Around The Clock.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Austin Powers_ International Man Of Mystery/01 The Magic Piper (Of Love).m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Austin Powers_ The Spy Who Shagged Me/04 American Woman.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Back To Dance/03 Long Cool Woman In A Black Dress.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Back To The 70's - O Album Da Década/03 American Pie.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Bambolê/09 In The Mood.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Bambolê - Volume II/03 Blue Moon.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Big Brother Brasil 2004/04 I Will Survive.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Collateral Soundtrack/03 Hands Of Time.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Forrest Gump - The Soundtrack/1-12 California Dreamin'.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Forrest Gump - The Soundtrack/1-16 Mrs. Robinson.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Ghost World - Original Motion Picture Soundtrack/01 Jaan Pechechaan Ho.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Grease [Original Soundtrack]/01 Grease.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/La Bamba/09 Summertime Blues.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Pretty Woman/10 Oh Pretty Woman.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents African Groove/01 Saye Mogo Bana.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Arabic Groove/02 Galbi.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Asian Groove/03 Remember Tomorrow.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Blues Lounge/01 Midnight Dream.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Blues Lounge/03 Banal Reality.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Blues Lounge/04 Parchman Blues.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Blues Lounge/06 Run On.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Brazilian Groove/01 Maria Moita.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Brazilian Lounge/08 E Depois....m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Brazilian Lounge/11 Os Grilos.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Euro Lounge/01 Un Simple Histoire.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Euro Lounge/02 Limbe.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Euro Lounge/05 Sempre Di Domenica.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Euro Lounge/12 Voulez-Vous_.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents World Lounge/03 Santa Maria.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents_ A New Groove/02 Dirty Laundry.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents_ Blues Around the World/02 Canceriano Sem Lar (Clinica Tobias Blues).m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents_ Euro Groove/03 Check In.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents_ World Groove/01 Attention.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Saturday Night Fever/01 Stayin' Alive.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Saturday Night Fever/03 Night Fever.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/The Best Air Guitar Album In The World... Ever!/2-06 Johnny B. Goode.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/The Full Monty - Soundtrack/02 You Sexy Thing.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/The Full Monty - Soundtrack/11 We Are Family.m4a",
"/Music/iTunes 1/iTunes Media/Music/Cut Copy/Zonoscope (Bonus Version)/10 Corner of the Sky.m4a",
"/Music/iTunes 1/iTunes Media/Music/David Bowie/Changesbowie/07 Diamond Dogs.m4a",
"/Music/iTunes 1/iTunes Media/Music/Douster & Savage Skulls/Get Rich or High Tryin' - EP/01 Bad Gal.m4a",
"/Music/iTunes 1/iTunes Media/Music/Elton John/Greatest Hits 1970-2002/1-04 Rocket Man (I Think It's Going to Be a Long, Long Time).m4a",
"/Music/iTunes 1/iTunes Media/Music/Elvis Presley/ELV1S 30 #1 Hits/02 Don't Be Cruel.m4a",
"/Music/iTunes 1/iTunes Media/Music/Eric Clapton/The Cream Of Clapton/03 I Feel Free.m4a",
"/Music/iTunes 1/iTunes Media/Music/Fleetwood Mac/The Very Best Of Fleetwood Mac/02 Don't Stop.m4a",
"/Music/iTunes 1/iTunes Media/Music/Françoise Hardy/Comment te dire adieu/Comment te dire adieu.m4a",
"/Music/iTunes 1/iTunes Media/Music/Games/That We Can Play - EP/01 Strawberry Skies.m4a",
"/Music/iTunes 1/iTunes Media/Music/Grand Funk Railroad/Collectors Series/The Loco-Motion.m4a",
"/Music/iTunes 1/iTunes Media/Music/Henry Mancini/The Pink Panther (Music from the Film Score)/The Pink Panther Theme.m4a",
"/Music/iTunes 1/iTunes Media/Music/Holy Ghost!/Do It Again - Single/01 Do It Again.m4a",
"/Music/iTunes 1/iTunes Media/Music/K.C. & The Sunshine Band/The Best of/03 I'm Your Boogie Man.m4a",
"/Music/iTunes 1/iTunes Media/Music/K.C. & The Sunshine Band/Unknown Album/Megamix (Thats The Way, Shake Your Booty, Get Down Tonight, Give It Up).m4a",
"/Music/iTunes 1/iTunes Media/Music/Kim Ann Foxman & Andy Butler/Creature - EP/01 Creature.m4a",
"/Music/iTunes 1/iTunes Media/Music/Nico/Chelsea Girl/01 The Fairest Of The Seasons.m4a",
"/Music/iTunes 1/iTunes Media/Music/oOoOO/oOoOO - EP/02 Burnout Eyess.m4a",
"/Music/iTunes 1/iTunes Media/Music/Peter Frampton/The Very Best of Peter Frampton/Baby, I Love Your Way.m4a",
"/Music/iTunes 1/iTunes Media/Music/Peter Frampton/The Very Best of Peter Frampton/Show Me The Way.m4a",
"/Music/iTunes 1/iTunes Media/Music/Raul Seixas/A Arte De Raul Seixas/03 Metamorfose Ambulante.m4a",
"/Music/iTunes 1/iTunes Media/Music/Raul Seixas/A Arte De Raul Seixas/18 Eu Nasci há 10 Mil Anos Atrás.m4a",
"/Music/iTunes 1/iTunes Media/Music/Rick James/Street Songs/Super Freak.m4a",
"/Music/iTunes 1/iTunes Media/Music/Rita Lee/Fruto Proibido/Agora Só Falta Você.m4a",
"/Music/iTunes 1/iTunes Media/Music/Rita Lee/Fruto Proibido/Esse Tal De Roque Enrow.m4a",
"/Music/iTunes 1/iTunes Media/Music/Roberto Carlos/Roberto Carlos 1966/05 Negro Gato.m4a",
"/Music/iTunes 1/iTunes Media/Music/SOHO/Goddess/02 Hippychick.m4a",
"/Music/iTunes 1/iTunes Media/Music/Stan Getz/Getz_Gilberto/05 Corcovado (Quiet Nights of Quiet Stars).m4a",
"/Music/iTunes 1/iTunes Media/Music/Steely Dan/Pretzel Logic/Rikki Don't Loose That Number.m4a",
"/Music/iTunes 1/iTunes Media/Music/Stevie Wonder/For Once In My Life/I Don't Know Why.m4a",
"/Music/iTunes 1/iTunes Media/Music/Teebs/Ardour/While You Doooo.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Beatles/Magical Mystery Tour/08 Strawberry Fields Forever.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Beatles/Past Masters, Vol. 1/10 Long Tall Sally.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Beatles/Please Please Me/14 Twist And Shout.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Beatles/Sgt. Pepper's Lonely Hearts Club Band/03 Lucy In The Sky With Diamonds.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/Amorica/09 Wiser Time.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/By Your Side/05 Only A Fool.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/Shake Your Money Maker/04 Could I''ve Been So Blind.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/The Southern Harmony And Musical Companion/01 Sting Me.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/Three Snakes And One Charm/02 Good Friday.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Doors/Strange Days (40th Anniversary Mixes)/01 Strange Days.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Rolling Stones/Forty Licks/1-03 (I Can't Get No) Satisfaction.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Velvet Underground/The Velvet Underground & Nico/02 I'm Waiting For The Man.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Velvet Underground/The Velvet Underground & Nico/03 Femme Fatale.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Velvet Underground/White Light_White Heat/04 Here She Comes Now.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Who/Sings My Generation/My Generation.m4a",
"/Music/iTunes 1/iTunes Media/Music/Village People/The Very Best Of Village People/Macho Man.m4a",
"/Music/iTunes 1/iTunes Media/Music/Vondelpark/Sauna - EP/01 California Analog Dream.m4a",
"/Music/iTunes 1/iTunes Media/Music/War/Why Can't We Be Friends/Low Rider.m4a",
"/Music/iTunes 1/iTunes Media/Music/Yes/Fragile/01 Roundabout.m4a",
}