ReplayGain

This commit is contained in:
zotify 2023-07-31 18:14:25 +12:00
parent 30721125ef
commit 911c29820a
13 changed files with 171 additions and 141 deletions

View file

@ -8,19 +8,19 @@ An unexpected reboot
- Most components have been completely rewritten to address some fundamental design issues with the previous codebase, This update will provide a better base for new features in the future.
- ~~Some~~ Most configuration options have been renamed, please check your configuration file.
- There is a new library path for podcasts, existing podcasts will stay where they are.
- There is a new library path for playlists, existing playlists will stay where they are.
### Changes
- Genre metadata available for tracks downloaded from an album
- Boolean command line options are now set like `--save-metadata` or `--no-save-metadata` for True or False
- Setting `--config` (formerly `--config-location`) can be set to "none" to not use any config file
- Setting `--config` (formerly `--config-location`) can be set to "None" to not use any config file
- Search result selector now accepts both comma-seperated and hyphen-seperated values at the same time
- Renamed `--liked`/`-l` to `--liked-tracks`/`-lt`
- Renamed `root_path` and `root_podcast_path` to `music_library` and `podcast_library`
- `--username` and `--password` arguments now take priority over saved credentials
- Regex pattern for cleaning filenames is now OS specific, allowing more usable characters on Linux & macOS.
- The default location for credentials.json on Linux is now ~/.config/zotify to keep it in the same place as config.json
- On Linux both `config.json` and `credentials.json` are now kept under `$XDG_CONFIG_HOME/zotify/`, (`~/.config/zotify/` by default).
- The output template used is now based on track info rather than search result category
- Search queries with spaces no longer need to be in quotes
- File metadata no longer uses sanitized file metadata, this will result in more accurate metadata.
@ -32,24 +32,24 @@ An unexpected reboot
- `--library`/`-l` overrides both `music_library` and `podcast_library` options similar to `--output`
- `--category`/`-c` will limit search results to a certain type, accepted values are "album", "artist", "playlist", "track", "show", "episode". Accepts multiple choices.
- `--debug` shows full tracebacks on crash instead of just the final error message
- Search results can be narrowed down using field filters
- Available filters are album, artist, track, year, upc, tag:hipster, tag:new, isrc, and genre.
- The artist and year filters can be used while searching albums, artists and tracks. You can filter on a single year or a range (e.g. 1970-1982).
- The album filter can be used while searching albums and tracks.
- The genre filter can be used while searching artists and tracks.
- The isrc and track filters can be used while searching tracks.
- The upc, tag:new and tag:hipster filters can only be used while searching albums. The tag:new filter will return albums released in the past two weeks and tag:hipster can be used to show only albums in the lowest 10% of popularity.
- Search results can be narrowed down using search filters
- Available filters are 'album', 'artist', 'track', 'year', 'upc', 'tag:hipster', 'tag:new', 'isrc', and 'genre'.
- The 'artist' and 'year' filters only shows results from the given year or a range (e.g. 1970-1982).
- The 'album' filter only shows results from the given album(s)
- The 'genre' filter only shows results from the given genre(s)
- The 'isrc' and 'track' filters can be used while searching tracks
- The 'upc', tag:new and tag:hipster filters can only be used while searching albums
- 'tag:new' filter will show albums released within the past two weeks
- 'tag:hipster' will only show albums in the lowest 10% of popularity
- Search has been expanded to include podcasts and episodes
- New output placeholders / metadata tags for tracks
- `{artists}`
- `{album_artist}`
- `{album_artists}`
- !!`{duration}` - In milliseconds
- `{explicit}`
- `{explicit_symbol}` - For output format, will be \[E] if track is explicit.
- `{duration}` (milliseconds)
- `{isrc}`
- `{licensor}`
- !!`{popularity}`
- `{popularity}`
- `{release_date}`
- `{track_number}`
- Genre information is now more accurate and is always enabled
@ -60,19 +60,19 @@ An unexpected reboot
- Added support for transcoding to wav and wavpack formats
- Unsynced lyrics are saved to a txt file instead of lrc
- Unsynced lyrics can now be embedded directly into file metadata (for supported file types)
- Added new option `save_lyrics`
- Added new option `save_lyrics_file`
- This option only affects the external lyrics files
- Embedded lyrics are tied to `save_metadata`
- Embedded lyrics are controlled with `save_metadata`
### Removals
- Removed "Zotify" ASCII banner
- Removed search prompt
- Removed search prompt, searches can only be done as cli arguments now.
- Removed song archive files
- Removed `{ext}` option in output formats as file extentions are managed automatically
- Removed `split_album_discs` because the same functionality cna be achieved by using output formatting and it was causing conflicts
- Removed `print_api_errors` because API errors are now trated like regular errors
- Removed the following config options due to lack of utility
- Removed `split_album_discs` because the same functionality can be achieved by using output formatting
- Removed `print_api_errors` because API errors are now treated like regular errors
- Removed the following config options due to their corresponding features being removed:
- `bulk_wait_time`
- `download_real_time`
- `md_allgenres`

View file

@ -93,7 +93,7 @@ Here's a very simple example of downloading a track and its metadata:
```python
import zotify
session = zotify.Session(username="username", password="password")
session = zotify.Session.from_userpass(username="username", password="password")
track = session.get_track("4cOdK2wGLETKBW3PvgPWqT")
output = track.create_output("./Music", "{artist} - {title}")

View file

@ -1,5 +1,5 @@
librespot>=0.0.9
music-tag
music-tag@git+https://zotify.xyz/zotify/music-tag
mutagen
Pillow
pwinput

View file

@ -3,3 +3,4 @@ flake8
mypy
pre-commit
types-requests
wheel

View file

@ -20,7 +20,7 @@ packages = zotify
python_requires = >=3.10
install_requires =
librespot>=0.0.9
music-tag
music-tag@git+https://zotify.xyz/zotify/music-tag
mutagen
Pillow
pwinput
@ -35,5 +35,3 @@ console_scripts =
# Conflicts with black
ignore = E203
max-line-length = 160
per-file-ignores =
zotify/file.py: E701

View file

@ -70,15 +70,9 @@ class Api(ApiClient):
class Session:
__api: Api
__country: str
__language: str
__session: LibrespotSession
__session_builder: LibrespotSession.Builder
def __init__(
self,
session_builder: LibrespotSession.Builder,
librespot_session: LibrespotSession,
language: str = "en",
) -> None:
"""
@ -87,10 +81,10 @@ class Session:
session_builder: An instance of the Librespot Session.Builder
langauge: ISO 639-1 language code
"""
self.__session_builder = session_builder
self.__session = self.__session_builder.create()
self.__session = librespot_session
self.__language = language
self.__api = Api(self.__session, language)
self.__country = self.api().invoke_url(API_URL + "me")["country"]
@staticmethod
def from_file(cred_file: Path, langauge: str = "en") -> Session:
@ -98,7 +92,7 @@ class Session:
Creates session using saved credentials file
Args:
cred_file: Path to credentials file
langauge: ISO 639-1 language code
langauge: ISO 639-1 language code for API responses
Returns:
Zotify session
"""
@ -107,9 +101,8 @@ class Session:
.set_store_credentials(False)
.build()
)
return Session(
LibrespotSession.Builder(conf).stored_file(str(cred_file)), langauge
)
session = LibrespotSession.Builder(conf).stored_file(str(cred_file))
return Session(session.create(), langauge)
@staticmethod
def from_userpass(
@ -124,7 +117,7 @@ class Session:
username: Account username
password: Account password
save_file: Path to save login credentials to, optional.
langauge: ISO 639-1 language code
langauge: ISO 639-1 language code for API responses
Returns:
Zotify session
"""
@ -132,22 +125,18 @@ class Session:
password = (
pwinput(prompt="Password: ", mask="*") if password == "" else password
)
builder = LibrespotSession.Configuration.Builder()
if save_file:
save_file.parent.mkdir(parents=True, exist_ok=True)
conf = (
LibrespotSession.Configuration.Builder()
.set_stored_credential_file(str(save_file))
.build()
)
builder.set_stored_credential_file(str(save_file))
else:
conf = (
LibrespotSession.Configuration.Builder()
.set_store_credentials(False)
.build()
)
return Session(
LibrespotSession.Builder(conf).user_pass(username, password), language
builder.set_store_credentials(False)
session = LibrespotSession.Builder(builder.build()).user_pass(
username, password
)
return Session(session.create(), language)
def __get_playable(
self, playable_id: PlayableId, quality: Quality
@ -188,11 +177,7 @@ class Session:
def country(self) -> str:
"""Returns two letter country code of user's account"""
try:
return self.__country
except AttributeError:
self.__country = self.api().invoke_url(API_URL + "me")["country"]
return self.__country
return self.__country
def is_premium(self) -> bool:
"""Returns users premium account status"""
@ -200,4 +185,4 @@ class Session:
def clone(self) -> Session:
"""Creates a copy of the session for use in a parallel thread"""
return Session(session_builder=self.__session_builder, language=self.__language)
return Session(self.__session, self.__language)

View file

@ -137,6 +137,8 @@ def main():
from traceback import format_exc
print(format_exc().splitlines()[-1])
except KeyboardInterrupt:
print("goodbye")
if __name__ == "__main__":

View file

@ -22,7 +22,7 @@ from zotify.printer import PrintChannel, Printer
from zotify.utils import API_URL, AudioFormat, b62_to_hex
class ParsingError(RuntimeError):
class ParseError(ValueError):
...
@ -36,6 +36,7 @@ class PlayableData(NamedTuple):
id: PlayableId
library: Path
output: str
metadata: dict[str, Any] = {}
class Selection:
@ -55,17 +56,18 @@ class Selection:
],
) -> list[str]:
categories = ",".join(category)
resp = self.__session.api().invoke_url(
API_URL + "search",
{
"q": search_text,
"type": categories,
"include_external": "audio",
"market": self.__session.country(),
},
limit=10,
offset=0,
)
with Loader("Searching..."):
resp = self.__session.api().invoke_url(
API_URL + "search",
{
"q": search_text,
"type": categories,
"include_external": "audio",
"market": self.__session.country(),
},
limit=10,
offset=0,
)
count = 0
links = []
@ -79,11 +81,22 @@ class Selection:
count += 1
return self.__get_selection(links)
def get(self, item: str, suffix: str) -> list[str]:
resp = self.__session.api().invoke_url(f"{API_URL}me/{item}", limit=50)[suffix]
def get(self, category: str, name: str = "", content: str = "") -> list[str]:
with Loader("Fetching items..."):
r = self.__session.api().invoke_url(f"{API_URL}me/{category}", limit=50)
if content != "":
r = r[content]
resp = r["items"]
items = []
for i in range(len(resp)):
self.__print(i + 1, resp[i])
return self.__get_selection(resp)
try:
item = resp[i][name]
except KeyError:
item = resp[i]
items.append(item)
self.__print(i + 1, item)
return self.__get_selection(items)
@staticmethod
def from_file(file_path: Path) -> list[str]:
@ -169,8 +182,6 @@ class Selection:
class App:
__config: Config
__session: Session
__playable_list: list[PlayableData] = []
def __init__(self, args: Namespace):
@ -204,7 +215,7 @@ class App:
with Loader("Parsing input..."):
try:
self.parse(ids)
except ParsingError as e:
except ParseError as e:
Printer.print(PrintChannel.ERRORS, str(e))
self.download_all()
@ -214,13 +225,13 @@ class App:
if args.search:
return selection.search(" ".join(args.search), args.category)
elif args.playlist:
return selection.get("playlists", "items")
return selection.get("playlists")
elif args.followed:
return selection.get("following?type=artist", "artists")
return selection.get("following?type=artist", content="artists")
elif args.liked_tracks:
return selection.get("tracks", "items")
return selection.get("tracks", "track")
elif args.liked_episodes:
return selection.get("episodes", "items")
return selection.get("episodes")
elif args.download:
ids = []
for x in args.download:
@ -228,9 +239,10 @@ class App:
return ids
elif args.urls:
return args.urls
except (FileNotFoundError, ValueError, KeyboardInterrupt):
pass
Printer.print(PrintChannel.WARNINGS, "there is nothing to do")
except (FileNotFoundError, ValueError):
Printer.print(PrintChannel.WARNINGS, "there is nothing to do")
except KeyboardInterrupt:
Printer.print(PrintChannel.WARNINGS, "\nthere is nothing to do")
exit()
def parse(self, links: list[str]) -> None:
@ -246,7 +258,7 @@ class App:
_id = split[-1]
id_type = split[-2]
except IndexError:
raise ParsingError(f'Could not parse "{link}"')
raise ParseError(f'Could not parse "{link}"')
match id_type:
case "album":
@ -262,7 +274,7 @@ class App:
case "playlist":
self.__parse_playlist(_id)
case _:
raise ParsingError(f'Unknown content type "{id_type}"')
raise ParseError(f'Unknown content type "{id_type}"')
def __parse_album(self, hex_id: str) -> None:
album = self.__session.api().get_metadata_4_album(AlbumId.from_hex(hex_id))
@ -279,9 +291,9 @@ class App:
def __parse_artist(self, hex_id: str) -> None:
artist = self.__session.api().get_metadata_4_artist(ArtistId.from_hex(hex_id))
for album in artist.album_group + artist.single_group:
for album_group in artist.album_group and artist.single_group:
album = self.__session.api().get_metadata_4_album(
AlbumId.from_hex(album.gid)
AlbumId.from_hex(album_group.album[0].gid)
)
for disc in album.disc:
for track in disc.track:
@ -373,7 +385,7 @@ class App:
self.__config.chunk_size,
)
if self.__config.save_lyrics and playable.type == PlayableType.TRACK:
if self.__config.save_lyrics_file and playable.type == PlayableType.TRACK:
with Loader("Fetching lyrics..."):
try:
track.get_lyrics().save(output)

View file

@ -1,5 +1,6 @@
from argparse import Namespace
from json import dump, load
from os import environ
from pathlib import Path
from sys import platform as PLATFORM
from typing import Any
@ -33,7 +34,7 @@ PRINT_PROGRESS = "print_progress"
PRINT_SKIPS = "print_skips"
PRINT_WARNINGS = "print_warnings"
REPLACE_EXISTING = "replace_existing"
SAVE_LYRICS = "save_lyrics"
SAVE_LYRICS_FILE = "save_lyrics_file"
SAVE_METADATA = "save_metadata"
SAVE_SUBTITLES = "save_subtitles"
SKIP_DUPLICATES = "skip_duplicates"
@ -42,8 +43,10 @@ TRANSCODE_BITRATE = "transcode_bitrate"
SYSTEM_PATHS = {
"win32": Path.home().joinpath("AppData/Roaming/Zotify"),
"linux": Path.home().joinpath(".config/zotify"),
"darwin": Path.home().joinpath("Library/Application Support/Zotify"),
"linux": Path(environ.get("XDG_CONFIG_HOME") or "~/.config")
.expanduser()
.joinpath("zotify"),
}
LIBRARY_PATHS = {
@ -171,10 +174,10 @@ CONFIG_VALUES = {
"arg": "--language",
"help": "Language for metadata",
},
SAVE_LYRICS: {
SAVE_LYRICS_FILE: {
"default": True,
"type": bool,
"arg": "--save-lyrics",
"arg": "--save-lyrics-file",
"help": "Save lyrics to a file",
},
LYRICS_ONLY: {
@ -277,7 +280,7 @@ class Config:
playlist_library: Path
podcast_library: Path
print_progress: bool
save_lyrics: bool
save_lyrics_file: bool
save_metadata: bool
transcode_bitrate: int

View file

@ -1,12 +1,11 @@
from errno import ENOENT
from pathlib import Path
from subprocess import PIPE, Popen
from typing import Any
from music_tag import load_file
from mutagen.oggvorbis import OggVorbisHeaderError
from zotify.utils import AudioFormat
from zotify.utils import AudioFormat, MetadataEntry
class TranscodingError(RuntimeError):
@ -87,7 +86,7 @@ class LocalFile:
self.__audio_format = audio_format
self.__bitrate = bitrate
def write_metadata(self, metadata: dict[str, Any]) -> None:
def write_metadata(self, metadata: list[MetadataEntry]) -> None:
"""
Write metadata to file
Args:
@ -95,9 +94,9 @@ class LocalFile:
"""
f = load_file(self.__path)
f.save()
for k, v in metadata.items():
for m in metadata:
try:
f[k] = str(v)
f[m.name] = m.value
except KeyError:
pass
try:

View file

@ -14,6 +14,7 @@ from zotify.utils import (
LYRICS_URL,
AudioFormat,
ImageSize,
MetadataEntry,
bytes_to_base62,
fix_filename,
)
@ -53,7 +54,7 @@ class Lyrics:
class Playable:
cover_images: list[Any]
metadata: dict[str, Any]
metadata: list[MetadataEntry]
name: str
input_stream: GeneralAudioStream
@ -67,13 +68,12 @@ class Playable:
Returns:
File path for the track
"""
for k, v in self.metadata.items():
output = output.replace(
"{" + k + "}", fix_filename(str(v).replace("\0", ", "))
)
for m in self.metadata:
if m.output is not None:
output = output.replace("{" + m.name + "}", fix_filename(m.output))
file_path = library.joinpath(output).expanduser()
if file_path.exists() and not replace:
raise FileExistsError("Output Creation Error: File already downloaded")
raise FileExistsError("File already downloaded")
else:
file_path.parent.mkdir(parents=True, exist_ok=True)
return file_path
@ -140,28 +140,34 @@ class Track(PlayableContentFeeder.LoadedStream, Playable):
except AttributeError:
return super().__getattribute__("track").__getattribute__(name)
def __default_metadata(self) -> dict[str, Any]:
def __default_metadata(self) -> list[MetadataEntry]:
date = self.album.date
return {
"album": self.album.name,
"album_artist": "\0".join([a.name for a in self.album.artist]),
"artist": self.artist[0].name,
"artists": "\0".join([a.name for a in self.artist]),
"date": f"{date.year}-{date.month}-{date.day}",
"disc_number": self.disc_number,
"duration": self.duration,
"explicit": self.explicit,
"explicit_symbol": "[E]" if self.explicit else "",
"isrc": self.external_id[0].id,
"popularity": (self.popularity * 255) / 100,
"track_number": str(self.number).zfill(2),
# "year": self.album.date.year,
"title": self.name,
"replaygain_track_gain": self.normalization_data.track_gain_db,
"replaygain_track_peak": self.normalization_data.track_peak,
"replaygain_album_gain": self.normalization_data.album_gain_db,
"replaygain_album_peak": self.normalization_data.album_peak,
}
return [
MetadataEntry("album", self.album.name),
MetadataEntry("album_artist", [a.name for a in self.album.artist]),
MetadataEntry("artist", self.artist[0].name),
MetadataEntry("artists", [a.name for a in self.artist]),
MetadataEntry("date", f"{date.year}-{date.month}-{date.day}"),
MetadataEntry("disc", self.disc_number),
MetadataEntry("duration", self.duration),
MetadataEntry("explicit", self.explicit, "[E]" if self.explicit else ""),
MetadataEntry("isrc", self.external_id[0].id),
MetadataEntry("popularity", int(self.popularity * 255) / 100),
MetadataEntry("track_number", self.number, str(self.number).zfill(2)),
MetadataEntry("title", self.name),
MetadataEntry(
"replaygain_track_gain", self.normalization_data.track_gain_db, ""
),
MetadataEntry(
"replaygain_track_peak", self.normalization_data.track_peak, ""
),
MetadataEntry(
"replaygain_album_gain", self.normalization_data.album_gain_db, ""
),
MetadataEntry(
"replaygain_album_peak", self.normalization_data.album_peak, ""
),
]
def get_lyrics(self) -> Lyrics:
"""Returns track lyrics if available"""
@ -198,17 +204,17 @@ class Episode(PlayableContentFeeder.LoadedStream, Playable):
except AttributeError:
return super().__getattribute__("episode").__getattribute__(name)
def __default_metadata(self) -> dict[str, Any]:
return {
"description": self.description,
"duration": self.duration,
"episode_number": self.number,
"explicit": self.explicit,
"language": self.language,
"podcast": self.show.name,
"date": self.publish_time,
"title": self.name,
}
def __default_metadata(self) -> list[MetadataEntry]:
return [
MetadataEntry("description", self.description),
MetadataEntry("duration", self.duration),
MetadataEntry("episode_number", self.number),
MetadataEntry("explicit", self.explicit, "[E]" if self.explicit else ""),
MetadataEntry("language", self.language),
MetadataEntry("podcast", self.show.name),
MetadataEntry("date", self.publish_time),
MetadataEntry("title", self.name),
]
def write_audio_stream(
self, output: Path, chunk_size: int = 128 * 1024
@ -221,7 +227,7 @@ class Episode(PlayableContentFeeder.LoadedStream, Playable):
Returns:
LocalFile object
"""
if bool(self.external_url):
if not bool(self.external_url):
return super().write_audio_stream(output, chunk_size)
file = f"{output}.{self.external_url.rsplit('.', 1)[-1]}"
with get(self.external_url, stream=True) as r, open(

View file

@ -71,11 +71,12 @@ class Printer:
unit_divisor=unit_divisor,
)
@staticmethod
def print_loader(msg: str) -> None:
@classmethod
def print_loader(cls, msg: str) -> None:
"""
Prints animated loading symbol
Args:
msg: Message to print
"""
print(msg, flush=True, end="")
if cls.__config.print_progress:
print(msg, flush=True, end="")

View file

@ -2,7 +2,7 @@ from argparse import Action, ArgumentError
from enum import Enum, IntEnum
from re import IGNORECASE, sub
from sys import platform as PLATFORM
from typing import NamedTuple
from typing import Any, NamedTuple
from librespot.audio.decoders import AudioQuality
from librespot.util import Base62, bytes_to_hex
@ -112,6 +112,29 @@ class OptionalOrFalse(Action):
)
class MetadataEntry:
def __init__(self, name: str, value: Any, output_value: str | None = None):
"""
Holds metadata entries
args:
name: name of metadata key
tag_val: Value to use in metadata tags
output_value: Value when used in output formatting
"""
self.name = name
if type(value) == list:
value = "\0".join(value)
self.value = value
if output_value is None:
output_value = value
if output_value == "":
output_value = None
if type(output_value) == list:
output_value = ", ".join(output_value)
self.output = str(output_value)
def fix_filename(filename: str, substitute: str = "_", platform: str = PLATFORM) -> str:
"""
Replace invalid characters on Linux/Windows/MacOS with underscores.