Compare commits

...

11 Commits

Author SHA1 Message Date
Alex b4d497f53d Merge pull request #937 from alexta69/copilot/check-issue-692
Propagate missing playlist context fields (playlist_count, playlist_autonumber, etc.)
2026-04-02 10:55:00 +03:00
Alex Shnitman 0cba61c9a4 update README 2026-04-02 10:52:56 +03:00
Alex Shnitman 9858157581 Merge branch 'copilot/fix-healthcheck-failure-ipvlan' of https://github.com/alexta69/metube into copilot/check-issue-692 (closes #936) 2026-04-02 10:52:11 +03:00
copilot-swe-agent[bot] d7eaaaa94b Add clarifying comments for n_entries and __last_playlist_index fields (closes #692)
Agent-Logs-Url: https://github.com/alexta69/metube/sessions/b5aeb55a-3197-4a14-b8b4-96c9a67796e8

Co-authored-by: alexta69 <7450369+alexta69@users.noreply.github.com>
2026-04-02 10:51:03 +03:00
copilot-swe-agent[bot] 771ba52d53 Use PORT env variable in Dockerfile HEALTHCHECK instead of hardcoded 8081
Agent-Logs-Url: https://github.com/alexta69/metube/sessions/899e7074-fd3d-4538-8bad-8ee6804d5052

Co-authored-by: alexta69 <7450369+alexta69@users.noreply.github.com>
2026-04-02 07:25:14 +00:00
copilot-swe-agent[bot] 1cc27d3f55 Initial plan 2026-04-02 07:23:14 +00:00
copilot-swe-agent[bot] 981e6c1003 Propagate missing playlist context fields (playlist_count, playlist_autonumber, n_entries, __last_playlist_index)
The playlist/channel processing loop now sets playlist_count,
playlist_autonumber, n_entries, and __last_playlist_index on each
video entry so that templates like %(playlist_autonumber)s,
%(playlist_count)s, and %(playlist_index&{} - |)s resolve correctly
instead of showing NA.

Also updates _compact_persisted_entry to preserve n_entries and
__last_playlist_index across restarts.

Fixes #692

Agent-Logs-Url: https://github.com/alexta69/metube/sessions/b5aeb55a-3197-4a14-b8b4-96c9a67796e8

Co-authored-by: alexta69 <7450369+alexta69@users.noreply.github.com>
2026-04-01 19:59:32 +00:00
copilot-swe-agent[bot] b17e1e5668 Add explanatory comment for fake STR_FORMAT_RE_TMPL key group in tests
Agent-Logs-Url: https://github.com/alexta69/metube/sessions/0ae5ff34-540f-4fc8-a81c-358fb92b7c15

Co-authored-by: alexta69 <7450369+alexta69@users.noreply.github.com>
2026-04-01 19:34:09 +00:00
copilot-swe-agent[bot] c1b5540332 Replace custom template substitution with yt-dlp's evaluate_outtmpl
Replace the hand-rolled _outtmpl_substitute_field() / _compile_outtmpl_pattern()
with a new _resolve_outtmpl_fields() that delegates to yt-dlp's
YoutubeDL.evaluate_outtmpl().  This gives playlist/channel output templates
access to yt-dlp's full template syntax: defaults (%(field|fallback)s),
conditional formatting (%(field&prefix {})s), math (%(field+N)d),
datetime formatting (%(field>%Y-%m-%d)s), and more.

Only field references whose root name matches the targeted prefix (e.g.
"playlist" or "channel") are resolved; all other references remain as
template placeholders for yt-dlp to fill during the actual download.

Agent-Logs-Url: https://github.com/alexta69/metube/sessions/0ae5ff34-540f-4fc8-a81c-358fb92b7c15

Co-authored-by: alexta69 <7450369+alexta69@users.noreply.github.com>
2026-04-01 19:31:27 +00:00
Alex Shnitman 483575d24a add subscriptions; change persistence file format to JSON (closes #901, #76, #113, #170, #242, #444, #503, #555, #566) 2026-04-01 14:33:24 +03:00
Alex Shnitman 84c6418f91 fix pickle (closes #814) 2026-03-21 12:42:17 +02:00
20 changed files with 3416 additions and 807 deletions
+2 -4
View File
@@ -26,9 +26,6 @@ RUN sed -i 's/\r$//g' docker-entrypoint.sh && \
gosu \
curl \
tini \
file \
gdbmtool \
sqlite3 \
build-essential && \
curl -LsSf https://astral.sh/uv/install.sh | UV_INSTALL_DIR=/usr/local/bin sh && \
UV_PROJECT_ENVIRONMENT=/usr/local uv sync --frozen --no-dev --compile-bytecode && \
@@ -66,9 +63,10 @@ ENV UMASK=022
ENV DOWNLOAD_DIR=/downloads
ENV STATE_DIR=/downloads/.metube
ENV TEMP_DIR=/downloads
ENV PORT=8081
VOLUME /downloads
EXPOSE 8081
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 CMD curl -fsS "http://localhost:8081/" || exit 1
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 CMD curl -fsS "http://localhost:${PORT}/" || exit 1
# Add build-time argument for version
ARG VERSION=dev
+10 -2
View File
@@ -3,7 +3,12 @@
![Build Status](https://github.com/alexta69/metube/actions/workflows/main.yml/badge.svg)
![Docker Pulls](https://img.shields.io/docker/pulls/alexta69/metube.svg)
Web GUI for youtube-dl (using the [yt-dlp](https://github.com/yt-dlp/yt-dlp) fork) with playlist support. Allows you to download videos from YouTube and [dozens of other sites](https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md).
MeTube is a self-hosted web UI for `yt-dlp`, for downloading media from YouTube and [dozens of other sites](https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md).
Key capabilities:
* Download videos, audio, captions, and thumbnails from a browser UI.
* Download playlists and channels, with configurable output and download options.
* Subscribe to channels and playlists, periodically check for new items, and queue new uploads automatically.
![screenshot1](https://github.com/alexta69/metube/raw/master/screenshot.gif)
@@ -36,6 +41,9 @@ Certain values can be set via environment variables, using the `-e` parameter on
* __MAX_CONCURRENT_DOWNLOADS__: Maximum number of simultaneous downloads allowed. For example, if set to `5`, then at most five downloads will run concurrently, and any additional downloads will wait until one of the active downloads completes. Defaults to `3`.
* __DELETE_FILE_ON_TRASHCAN__: if `true`, downloaded files are deleted on the server, when they are trashed from the "Completed" section of the UI. Defaults to `false`.
* __DEFAULT_OPTION_PLAYLIST_ITEM_LIMIT__: Maximum number of playlist items that can be downloaded. Defaults to `0` (no limit).
* __SUBSCRIPTION_DEFAULT_CHECK_INTERVAL__: Default minutes between automatic checks for each subscription. Defaults to `60`.
* __SUBSCRIPTION_SCAN_PLAYLIST_END__: Maximum playlist/channel entries to fetch per subscription check (newest-first). Defaults to `50`.
* __SUBSCRIPTION_MAX_SEEN_IDS__: Cap on stored video IDs per subscription to limit state file growth. Defaults to `50000`.
* __CLEAR_COMPLETED_AFTER__: Number of seconds after which completed (and failed) downloads are automatically removed from the "Completed" list. Defaults to `0` (disabled).
### 📁 Storage & Directories
@@ -46,7 +54,7 @@ Certain values can be set via environment variables, using the `-e` parameter on
* __CREATE_CUSTOM_DIRS__: Whether to support automatically creating directories within the __DOWNLOAD_DIR__ (or __AUDIO_DOWNLOAD_DIR__) if they do not exist. When enabled, the download directory selector supports free-text input, and the specified directory will be created recursively. Defaults to `true`.
* __CUSTOM_DIRS_EXCLUDE_REGEX__: Regular expression to exclude some custom directories from the dropdown. Empty regex disables exclusion. Defaults to `(^|/)[.@].*$`, which means directories starting with `.` or `@`.
* __DOWNLOAD_DIRS_INDEXABLE__: If `true`, the download directories (__DOWNLOAD_DIR__ and __AUDIO_DOWNLOAD_DIR__) are indexable on the web server. Defaults to `false`.
* __STATE_DIR__: Path to where the queue persistence files will be saved. Defaults to `/downloads/.metube` in the Docker image, and `.` otherwise.
* __STATE_DIR__: Path to where MeTube will store its persistent state files (`queue.json`, `pending.json`, `completed.json`, `subscriptions.json`). Defaults to `/downloads/.metube` in the Docker image, and `.` otherwise.
* __TEMP_DIR__: Path where intermediary download files will be saved. Defaults to `/downloads` in the Docker image, and `.` otherwise.
* Set this to an SSD or RAM filesystem (e.g., `tmpfs`) for better performance.
* __Note__: Using a RAM filesystem may prevent downloads from being resumed.
+167 -28
View File
@@ -17,6 +17,7 @@ import re
from watchfiles import DefaultFilter, Change, awatch
from ytdl import DownloadQueueNotifier, DownloadQueue, Download
from subscriptions import SubscriptionManager, SubscriptionNotifier, SubscriptionInfo
from yt_dlp.version import __version__ as yt_dlp_version
log = logging.getLogger('main')
@@ -50,6 +51,9 @@ class Config:
'OUTPUT_TEMPLATE_PLAYLIST': '%(playlist_title)s/%(title)s.%(ext)s',
'OUTPUT_TEMPLATE_CHANNEL': '%(channel)s/%(title)s.%(ext)s',
'DEFAULT_OPTION_PLAYLIST_ITEM_LIMIT' : '0',
'SUBSCRIPTION_DEFAULT_CHECK_INTERVAL': '60',
'SUBSCRIPTION_SCAN_PLAYLIST_END': '50',
'SUBSCRIPTION_MAX_SEEN_IDS': '50000',
'CLEAR_COMPLETED_AFTER': '0',
'YTDL_OPTIONS': '{}',
'YTDL_OPTIONS_FILE': '',
@@ -114,6 +118,7 @@ class Config:
'PUBLIC_HOST_URL',
'PUBLIC_HOST_AUDIO_URL',
'DEFAULT_OPTION_PLAYLIST_ITEM_LIMIT',
'SUBSCRIPTION_DEFAULT_CHECK_INTERVAL',
)
def frontend_safe(self) -> dict:
@@ -272,6 +277,34 @@ dqueue = DownloadQueue(config, Notifier())
app.on_startup.append(lambda app: dqueue.initialize())
app.on_cleanup.append(lambda app: Download.shutdown_manager())
class MetubeSubscriptionNotifier(SubscriptionNotifier):
async def subscription_added(self, sub: SubscriptionInfo):
log.info("Subscription added: %s", sub.name)
await sio.emit('subscription_added', serializer.encode(sub.to_public_dict()))
async def subscription_updated(self, sub: SubscriptionInfo):
await sio.emit('subscription_updated', serializer.encode(sub.to_public_dict()))
async def subscription_removed(self, sub_id: str):
log.info("Subscription removed: %s", sub_id)
await sio.emit('subscription_removed', serializer.encode(sub_id))
async def subscriptions_all(self, subs: list[SubscriptionInfo]):
await sio.emit('subscriptions_all', serializer.encode([s.to_public_dict() for s in subs]))
submgr = SubscriptionManager(config, dqueue, MetubeSubscriptionNotifier())
app.on_cleanup.append(lambda app: submgr.close())
async def _subscription_loop_startup(app):
"""aiohttp on_startup requires awaitable receivers; start_background_loop is sync."""
submgr.start_background_loop()
app.on_startup.append(_subscription_loop_startup)
class FileOpsFilter(DefaultFilter):
def __call__(self, change_type: int, path: str) -> bool:
# Check if this path matches our YTDL_OPTIONS_FILE
@@ -332,27 +365,17 @@ async def _read_json_request(request: web.Request) -> dict:
return post
@routes.post(config.URL_PREFIX + 'add')
async def add(request):
log.info("Received request to add download")
post = await _read_json_request(request)
post = _migrate_legacy_request(post)
log.info(
"Add download request: type=%s quality=%s format=%s has_folder=%s auto_start=%s",
post.get('download_type'),
post.get('quality'),
post.get('format'),
bool(post.get('folder')),
post.get('auto_start'),
)
def parse_download_options(post: dict) -> dict:
"""Validate add/subscribe body; raise HTTPBadRequest on invalid input."""
post = _migrate_legacy_request(dict(post))
url = post.get('url')
download_type = post.get('download_type')
codec = post.get('codec')
format = post.get('format')
quality = post.get('quality')
if not url or not quality or not download_type:
log.error("Bad request: missing 'url', 'download_type', or 'quality'")
raise web.HTTPBadRequest()
raise web.HTTPBadRequest(reason="missing 'url', 'download_type', or 'quality'")
url = str(url).strip()
folder = post.get('folder')
custom_name_prefix = post.get('custom_name_prefix')
playlist_item_limit = post.get('playlist_item_limit')
@@ -429,20 +452,54 @@ async def add(request):
except (TypeError, ValueError) as exc:
raise web.HTTPBadRequest(reason='playlist_item_limit must be an integer') from exc
return {
'url': url,
'download_type': download_type,
'codec': codec,
'format': format,
'quality': quality,
'folder': folder,
'custom_name_prefix': custom_name_prefix,
'playlist_item_limit': playlist_item_limit,
'auto_start': auto_start,
'split_by_chapters': split_by_chapters,
'chapter_template': chapter_template,
'subtitle_language': subtitle_language,
'subtitle_mode': subtitle_mode,
}
@routes.post(config.URL_PREFIX + 'add')
async def add(request):
log.info("Received request to add download")
post = await _read_json_request(request)
try:
o = parse_download_options(post)
except web.HTTPBadRequest as e:
log.error("Bad request: %s", e.reason)
raise
log.info(
"Add download request: type=%s quality=%s format=%s has_folder=%s auto_start=%s",
o['download_type'],
o['quality'],
o['format'],
bool(o.get('folder')),
o['auto_start'],
)
status = await dqueue.add(
url,
download_type,
codec,
format,
quality,
folder,
custom_name_prefix,
playlist_item_limit,
auto_start,
split_by_chapters,
chapter_template,
subtitle_language,
subtitle_mode,
o['url'],
o['download_type'],
o['codec'],
o['format'],
o['quality'],
o['folder'],
o['custom_name_prefix'],
o['playlist_item_limit'],
o['auto_start'],
o['split_by_chapters'],
o['chapter_template'],
o['subtitle_language'],
o['subtitle_mode'],
)
return web.Response(text=serializer.encode(status))
@@ -451,6 +508,82 @@ async def cancel_add(request):
dqueue.cancel_add()
return web.Response(text=serializer.encode({'status': 'ok'}), content_type='application/json')
@routes.post(config.URL_PREFIX + 'subscribe')
async def subscribe(request):
post = await _read_json_request(request)
try:
o = parse_download_options(post)
except web.HTTPBadRequest:
raise
cic = post.get('check_interval_minutes')
if cic is None:
cic = config.SUBSCRIPTION_DEFAULT_CHECK_INTERVAL
try:
cic = int(cic)
except (TypeError, ValueError) as exc:
raise web.HTTPBadRequest(reason='check_interval_minutes must be an integer') from exc
if cic < 1:
raise web.HTTPBadRequest(reason='check_interval_minutes must be at least 1')
result = await submgr.add_subscription(
o['url'],
check_interval_minutes=cic,
download_type=o['download_type'],
codec=o['codec'],
format=o['format'],
quality=o['quality'],
folder=o['folder'] or '',
custom_name_prefix=o['custom_name_prefix'],
auto_start=o['auto_start'],
playlist_item_limit=o['playlist_item_limit'],
split_by_chapters=o['split_by_chapters'],
chapter_template=o['chapter_template'],
subtitle_language=o['subtitle_language'],
subtitle_mode=o['subtitle_mode'],
)
return web.Response(text=serializer.encode(result))
@routes.get(config.URL_PREFIX + 'subscriptions')
async def subscriptions_list(request):
return web.Response(text=serializer.encode([s.to_public_dict() for s in submgr.list_all()]))
@routes.post(config.URL_PREFIX + 'subscriptions/update')
async def subscriptions_update(request):
post = await _read_json_request(request)
sub_id = post.get('id')
if not sub_id:
raise web.HTTPBadRequest(reason='missing subscription id')
changes = {k: v for k, v in post.items() if k != 'id' and k in ('enabled', 'check_interval_minutes', 'name')}
if not changes:
raise web.HTTPBadRequest(reason='no valid fields to update')
log.info("Subscription update requested for %s: %s", sub_id, sorted(changes.keys()))
result = await submgr.update_subscription(str(sub_id), changes)
return web.Response(text=serializer.encode(result))
@routes.post(config.URL_PREFIX + 'subscriptions/delete')
async def subscriptions_delete(request):
post = await _read_json_request(request)
ids = post.get('ids')
if not ids or not isinstance(ids, list):
raise web.HTTPBadRequest(reason='missing ids list')
result = await submgr.delete_subscriptions([str(i) for i in ids])
return web.Response(text=serializer.encode(result))
@routes.post(config.URL_PREFIX + 'subscriptions/check')
async def subscriptions_check(request):
post = await _read_json_request(request)
ids = post.get('ids')
if ids is not None and not isinstance(ids, list):
raise web.HTTPBadRequest(reason='ids must be a list')
log.info("Subscription check-now requested for ids=%s", ids if ids else "all-enabled")
result = await submgr.check_now([str(i) for i in ids] if ids else None)
return web.Response(text=serializer.encode(result))
@routes.post(config.URL_PREFIX + 'delete')
async def delete(request):
post = await _read_json_request(request)
@@ -554,6 +687,7 @@ async def history(request):
async def connect(sid, environ):
log.info(f"Client connected: {sid}")
await sio.emit('all', serializer.encode(dqueue.get()), to=sid)
await sio.emit('subscriptions_all', serializer.encode([s.to_public_dict() for s in submgr.list_all()]), to=sid)
await sio.emit('configuration', serializer.encode(config.frontend_safe()), to=sid)
if config.CUSTOM_DIRS:
await sio.emit('custom_dirs', serializer.encode(get_custom_dirs()), to=sid)
@@ -672,6 +806,11 @@ async def add_cors(request):
app.router.add_route('OPTIONS', config.URL_PREFIX + 'add', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'cancel-add', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'subscribe', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'subscriptions', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'subscriptions/update', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'subscriptions/delete', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'subscriptions/check', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'upload-cookies', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'delete-cookies', add_cors)
+156
View File
@@ -0,0 +1,156 @@
from __future__ import annotations
import base64
import collections.abc
import json
import logging
import os
import shelve
import tempfile
import time
from datetime import datetime
from typing import Any, Optional
log = logging.getLogger("state_store")
STATE_SCHEMA_VERSION = 2
_BYTES_MARKER = "__metube_bytes__"
_DATETIME_MARKER = "__metube_datetime__"
def to_json_compatible(value: Any) -> Any:
if value is None or isinstance(value, (bool, int, float, str)):
return value
if isinstance(value, bytes):
return {_BYTES_MARKER: base64.b64encode(value).decode("ascii")}
if isinstance(value, datetime):
return {_DATETIME_MARKER: value.isoformat()}
if isinstance(value, collections.abc.Mapping):
return {str(k): to_json_compatible(v) for k, v in value.items()}
if isinstance(value, (list, tuple, set, frozenset)):
return [to_json_compatible(v) for v in value]
if isinstance(value, collections.abc.Iterable):
return [to_json_compatible(v) for v in value]
raise TypeError(f"Value of type {type(value).__name__} is not JSON serializable")
def from_json_compatible(value: Any) -> Any:
if isinstance(value, list):
return [from_json_compatible(v) for v in value]
if isinstance(value, dict):
if set(value.keys()) == {_BYTES_MARKER}:
return base64.b64decode(value[_BYTES_MARKER].encode("ascii"))
if set(value.keys()) == {_DATETIME_MARKER}:
return datetime.fromisoformat(value[_DATETIME_MARKER])
return {k: from_json_compatible(v) for k, v in value.items()}
return value
def read_legacy_shelf(path: str) -> Optional[list[tuple[Any, Any]]]:
if not os.path.exists(path):
return None
try:
with shelve.open(path, "r") as shelf:
return list(shelf.items())
except Exception as exc:
log.warning("Could not read legacy shelf at %s: %s", path, exc)
return None
class AtomicJsonStore:
def __init__(self, path: str, *, kind: str, schema_version: int = STATE_SCHEMA_VERSION):
self.path = path
self.kind = kind
self.schema_version = schema_version
def _ensure_parent(self) -> None:
parent = os.path.dirname(self.path)
if parent and not os.path.isdir(parent):
os.makedirs(parent, exist_ok=True)
def _build_payload(self, data: dict[str, Any]) -> dict[str, Any]:
payload = {
"schema_version": self.schema_version,
"kind": self.kind,
}
payload.update(data)
return payload
def load(self) -> Optional[dict[str, Any]]:
if not os.path.exists(self.path):
return None
try:
with open(self.path, encoding="utf-8") as f:
payload = json.load(f)
if not isinstance(payload, dict):
raise ValueError("State file must contain a JSON object")
if payload.get("kind") != self.kind:
raise ValueError(
f"State file kind mismatch: expected {self.kind}, got {payload.get('kind')}"
)
return payload
except Exception as exc:
self.quarantine_invalid_file(exc)
return None
def save(self, data: dict[str, Any]) -> None:
self._ensure_parent()
payload = self._build_payload(data)
parent = os.path.dirname(self.path) or "."
fd, tmp_path = tempfile.mkstemp(
prefix=f".{os.path.basename(self.path)}.",
suffix=".tmp",
dir=parent,
text=True,
)
try:
with os.fdopen(fd, "w", encoding="utf-8") as f:
json.dump(payload, f, ensure_ascii=False, separators=(",", ":"))
f.write("\n")
f.flush()
os.fsync(f.fileno())
os.replace(tmp_path, self.path)
self._fsync_directory(parent)
except Exception:
try:
os.remove(tmp_path)
except OSError:
pass
raise
def quarantine_invalid_file(self, exc: Exception) -> None:
if not os.path.exists(self.path):
return
ts = time.strftime("%Y%m%d%H%M%S")
backup_path = f"{self.path}.invalid.{ts}"
try:
os.replace(self.path, backup_path)
log.warning(
"State file at %s was invalid (%s); moved it to %s",
self.path,
exc,
backup_path,
)
except OSError as move_exc:
log.warning(
"State file at %s was invalid (%s) and could not be moved aside: %s",
self.path,
exc,
move_exc,
)
@staticmethod
def _fsync_directory(path: str) -> None:
try:
flags = os.O_RDONLY
if hasattr(os, "O_DIRECTORY"):
flags |= os.O_DIRECTORY
fd = os.open(path, flags)
except OSError:
return
try:
os.fsync(fd)
except OSError:
pass
finally:
os.close(fd)
+666
View File
@@ -0,0 +1,666 @@
"""Channel/playlist subscriptions: periodic yt-dlp flat extract + queue new videos."""
from __future__ import annotations
import asyncio
import copy
import logging
import os
import time
import types
import uuid
from dataclasses import dataclass, field, fields
from typing import Any, Optional
import yt_dlp
import yt_dlp.networking.impersonate
from state_store import AtomicJsonStore, read_legacy_shelf
log = logging.getLogger("subscriptions")
VIDEO_ONLY_MSG = (
"This URL points to a single video, not a channel or playlist. Use Download instead."
)
_MEDIA_HINT_FIELDS = (
"duration",
"timestamp",
"release_timestamp",
"upload_date",
"view_count",
"live_status",
"availability",
)
def _impersonate_opt(ytdl_options: dict) -> dict:
opts = dict(ytdl_options)
if "impersonate" in opts:
opts["impersonate"] = yt_dlp.networking.impersonate.ImpersonateTarget.from_str(
opts["impersonate"]
)
return opts
def _build_ydl_params(config, *, playlistend: Optional[int] = None) -> dict:
params: dict[str, Any] = {
"quiet": not logging.getLogger().isEnabledFor(logging.DEBUG),
"verbose": logging.getLogger().isEnabledFor(logging.DEBUG),
"no_color": True,
"extract_flat": True,
"ignore_no_formats_error": True,
"lazy_playlist": True,
"paths": {"home": config.DOWNLOAD_DIR, "temp": config.TEMP_DIR},
**config.YTDL_OPTIONS,
}
params = _impersonate_opt(params)
if playlistend is not None and playlistend > 0:
params["playlistend"] = playlistend
return params
def _is_media_entry(entry: Any) -> bool:
if not isinstance(entry, dict):
return False
etype = str(entry.get("_type") or "")
if etype in ("playlist", "multi_video", "channel"):
return False
if entry.get("entries"):
return False
url = _entry_video_url(entry)
if not url:
return False
ie_key = str(entry.get("ie_key") or entry.get("extractor_key") or "").lower()
if any(token in ie_key for token in ("playlist", "channel", "tab")):
return any(entry.get(field) is not None for field in _MEDIA_HINT_FIELDS)
return True
def extract_flat_playlist(config, url: str, playlistend: int, *, _depth: int = 0):
"""Return (info_dict, entries_list) for playlist/channel URLs."""
params = _build_ydl_params(config, playlistend=playlistend)
with yt_dlp.YoutubeDL(params=params) as ydl:
info = ydl.extract_info(url, download=False)
if not info:
return None, []
etype = info.get("_type") or "video"
if etype == "video":
return info, []
if etype in ("playlist", "channel"):
entries = info.get("entries") or []
if isinstance(entries, types.GeneratorType):
entries = list(entries)
# Drop None placeholders from incomplete flat playlists
entries = [e for e in entries if e]
media_entries = [e for e in entries if _is_media_entry(e)]
if media_entries:
return info, media_entries
if _depth < 1:
for ent in entries[:5]:
nested_url = _entry_video_url(ent)
if not nested_url:
continue
nested_info, nested_entries = extract_flat_playlist(
config,
nested_url,
playlistend,
_depth=_depth + 1,
)
if nested_entries:
return nested_info, nested_entries
return info, entries
if etype.startswith("url") and info.get("url"):
# Single nested URL without playlist wrapper — treat as non-subscribable
return info, []
return info, []
def _entry_video_url(entry: dict) -> Optional[str]:
return entry.get("webpage_url") or entry.get("url")
def _entry_id(entry: dict) -> Optional[str]:
eid = entry.get("id")
if eid is not None:
return str(eid)
url = _entry_video_url(entry)
return url
@dataclass
class SubscriptionInfo:
id: str
name: str
url: str
enabled: bool = True
check_interval_minutes: int = 60
download_type: str = "video"
codec: str = "auto"
format: str = "any"
quality: str = "best"
folder: str = ""
custom_name_prefix: str = ""
auto_start: bool = True
playlist_item_limit: int = 0
split_by_chapters: bool = False
chapter_template: str = ""
subtitle_language: str = "en"
subtitle_mode: str = "prefer_manual"
last_checked: Optional[float] = None
seen_ids: list[str] = field(default_factory=list)
error: Optional[str] = None
timestamp: float = field(default_factory=time.time)
def seen_set(self) -> set[str]:
return set(self.seen_ids)
def to_public_dict(self) -> dict:
return {
"id": self.id,
"name": self.name,
"url": self.url,
"enabled": self.enabled,
"check_interval_minutes": self.check_interval_minutes,
"download_type": self.download_type,
"codec": self.codec,
"format": self.format,
"quality": self.quality,
"folder": self.folder,
"last_checked": self.last_checked,
"seen_count": len(self.seen_ids),
"error": self.error,
}
def _subscription_to_record(sub: SubscriptionInfo) -> dict[str, Any]:
return {
"id": sub.id,
"name": sub.name,
"url": sub.url,
"enabled": sub.enabled,
"check_interval_minutes": sub.check_interval_minutes,
"download_type": sub.download_type,
"codec": sub.codec,
"format": sub.format,
"quality": sub.quality,
"folder": sub.folder,
"custom_name_prefix": sub.custom_name_prefix,
"auto_start": sub.auto_start,
"playlist_item_limit": sub.playlist_item_limit,
"split_by_chapters": sub.split_by_chapters,
"chapter_template": sub.chapter_template,
"subtitle_language": sub.subtitle_language,
"subtitle_mode": sub.subtitle_mode,
"last_checked": sub.last_checked,
"seen_ids": list(sub.seen_ids),
"error": sub.error,
}
def _subscription_from_record(record: Any) -> Optional[SubscriptionInfo]:
field_names = {f.name for f in fields(SubscriptionInfo)}
if isinstance(record, SubscriptionInfo):
return record
if isinstance(record, dict):
try:
return SubscriptionInfo(**{k: v for k, v in record.items() if k in field_names})
except TypeError:
return None
return None
class SubscriptionNotifier:
"""Hook for Socket.IO / UI updates."""
async def subscription_added(self, sub: SubscriptionInfo) -> None:
raise NotImplementedError
async def subscription_updated(self, sub: SubscriptionInfo) -> None:
raise NotImplementedError
async def subscription_removed(self, sub_id: str) -> None:
raise NotImplementedError
async def subscriptions_all(self, subs: list[SubscriptionInfo]) -> None:
raise NotImplementedError
class SubscriptionManager:
def __init__(self, config, download_queue, notifier: SubscriptionNotifier):
self.config = config
self.dqueue = download_queue
self.notifier = notifier
pdir = config.STATE_DIR
if not os.path.isdir(pdir):
os.makedirs(pdir, exist_ok=True)
self._legacy_path = os.path.join(pdir, "subscriptions")
self._path = os.path.join(pdir, "subscriptions.json")
self._store = AtomicJsonStore(self._path, kind="subscriptions")
self._subs: dict[str, SubscriptionInfo] = {}
self._url_index: dict[str, str] = {} # normalized url -> id
self._pending_urls: set[str] = set()
self._lock = asyncio.Lock()
self._loop_task: Optional[asyncio.Task] = None
self._load_all()
def close(self) -> None:
# No persistent shelf handle to close.
return
def _normalize_url(self, url: str) -> str:
return (url or "").strip()
def _normalize_seen_ids(self, seen_ids: list[str]) -> list[str]:
max_seen = int(getattr(self.config, "SUBSCRIPTION_MAX_SEEN_IDS", 50000))
normalized = [str(sid) for sid in dict.fromkeys(seen_ids)]
if len(normalized) > max_seen:
normalized = normalized[:max_seen]
return normalized
def _load_all(self) -> None:
payload = self._store.load()
loaded_from_legacy = False
if payload is not None:
records = payload.get("items") or []
else:
legacy_items = read_legacy_shelf(self._legacy_path)
records = [raw for _key, raw in legacy_items] if legacy_items else []
if records:
loaded_from_legacy = True
loaded_subs = self._iter_valid_subs(records)
compact_records = []
for sub in loaded_subs:
sub.seen_ids = self._normalize_seen_ids(sub.seen_ids)
self._subs[sub.id] = sub
self._url_index[self._normalize_url(sub.url)] = sub.id
compact_records.append(_subscription_to_record(sub))
if loaded_from_legacy or (
payload is not None
and (
payload.get("schema_version") != self._store.schema_version
or compact_records != records
)
):
self._store.save({"items": compact_records})
def _iter_valid_subs(self, records: list[Any]) -> list[SubscriptionInfo]:
subs: list[SubscriptionInfo] = []
for record in records:
sub = _subscription_from_record(record)
if sub is not None:
subs.append(sub)
return subs
def _save_locked(self) -> None:
self._store.save({"items": [_subscription_to_record(sub) for sub in self._subs.values()]})
async def _queue_subscription_entries(
self,
entries: list[dict],
*,
download_type: str,
codec: str,
format: str,
quality: str,
folder: str,
custom_name_prefix: str,
playlist_item_limit: int,
auto_start: bool,
split_by_chapters: bool,
chapter_template: str,
subtitle_language: str,
subtitle_mode: str,
) -> tuple[list[str], list[str]]:
queued_ids: list[str] = []
queue_errors: list[str] = []
for ent in entries:
eid = _entry_id(ent)
vurl = _entry_video_url(ent)
if not eid or not vurl:
continue
queue_entry = dict(ent)
queue_entry["_type"] = "video"
queue_entry["webpage_url"] = vurl
result = await self.dqueue.add_entry(
queue_entry,
download_type,
codec,
format,
quality,
folder or None,
custom_name_prefix,
playlist_item_limit,
auto_start,
split_by_chapters,
chapter_template or None,
subtitle_language,
subtitle_mode,
)
if isinstance(result, dict) and result.get("status") == "error":
msg = str(result.get("msg") or f"Queueing failed for {vurl}")
queue_errors.append(msg)
log.warning("Subscription queueing failed for %s: %s", vurl, msg)
continue
queued_ids.append(eid)
return queued_ids, queue_errors
def list_all(self) -> list[SubscriptionInfo]:
return list(self._subs.values())
def get(self, sub_id: str) -> Optional[SubscriptionInfo]:
return self._subs.get(sub_id)
def start_background_loop(self) -> None:
if self._loop_task is not None and not self._loop_task.done():
return
self._loop_task = asyncio.create_task(self._periodic_loop())
self._loop_task.add_done_callback(
lambda t: log.error("Subscription loop failed: %s", t.exception())
if not t.cancelled() and t.exception()
else None
)
async def _periodic_loop(self) -> None:
while True:
await asyncio.sleep(60)
try:
await self.run_due_checks()
except Exception as e:
log.exception("Subscription periodic check error: %s", e)
async def run_due_checks(self) -> None:
now = time.time()
due: list[SubscriptionInfo] = []
async with self._lock:
for sub in list(self._subs.values()):
if not sub.enabled:
continue
interval_sec = max(60, int(sub.check_interval_minutes) * 60)
if sub.last_checked is None:
due.append(sub)
continue
if now - sub.last_checked < interval_sec:
continue
due.append(sub)
for sub in due:
await self._check_one_unlocked(sub)
async def add_subscription(
self,
url: str,
*,
check_interval_minutes: int,
download_type: str,
codec: str,
format: str,
quality: str,
folder: str,
custom_name_prefix: str,
auto_start: bool,
playlist_item_limit: int,
split_by_chapters: bool,
chapter_template: str,
subtitle_language: str,
subtitle_mode: str,
) -> dict:
url = self._normalize_url(url)
if not url:
return {"status": "error", "msg": "Missing URL"}
async with self._lock:
if url in self._url_index or url in self._pending_urls:
return {"status": "error", "msg": "This URL is already subscribed"}
self._pending_urls.add(url)
try:
scan_first = max(int(getattr(self.config, "SUBSCRIPTION_SCAN_PLAYLIST_END", 50)), 1)
try:
info, entries = extract_flat_playlist(self.config, url, scan_first)
except yt_dlp.utils.YoutubeDLError as exc:
return {"status": "error", "msg": str(exc)}
if not info:
return {"status": "error", "msg": "Could not resolve URL"}
etype = info.get("_type") or "video"
if etype not in ("playlist", "channel"):
return {"status": "error", "msg": VIDEO_ONLY_MSG}
name = (
info.get("title")
or info.get("channel")
or info.get("playlist_title")
or info.get("uploader")
or url
)
seen_entries = [ent for ent in entries if _is_media_entry(ent)]
all_ids: list[str] = []
for ent in seen_entries:
eid = _entry_id(ent)
if eid:
all_ids.append(eid)
sub = SubscriptionInfo(
id=str(uuid.uuid4()),
name=str(name),
url=url,
enabled=True,
check_interval_minutes=max(1, int(check_interval_minutes)),
download_type=download_type,
codec=codec,
format=format,
quality=quality,
folder=folder or "",
custom_name_prefix=custom_name_prefix or "",
auto_start=bool(auto_start),
playlist_item_limit=int(playlist_item_limit),
split_by_chapters=bool(split_by_chapters),
chapter_template=chapter_template or "",
subtitle_language=subtitle_language,
subtitle_mode=subtitle_mode,
last_checked=time.time(),
seen_ids=list(dict.fromkeys(all_ids)),
error=None,
)
async with self._lock:
if url in self._url_index:
return {"status": "error", "msg": "This URL is already subscribed"}
self._subs[sub.id] = sub
self._url_index[url] = sub.id
try:
self._save_locked()
except Exception:
self._subs.pop(sub.id, None)
self._url_index.pop(url, None)
raise
await self.notifier.subscription_added(sub)
return {"status": "ok", "subscription": sub.to_public_dict()}
finally:
async with self._lock:
self._pending_urls.discard(url)
async def delete_subscriptions(self, ids: list[str]) -> dict:
removed: list[str] = []
async with self._lock:
previous_subs = self._subs.copy()
previous_index = self._url_index.copy()
for sid in ids:
sub = self._subs.pop(sid, None)
if sub:
normalized_url = self._normalize_url(sub.url)
self._url_index.pop(normalized_url, None)
removed.append(sid)
if removed:
try:
self._save_locked()
except Exception:
self._subs = previous_subs
self._url_index = previous_index
raise
for sid in removed:
await self.notifier.subscription_removed(sid)
return {"status": "ok"}
async def update_subscription(self, sub_id: str, changes: dict) -> dict:
async with self._lock:
sub = self._subs.get(sub_id)
if not sub:
return {"status": "error", "msg": "Subscription not found"}
previous = copy.deepcopy(sub)
old_enabled = sub.enabled
if "enabled" in changes:
sub.enabled = bool(changes["enabled"])
if "check_interval_minutes" in changes:
sub.check_interval_minutes = max(1, int(changes["check_interval_minutes"]))
if "name" in changes and changes["name"]:
sub.name = str(changes["name"])
try:
self._save_locked()
except Exception:
self._subs[sub_id] = previous
raise
updated = sub
if "enabled" in changes and updated.enabled != old_enabled:
log.info(
"Subscription %s %s",
updated.name,
"resumed" if updated.enabled else "paused",
)
await self.notifier.subscription_updated(updated)
return {"status": "ok", "subscription": updated.to_public_dict()}
async def check_now(self, ids: Optional[list[str]] = None) -> dict:
async with self._lock:
targets = (
[self._subs[i] for i in ids if i in self._subs]
if ids
else [s for s in self._subs.values() if s.enabled]
)
log.info(
"Manual subscription check requested for %d subscription(s)",
len(targets),
)
for sub in targets:
await self._check_one_unlocked(sub)
return {"status": "ok"}
async def _check_one_unlocked(self, sub: SubscriptionInfo) -> None:
sid = sub.id
scan = int(getattr(self.config, "SUBSCRIPTION_SCAN_PLAYLIST_END", 50))
log.info("Checking subscription: %s", sub.name)
try:
info, entries = extract_flat_playlist(self.config, sub.url, scan)
except yt_dlp.utils.YoutubeDLError as exc:
async with self._lock:
cur = self._subs.get(sid)
if cur:
previous = copy.deepcopy(cur)
cur.error = str(exc)
try:
self._save_locked()
except Exception:
self._subs[sid] = previous
raise
sub = cur
log.warning("Subscription check failed for %s: %s", sub.name, exc)
await self.notifier.subscription_updated(sub)
return
entries = [ent for ent in entries if _is_media_entry(ent)]
etype = (info or {}).get("_type") or "video"
if etype == "video" or not entries:
async with self._lock:
cur = self._subs.get(sid)
if cur:
previous = copy.deepcopy(cur)
cur.error = VIDEO_ONLY_MSG
try:
self._save_locked()
except Exception:
self._subs[sid] = previous
raise
sub = cur
log.warning("Subscription %s no longer resolves to a subscribable feed", sub.name)
await self.notifier.subscription_updated(sub)
return
async with self._lock:
cur = self._subs.get(sid)
if not cur:
return
seen = cur.seen_set()
seen_ids_snapshot = list(cur.seen_ids)
dl_type = cur.download_type
dl_codec = cur.codec
dl_format = cur.format
dl_quality = cur.quality
dl_folder = cur.folder
dl_prefix = cur.custom_name_prefix
dl_plimit = cur.playlist_item_limit
dl_autostart = cur.auto_start
dl_split = cur.split_by_chapters
dl_chapter = cur.chapter_template
dl_sublang = cur.subtitle_language
dl_submode = cur.subtitle_mode
new_entries: list[dict] = []
new_ids: list[str] = []
for ent in entries:
eid = _entry_id(ent)
if not eid or eid in seen:
continue
new_entries.append(ent)
new_ids.append(eid)
queued_ids, queue_errors = await self._queue_subscription_entries(
new_entries,
download_type=dl_type,
codec=dl_codec,
format=dl_format,
quality=dl_quality,
folder=dl_folder,
custom_name_prefix=dl_prefix,
playlist_item_limit=dl_plimit,
auto_start=dl_autostart,
split_by_chapters=dl_split,
chapter_template=dl_chapter or "",
subtitle_language=dl_sublang,
subtitle_mode=dl_submode,
)
log.info(
"Subscription check finished for %s: %d new, %d queued, %d failed",
sub.name,
len(new_entries),
len(queued_ids),
len(queue_errors),
)
merged = list(dict.fromkeys(queued_ids + seen_ids_snapshot))
max_seen = int(getattr(self.config, "SUBSCRIPTION_MAX_SEEN_IDS", 50000))
if len(merged) > max_seen:
merged = merged[:max_seen]
async with self._lock:
cur = self._subs.get(sid)
if not cur:
return
previous = copy.deepcopy(cur)
cur.seen_ids = merged
cur.last_checked = time.time()
cur.error = "; ".join(queue_errors[:3]) if queue_errors else None
try:
self._save_locked()
except Exception:
self._subs[sid] = previous
raise
sub = cur
await self.notifier.subscription_updated(sub)
async def emit_all(self) -> None:
await self.notifier.subscriptions_all(self.list_all())
+31
View File
@@ -144,3 +144,34 @@ async def test_start_pending_moves_to_queue(dq_env):
with patch.object(DownloadQueue, "_DownloadQueue__start_download", AsyncMock()):
await dq.start_pending([url])
assert not dq.pending.exists(url)
@pytest.mark.asyncio
async def test_add_entry_queues_single_video_without_reextracting(dq_env):
notifier = AsyncMock()
dq = DownloadQueue(dq_env, notifier)
entry = {
"_type": "video",
"id": "vid1",
"title": "Test Video",
"url": "https://example.com/watch?v=1",
"webpage_url": "https://example.com/watch?v=1",
"playlist_index": "01",
"playlist_title": "Playlist",
}
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", side_effect=AssertionError("should not re-extract")):
result = await dq.add_entry(
entry,
"video",
"auto",
"any",
"best",
"",
"",
0,
auto_start=False,
)
assert result["status"] == "ok"
assert dq.pending.exists("https://example.com/watch?v=1")
+217 -2
View File
@@ -1,10 +1,38 @@
"""Integration tests for ``PersistentQueue`` (shelve-backed storage)."""
"""Integration tests for ``PersistentQueue`` using the JSON state store."""
from __future__ import annotations
import json
import os
import shelve
import sys
import tempfile
import types
import unittest
from unittest.mock import patch
fake_yt_dlp = types.ModuleType("yt_dlp")
fake_networking = types.ModuleType("yt_dlp.networking")
fake_impersonate = types.ModuleType("yt_dlp.networking.impersonate")
fake_utils = types.ModuleType("yt_dlp.utils")
class _ImpersonateTarget:
@staticmethod
def from_str(value):
return value
fake_impersonate.ImpersonateTarget = _ImpersonateTarget
fake_networking.impersonate = fake_impersonate
fake_utils.STR_FORMAT_RE_TMPL = r"(?P<prefix>)%\((?P<has_key>{})\)(?P<format>[-0-9.]*{})"
fake_utils.STR_FORMAT_TYPES = "diouxXeEfFgGcrsa"
fake_yt_dlp.networking = fake_networking
fake_yt_dlp.utils = fake_utils
sys.modules.setdefault("yt_dlp", fake_yt_dlp)
sys.modules.setdefault("yt_dlp.networking", fake_networking)
sys.modules.setdefault("yt_dlp.networking.impersonate", fake_impersonate)
sys.modules.setdefault("yt_dlp.utils", fake_utils)
from ytdl import DownloadInfo, PersistentQueue
@@ -35,6 +63,12 @@ def _make_info(url: str = "https://example.com/v") -> DownloadInfo:
)
def _create_legacy_shelf(path: str, *infos: DownloadInfo) -> None:
with shelve.open(path, "c") as shelf:
for info in infos:
shelf[info.url] = info
class PersistentQueueTests(unittest.TestCase):
def test_put_get_delete_roundtrip(self):
with tempfile.TemporaryDirectory() as tmp:
@@ -42,6 +76,7 @@ class PersistentQueueTests(unittest.TestCase):
pq = PersistentQueue("queue", path)
dl = _FakeDownload(_make_info("http://a.example"))
pq.put(dl)
self.assertTrue(os.path.exists(path + ".json"))
self.assertTrue(pq.exists("http://a.example"))
self.assertFalse(pq.empty())
got = pq.get("http://a.example")
@@ -62,7 +97,7 @@ class PersistentQueueTests(unittest.TestCase):
keys = [k for k, _ in pq.saved_items()]
self.assertEqual(keys, ["http://first.example", "http://second.example"])
def test_load_restores_from_shelve(self):
def test_load_restores_from_json(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue")
pq1 = PersistentQueue("queue", path)
@@ -71,6 +106,186 @@ class PersistentQueueTests(unittest.TestCase):
pq2.load()
self.assertTrue(pq2.exists("http://load.example"))
def test_load_imports_legacy_shelve(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue")
_create_legacy_shelf(path, _make_info("http://legacy.example"))
pq = PersistentQueue("queue", path)
pq.load()
self.assertTrue(pq.exists("http://legacy.example"))
self.assertTrue(os.path.exists(path + ".json"))
def test_queue_persists_only_compact_entry_subset(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue")
pq = PersistentQueue("queue", path)
info = _make_info("http://entry.example")
info.entry = {
"playlist_index": "01",
"playlist_title": "Playlist",
"channel_index": "02",
"channel_title": "Channel",
"formats": [{"id": "huge"}],
"description": "very large payload",
}
pq.put(_FakeDownload(info))
with open(path + ".json", encoding="utf-8") as f:
payload = json.load(f)
record = payload["items"][0]["info"]
self.assertEqual(
record["entry"],
{
"playlist_index": "01",
"playlist_title": "Playlist",
"channel_index": "02",
"channel_title": "Channel",
},
)
self.assertNotIn("formats", record["entry"])
self.assertNotIn("description", record["entry"])
def test_completed_queue_does_not_persist_entry_or_transient_progress(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "completed")
pq = PersistentQueue("completed", path)
info = _make_info("http://done.example")
info.status = "finished"
info.percent = 88
info.speed = 123
info.eta = 9
info.entry = {
"playlist_index": "01",
"playlist_title": "Playlist",
"formats": [{"id": "huge"}],
}
info.filename = "done.mp4"
pq.put(_FakeDownload(info))
with open(path + ".json", encoding="utf-8") as f:
payload = json.load(f)
record = payload["items"][0]["info"]
self.assertNotIn("entry", record)
self.assertNotIn("percent", record)
self.assertNotIn("speed", record)
self.assertNotIn("eta", record)
self.assertEqual(record["filename"], "done.mp4")
def test_invalid_json_is_quarantined_and_legacy_is_imported(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue")
_create_legacy_shelf(path, _make_info("http://legacy.example"))
with open(path + ".json", "w", encoding="utf-8") as f:
f.write("{not valid json")
pq = PersistentQueue("queue", path)
pq.load()
self.assertTrue(pq.exists("http://legacy.example"))
self.assertTrue(
any(name.startswith("queue.json.invalid.") for name in os.listdir(tmp))
)
def test_loading_old_json_rewrites_to_compact_format(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue")
with open(path + ".json", "w", encoding="utf-8") as f:
json.dump(
{
"schema_version": 1,
"kind": "persistent_queue:queue",
"items": [
{
"key": "http://legacy-json.example",
"info": {
"id": "id1",
"title": "Title",
"url": "http://legacy-json.example",
"quality": "best",
"download_type": "video",
"codec": "auto",
"format": "any",
"folder": "",
"custom_name_prefix": "",
"playlist_item_limit": 0,
"split_by_chapters": False,
"chapter_template": "",
"subtitle_language": "en",
"subtitle_mode": "prefer_manual",
"status": "pending",
"timestamp": 1,
"entry": {
"playlist_index": "01",
"playlist_title": "Playlist",
"formats": [{"id": "huge"}],
},
"percent": 15,
"speed": 20,
"eta": 30,
},
}
],
},
f,
)
pq = PersistentQueue("queue", path)
pq.load()
with open(path + ".json", encoding="utf-8") as f:
payload = json.load(f)
record = payload["items"][0]["info"]
self.assertEqual(payload["schema_version"], 2)
self.assertEqual(record["entry"], {"playlist_index": "01", "playlist_title": "Playlist"})
self.assertNotIn("percent", record)
self.assertNotIn("speed", record)
self.assertNotIn("eta", record)
def test_put_rollbacks_in_memory_queue_when_state_write_fails(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue")
pq = PersistentQueue("queue", path)
dl = _FakeDownload(_make_info("http://rollback.example"))
self.assertFalse(pq.exists("http://rollback.example"))
orig_save = __import__("state_store").AtomicJsonStore.save
def bad_save(store, data):
if store.path == path + ".json":
raise OSError("simulated shelf failure")
return orig_save(store, data)
with patch("ytdl.AtomicJsonStore.save", bad_save):
with self.assertRaises(OSError):
pq.put(dl)
self.assertFalse(pq.exists("http://rollback.example"))
def test_put_rollbacks_to_previous_download_when_replace_fails(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue")
pq = PersistentQueue("queue", path)
first = _FakeDownload(_make_info("http://same.example"))
second = _FakeDownload(_make_info("http://same.example"))
second.info.title = "Replaced title"
pq.put(first)
orig_save = __import__("state_store").AtomicJsonStore.save
def bad_save(store, data):
if store.path == path + ".json":
raise OSError("simulated shelf failure")
return orig_save(store, data)
with patch("ytdl.AtomicJsonStore.save", bad_save):
with self.assertRaises(OSError):
pq.put(second)
self.assertEqual(pq.get("http://same.example").info.title, "Title")
if __name__ == "__main__":
unittest.main()
+53
View File
@@ -0,0 +1,53 @@
from __future__ import annotations
import os
import tempfile
import unittest
from datetime import datetime
from state_store import AtomicJsonStore, from_json_compatible, to_json_compatible
class StateStoreTests(unittest.TestCase):
def test_save_and_load_roundtrip(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue.json")
store = AtomicJsonStore(path, kind="persistent_queue:queue")
store.save({"items": [{"key": "a", "info": {"title": "hello"}}]})
payload = store.load()
self.assertEqual(payload["kind"], "persistent_queue:queue")
self.assertEqual(payload["schema_version"], 2)
self.assertEqual(payload["items"][0]["info"]["title"], "hello")
def test_invalid_file_is_quarantined(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue.json")
with open(path, "w", encoding="utf-8") as f:
f.write("{broken")
store = AtomicJsonStore(path, kind="persistent_queue:queue")
payload = store.load()
self.assertIsNone(payload)
self.assertTrue(
any(name.startswith("queue.json.invalid.") for name in os.listdir(tmp))
)
def test_json_compat_helpers_roundtrip_bytes_and_datetime(self):
raw = {
"payload": b"abc",
"timestamp": datetime(2024, 1, 2, 3, 4, 5),
"items": (1, 2, 3),
}
restored = from_json_compatible(to_json_compatible(raw))
self.assertEqual(restored["payload"], b"abc")
self.assertEqual(restored["timestamp"], datetime(2024, 1, 2, 3, 4, 5))
self.assertEqual(restored["items"], [1, 2, 3])
if __name__ == "__main__":
unittest.main()
+443
View File
@@ -0,0 +1,443 @@
from __future__ import annotations
import json
import os
import shelve
import sys
import tempfile
import types
import unittest
from unittest.mock import patch
fake_yt_dlp = types.ModuleType("yt_dlp")
fake_networking = types.ModuleType("yt_dlp.networking")
fake_impersonate = types.ModuleType("yt_dlp.networking.impersonate")
class _ImpersonateTarget:
@staticmethod
def from_str(value):
return value
fake_impersonate.ImpersonateTarget = _ImpersonateTarget
fake_networking.impersonate = fake_impersonate
fake_yt_dlp.networking = fake_networking
fake_yt_dlp.utils = types.SimpleNamespace(YoutubeDLError=Exception)
sys.modules.setdefault("yt_dlp", fake_yt_dlp)
sys.modules.setdefault("yt_dlp.networking", fake_networking)
sys.modules.setdefault("yt_dlp.networking.impersonate", fake_impersonate)
from subscriptions import SubscriptionManager, extract_flat_playlist
class _Config:
def __init__(self, state_dir: str):
self.STATE_DIR = state_dir
self.SUBSCRIPTION_SCAN_PLAYLIST_END = 50
self.SUBSCRIPTION_MAX_SEEN_IDS = 50000
self.DOWNLOAD_DIR = state_dir
self.TEMP_DIR = state_dir
self.YTDL_OPTIONS = {}
class _Queue:
def __init__(self):
self.entries = []
self.fail = False
async def add(self, *args, **kwargs):
return None
async def add_entry(self, entry, *args, **kwargs):
if self.fail:
return {"status": "error", "msg": "queue failed"}
self.entries.append((entry, args, kwargs))
return {"status": "ok"}
class _Notifier:
async def subscription_added(self, sub):
return None
async def subscription_updated(self, sub):
return None
async def subscription_removed(self, sub_id):
return None
async def subscriptions_all(self, subs):
return None
def _create_legacy_shelf(path: str, record) -> None:
with shelve.open(path, "c") as shelf:
shelf["sub-1"] = record
class SubscriptionPersistenceTests(unittest.IsolatedAsyncioTestCase):
def test_load_imports_legacy_subscription_shelf(self):
with tempfile.TemporaryDirectory() as tmp:
legacy_path = os.path.join(tmp, "subscriptions")
json_path = os.path.join(tmp, "subscriptions.json")
_create_legacy_shelf(
legacy_path,
{
"id": "sub-1",
"name": "Channel",
"url": "https://example.com/channel",
"timestamp": 1.0,
},
)
mgr = SubscriptionManager(_Config(tmp), _Queue(), _Notifier())
self.assertEqual(len(mgr.list_all()), 1)
self.assertTrue(os.path.exists(json_path))
with open(json_path, encoding="utf-8") as f:
payload = json.load(f)
self.assertEqual(payload["schema_version"], 2)
self.assertNotIn("timestamp", payload["items"][0])
def test_invalid_json_is_quarantined_and_legacy_is_imported(self):
with tempfile.TemporaryDirectory() as tmp:
legacy_path = os.path.join(tmp, "subscriptions")
json_path = os.path.join(tmp, "subscriptions.json")
_create_legacy_shelf(
legacy_path,
{
"id": "sub-1",
"name": "Channel",
"url": "https://example.com/channel",
"timestamp": 1.0,
},
)
with open(json_path, "w", encoding="utf-8") as f:
f.write("{not valid json")
mgr = SubscriptionManager(_Config(tmp), _Queue(), _Notifier())
self.assertEqual(len(mgr.list_all()), 1)
self.assertTrue(
any(name.startswith("subscriptions.json.invalid.") for name in os.listdir(tmp))
)
def test_load_rewrites_old_json_and_trims_seen_ids(self):
with tempfile.TemporaryDirectory() as tmp:
json_path = os.path.join(tmp, "subscriptions.json")
cfg = _Config(tmp)
cfg.SUBSCRIPTION_MAX_SEEN_IDS = 2
with open(json_path, "w", encoding="utf-8") as f:
json.dump(
{
"schema_version": 1,
"kind": "subscriptions",
"items": [
{
"id": "sub-1",
"name": "Channel",
"url": "https://example.com/channel",
"enabled": True,
"check_interval_minutes": 60,
"download_type": "video",
"codec": "auto",
"format": "any",
"quality": "best",
"folder": "",
"custom_name_prefix": "",
"auto_start": True,
"playlist_item_limit": 0,
"split_by_chapters": False,
"chapter_template": "",
"subtitle_language": "en",
"subtitle_mode": "prefer_manual",
"last_checked": None,
"seen_ids": ["a", "b", "a", "c"],
"error": None,
"timestamp": 123,
}
],
},
f,
)
mgr = SubscriptionManager(cfg, _Queue(), _Notifier())
self.assertEqual(mgr.list_all()[0].seen_ids, ["a", "b"])
with open(json_path, encoding="utf-8") as f:
payload = json.load(f)
self.assertEqual(payload["schema_version"], 2)
self.assertEqual(payload["items"][0]["seen_ids"], ["a", "b"])
self.assertNotIn("timestamp", payload["items"][0])
async def test_add_subscription_rolls_back_when_state_write_fails(self):
with tempfile.TemporaryDirectory() as tmp:
mgr = SubscriptionManager(_Config(tmp), _Queue(), _Notifier())
orig_save = __import__("state_store").AtomicJsonStore.save
def bad_save(store, data):
if store.path == mgr._path:
raise OSError("simulated shelf failure")
return orig_save(store, data)
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "webpage_url": "https://example.com/v1"}],
),
):
with patch("subscriptions.AtomicJsonStore.save", bad_save):
with self.assertRaises(OSError):
await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
self.assertEqual(mgr.list_all(), [])
self.assertNotIn("https://example.com/channel", mgr._url_index)
async def test_add_subscription_marks_existing_videos_seen_without_queueing(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Channel"},
[
{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"},
{"id": "v2", "title": "Two", "webpage_url": "https://example.com/v2"},
{"id": "v3", "title": "Three", "webpage_url": "https://example.com/v3"},
],
),
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
assert result["status"] == "ok"
sub = mgr.list_all()[0]
self.assertEqual(sub.seen_ids, ["v1", "v2", "v3"])
self.assertIsNone(sub.error)
self.assertEqual(queue.entries, [])
async def test_add_subscription_skips_collection_tab_entries(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Channel"},
[
{
"_type": "url",
"ie_key": "YoutubeTab",
"title": "Channel - Live",
"url": "https://example.com/live",
"webpage_url": "https://example.com/live",
},
{
"_type": "url",
"ie_key": "Youtube",
"id": "v1",
"title": "One",
"duration": 10,
"webpage_url": "https://example.com/v1",
},
],
),
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
self.assertEqual(result["status"], "ok")
sub = mgr.list_all()[0]
self.assertEqual(sub.seen_ids, ["v1"])
self.assertEqual(queue.entries, [])
async def test_check_now_keeps_failed_queue_items_unseen_and_sets_error(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
side_effect=[
(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
(
{"_type": "channel", "title": "Channel"},
[{"id": "v2", "title": "Two", "webpage_url": "https://example.com/v2"}],
),
],
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
queue.fail = True
await mgr.check_now([result["subscription"]["id"]])
sub = mgr.list_all()[0]
self.assertEqual(sub.error, "queue failed")
self.assertEqual(sub.seen_ids, ["v1"])
async def test_check_now_queues_new_video_and_updates_seen_ids(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
side_effect=[
(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
(
{"_type": "channel", "title": "Channel"},
[
{"id": "v2", "title": "Two", "webpage_url": "https://example.com/v2"},
{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"},
],
),
],
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
await mgr.check_now([result["subscription"]["id"]])
sub = mgr.list_all()[0]
self.assertIsNotNone(sub.last_checked)
self.assertIsNone(sub.error)
self.assertEqual(sub.seen_ids[:2], ["v2", "v1"])
self.assertEqual([entry["webpage_url"] for entry, _, _ in queue.entries], ["https://example.com/v2"])
class ExtractFlatPlaylistTests(unittest.TestCase):
def test_descends_one_level_when_root_entries_are_nested_collections(self):
responses = iter(
[
{
"_type": "channel",
"entries": [
{
"_type": "url",
"ie_key": "YoutubeTab",
"title": "Channel - Videos",
"url": "https://example.com/videos",
"webpage_url": "https://example.com/videos",
}
],
},
{
"_type": "playlist",
"entries": [
{
"_type": "url",
"ie_key": "Youtube",
"id": "v1",
"title": "One",
"duration": 10,
"webpage_url": "https://example.com/v1",
}
],
},
]
)
class _FakeYDL:
def __init__(self, params):
self.params = params
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
def extract_info(self, url, download=False):
return next(responses)
cfg = _Config(tempfile.mkdtemp())
with patch("subscriptions.yt_dlp.YoutubeDL", _FakeYDL, create=True):
info, entries = extract_flat_playlist(cfg, "https://example.com/channel", 50)
self.assertEqual(info.get("_type"), "playlist")
self.assertEqual([entry["webpage_url"] for entry in entries], ["https://example.com/v1"])
if __name__ == "__main__":
unittest.main()
+178 -11
View File
@@ -2,18 +2,53 @@
from __future__ import annotations
import pickle
import sys
import tempfile
import threading
import types
import unittest
from pathlib import Path
fake_yt_dlp = types.ModuleType("yt_dlp")
fake_networking = types.ModuleType("yt_dlp.networking")
fake_impersonate = types.ModuleType("yt_dlp.networking.impersonate")
fake_utils = types.ModuleType("yt_dlp.utils")
class _ImpersonateTarget:
@staticmethod
def from_str(value):
return value
fake_impersonate.ImpersonateTarget = _ImpersonateTarget
fake_networking.impersonate = fake_impersonate
# The inner ``key`` group mirrors the real ``STR_FORMAT_RE_TMPL`` so that
# ``_OUTTMPL_FIELD_RE`` (compiled at import time) has the named group that
# ``_resolve_outtmpl_fields`` reads via ``match.group('key')``.
fake_utils.STR_FORMAT_RE_TMPL = r"(?P<prefix>)%\((?P<has_key>(?P<key>{}))\)(?P<format>[-0-9.]*{})"
fake_utils.STR_FORMAT_TYPES = "diouxXeEfFgGcrsa"
fake_yt_dlp.networking = fake_networking
fake_yt_dlp.utils = fake_utils
sys.modules.setdefault("yt_dlp", fake_yt_dlp)
sys.modules.setdefault("yt_dlp.networking", fake_networking)
sys.modules.setdefault("yt_dlp.networking.impersonate", fake_impersonate)
sys.modules.setdefault("yt_dlp.utils", fake_utils)
from ytdl import (
DownloadInfo,
_convert_generators_to_lists,
_compact_persisted_entry,
_convert_srt_to_txt_file,
_outtmpl_substitute_field,
_resolve_outtmpl_fields,
_sanitize_entry_for_pickle,
_sanitize_path_component,
)
# Detect whether the real yt-dlp is loaded (as opposed to the minimal fake
# shim above). _resolve_outtmpl_fields needs YoutubeDL at runtime.
_has_real_ytdlp = hasattr(sys.modules.get("yt_dlp"), "YoutubeDL")
class SanitizePathComponentTests(unittest.TestCase):
def test_replaces_windows_invalid_chars(self):
@@ -24,28 +59,105 @@ class SanitizePathComponentTests(unittest.TestCase):
self.assertEqual(_sanitize_path_component(42), 42)
class OuttmplSubstituteFieldTests(unittest.TestCase):
def test_simple_substitution(self):
self.assertEqual(_outtmpl_substitute_field("%(title)s", "title", "Hello"), "Hello")
@unittest.skipUnless(_has_real_ytdlp, "requires real yt-dlp")
class ResolveOuttmplFieldsTests(unittest.TestCase):
"""Tests for _resolve_outtmpl_fields (delegates to yt-dlp's template engine)."""
def test_simple_playlist_substitution(self):
info = {"playlist_title": "My PL", "playlist_index": "03"}
result = _resolve_outtmpl_fields("%(playlist_title)s/%(title)s.%(ext)s", info, ("playlist",))
self.assertEqual(result, "My PL/%(title)s.%(ext)s")
def test_format_spec_int(self):
self.assertEqual(_outtmpl_substitute_field("%(idx)02d", "idx", 3), "03")
info = {"playlist_index": "3"}
result = _resolve_outtmpl_fields("%(playlist_index)02d-%(title)s", info, ("playlist",))
self.assertEqual(result, "03-%(title)s")
def test_missing_field_unchanged(self):
self.assertEqual(_outtmpl_substitute_field("%(other)s", "title", "x"), "%(other)s")
def test_non_targeted_fields_unchanged(self):
info = {"playlist_title": "PL"}
result = _resolve_outtmpl_fields("%(title)s/%(ext)s", info, ("playlist",))
self.assertEqual(result, "%(title)s/%(ext)s")
def test_default_value(self):
info = {"playlist_index": "1"}
result = _resolve_outtmpl_fields("%(playlist_title|Unknown)s/%(playlist_index)s", info, ("playlist",))
self.assertEqual(result, "Unknown/1")
def test_channel_prefix(self):
info = {"channel": "MyChan", "channel_index": "05"}
result = _resolve_outtmpl_fields("%(channel)s/%(channel_index)02d-%(title)s", info, ("channel",))
self.assertEqual(result, "MyChan/05-%(title)s")
def test_math_operation(self):
info = {"playlist_index": "3"}
result = _resolve_outtmpl_fields("%(playlist_index+100)d", info, ("playlist",))
self.assertEqual(result, "103")
def test_playlist_count_and_autonumber(self):
info = {
"playlist_title": "My PL",
"playlist_index": "03",
"playlist_count": 10,
"playlist_autonumber": 3,
"n_entries": 10,
"__last_playlist_index": 10,
}
result = _resolve_outtmpl_fields(
"%(playlist_title)s/%(playlist_autonumber)s of %(playlist_count)s - %(title)s.%(ext)s",
info,
("playlist",),
)
# playlist_autonumber is auto-padded by yt-dlp using __last_playlist_index
self.assertEqual(result, "My PL/03 of 10 - %(title)s.%(ext)s")
def test_conditional_playlist_index(self):
info = {
"playlist_index": "5",
"playlist_count": 10,
}
result = _resolve_outtmpl_fields(
"%(playlist_index&{} - |)s%(title)s.%(ext)s",
info,
("playlist",),
)
self.assertEqual(result, "5 - %(title)s.%(ext)s")
class ConvertGeneratorsToListsTests(unittest.TestCase):
class SanitizeEntryForPickleTests(unittest.TestCase):
def test_nested(self):
def g():
yield 1
obj = {"a": g(), "b": [g()]}
out = _convert_generators_to_lists(obj)
out = _sanitize_entry_for_pickle(obj)
self.assertEqual(out, {"a": [1], "b": [[1]]})
pickle.dumps(out)
def test_plain(self):
self.assertEqual(_convert_generators_to_lists(5), 5)
self.assertEqual(_sanitize_entry_for_pickle(5), 5)
def test_set_converted_to_list(self):
obj = {"s": {1, 2}}
out = _sanitize_entry_for_pickle(obj)
self.assertEqual(sorted(out["s"]), [1, 2])
pickle.dumps(out)
def test_map_iterator(self):
out = _sanitize_entry_for_pickle({"m": map(int, ["1", "2"])})
self.assertEqual(out, {"m": [1, 2]})
def test_lock_replaced_with_none(self):
lock = threading.Lock()
out = _sanitize_entry_for_pickle({"k": lock})
self.assertIsNone(out["k"])
pickle.dumps(out)
def test_ordered_dict(self):
from collections import OrderedDict
od = OrderedDict([("z", 1), ("a", 2)])
out = _sanitize_entry_for_pickle(od)
self.assertEqual(out, {"z": 1, "a": 2})
class ConvertSrtToTxtTests(unittest.TestCase):
@@ -141,6 +253,61 @@ class DownloadInfoSetstateTests(unittest.TestCase):
di.__setstate__(state)
self.assertEqual(di.subtitle_files, [])
def test_missing_optional_fields_are_defaulted(self):
state = self._base_state(
download_type="video",
codec="auto",
format="any",
quality="best",
)
state.pop("folder")
state.pop("custom_name_prefix")
state.pop("playlist_item_limit")
state.pop("split_by_chapters")
state.pop("chapter_template")
di = DownloadInfo.__new__(DownloadInfo)
di.__setstate__(state)
self.assertEqual(di.folder, "")
self.assertEqual(di.custom_name_prefix, "")
self.assertEqual(di.playlist_item_limit, 0)
self.assertFalse(di.split_by_chapters)
self.assertEqual(di.chapter_template, "")
class CompactPersistedEntryTests(unittest.TestCase):
def test_keeps_only_playlist_and_channel_keys(self):
entry = {
"playlist_index": "01",
"playlist_title": "Playlist",
"playlist_count": 10,
"playlist_autonumber": 1,
"channel_index": "02",
"channel_title": "Channel",
"n_entries": 10,
"__last_playlist_index": 10,
"formats": [{"id": "huge"}],
"description": "big blob",
}
compact = _compact_persisted_entry(entry)
self.assertEqual(
compact,
{
"playlist_index": "01",
"playlist_title": "Playlist",
"playlist_count": 10,
"playlist_autonumber": 1,
"channel_index": "02",
"channel_title": "Channel",
"n_entries": 10,
"__last_playlist_index": 10,
},
)
def test_returns_none_when_no_restart_relevant_keys_exist(self):
self.assertIsNone(_compact_persisted_entry({"id": "x", "title": "y"}))
if __name__ == "__main__":
unittest.main()
+298 -152
View File
@@ -1,34 +1,28 @@
import os
import shutil
import yt_dlp
import collections
import collections.abc
import copy
import pickle
from collections import OrderedDict
import shelve
import time
import asyncio
import multiprocessing
import logging
import re
import types
import dbm
import subprocess
from typing import Any
from functools import lru_cache
from typing import Any, Optional
import yt_dlp.networking.impersonate
from yt_dlp.utils import STR_FORMAT_RE_TMPL, STR_FORMAT_TYPES
from dl_formats import get_format, get_opts, AUDIO_FORMATS
from datetime import datetime
from state_store import AtomicJsonStore, from_json_compatible, read_legacy_shelf, to_json_compatible
log = logging.getLogger('ytdl')
@lru_cache(maxsize=None)
def _compile_outtmpl_pattern(field: str) -> re.Pattern:
"""Compile a regex pattern to match a specific field in an output template, including optional format specifiers."""
conversion_types = f"[{re.escape(STR_FORMAT_TYPES)}]"
return re.compile(STR_FORMAT_RE_TMPL.format(re.escape(field), conversion_types))
# Characters that are invalid in Windows/NTFS path components. These are pre-
# sanitised when substituting playlist/channel titles into output templates so
# that downloads do not fail on NTFS-mounted volumes or Windows Docker hosts.
@@ -39,55 +33,86 @@ def _sanitize_path_component(value: Any) -> Any:
"""Replace characters that are invalid in Windows path components with '_'.
Non-string values (int, float, None, …) are passed through unchanged so
that ``_outtmpl_substitute_field`` can still coerce them with format specs
(e.g. ``%(playlist_index)02d``). Only string values are sanitised because
Windows-invalid characters are only a concern for human-readable strings
(titles, channel names, etc.) that may end up as directory names.
that numeric format specs (e.g. ``%(playlist_index)02d``) still work.
Only string values are sanitised because Windows-invalid characters are
only a concern for human-readable strings (titles, channel names, etc.)
that may end up as directory names.
"""
if not isinstance(value, str):
return value
return _WINDOWS_INVALID_PATH_CHARS.sub('_', value)
def _outtmpl_substitute_field(template: str, field: str, value: Any) -> str:
"""Substitute a single field in an output template, applying any format specifiers to the value."""
pattern = _compile_outtmpl_pattern(field)
# Regex matching yt-dlp output-template field references, e.g. ``%(title)s``
# or ``%(playlist_index)03d``. Built from yt-dlp's own ``STR_FORMAT_RE_TMPL``
# so that it stays in sync with upstream changes to the template syntax.
_OUTTMPL_FIELD_RE = re.compile(
STR_FORMAT_RE_TMPL.format('[^)]+', f'[{STR_FORMAT_TYPES}ljhqBUDS]')
)
def replacement(match: re.Match) -> str:
if match.group("has_key") is None:
return match.group(0)
prefix = match.group("prefix") or ""
format_spec = match.group("format")
def _resolve_outtmpl_fields(template: str, info_dict: dict, prefixes: tuple[str, ...]) -> str:
"""Resolve specific fields in an output template using yt-dlp's template engine.
if not format_spec:
return f"{prefix}{value}"
Only field references whose root name starts with one of *prefixes* are
evaluated. All other references are left untouched so that yt-dlp can
resolve them later during the actual download.
conversion_type = format_spec[-1]
try:
if conversion_type in "diouxX":
coerced_value = int(value)
elif conversion_type in "eEfFgG":
coerced_value = float(value)
else:
coerced_value = value
This delegates to ``YoutubeDL.evaluate_outtmpl`` for each targeted field
reference, giving access to the full yt-dlp template syntax (defaults,
conditional formatting, math operations, datetime formatting, etc.).
"""
matches = list(_OUTTMPL_FIELD_RE.finditer(template))
if not matches:
return template
return f"{prefix}{('%' + format_spec) % coerced_value}"
except (ValueError, TypeError):
return f"{prefix}{value}"
with yt_dlp.YoutubeDL({'quiet': True}) as ydl:
for match in reversed(matches):
key = match.group('key')
if key is None:
continue
root = re.match(r'\w+', key)
if root is None or not root.group(0).startswith(prefixes):
continue
resolved = ydl.evaluate_outtmpl(match.group(0), info_dict)
template = template[:match.start()] + resolved + template[match.end():]
return pattern.sub(replacement, template)
return template
def _convert_generators_to_lists(obj):
"""Recursively convert generators to lists in a dictionary to make it pickleable."""
if isinstance(obj, types.GeneratorType):
return list(obj)
elif isinstance(obj, dict):
return {k: _convert_generators_to_lists(v) for k, v in obj.items()}
elif isinstance(obj, (list, tuple)):
return type(obj)(_convert_generators_to_lists(item) for item in obj)
else:
_MAX_ENTRY_SANITIZE_DEPTH = 64
def _sanitize_entry_for_pickle(obj, _depth=0):
"""Recursively normalize yt-dlp ``info_dict`` data so it can be stored in shelve/pickle.
Live streams and newer yt-dlp versions may nest generators, iterators, sets, or
non-serializable objects (e.g. locks) inside the extracted metadata. The previous
helper only walked plain dict/list/tuple and only expanded ``types.GeneratorType``.
"""
if _depth > _MAX_ENTRY_SANITIZE_DEPTH:
return None
if obj is None or isinstance(obj, (bool, int, float, str, bytes)):
return obj
if isinstance(obj, types.GeneratorType):
return _sanitize_entry_for_pickle(list(obj), _depth + 1)
if isinstance(obj, collections.abc.Mapping):
return {k: _sanitize_entry_for_pickle(v, _depth + 1) for k, v in obj.items()}
if isinstance(obj, (list, tuple)):
return type(obj)(_sanitize_entry_for_pickle(x, _depth + 1) for x in obj)
if isinstance(obj, (set, frozenset)):
return [_sanitize_entry_for_pickle(x, _depth + 1) for x in obj]
if isinstance(obj, collections.deque):
return [_sanitize_entry_for_pickle(x, _depth + 1) for x in obj]
if isinstance(obj, collections.abc.Iterator):
try:
return _sanitize_entry_for_pickle(list(obj), _depth + 1)
except Exception:
return None
try:
pickle.dumps(obj, protocol=pickle.HIGHEST_PROTOCOL)
return obj
except Exception:
return None
def _convert_srt_to_txt_file(subtitle_path: str):
@@ -178,8 +203,8 @@ class DownloadInfo:
self.size = None
self.timestamp = time.time_ns()
self.error = error
# Convert generators to lists to make entry pickleable
self.entry = _convert_generators_to_lists(entry) if entry is not None else None
# Strip non-pickleable values (generators, iterators, locks, etc.) for shelve
self.entry = _sanitize_entry_for_pickle(entry) if entry is not None else None
self.playlist_item_limit = playlist_item_limit
self.split_by_chapters = split_by_chapters
self.chapter_template = chapter_template
@@ -223,8 +248,103 @@ class DownloadInfo:
if not getattr(self, "codec", None):
self.codec = "auto"
if not hasattr(self, "folder"):
self.folder = ""
if not hasattr(self, "custom_name_prefix"):
self.custom_name_prefix = ""
if not hasattr(self, "playlist_item_limit"):
self.playlist_item_limit = 0
if not hasattr(self, "split_by_chapters"):
self.split_by_chapters = False
if not hasattr(self, "chapter_template"):
self.chapter_template = ""
if not hasattr(self, "subtitle_language"):
self.subtitle_language = "en"
if not hasattr(self, "subtitle_mode"):
self.subtitle_mode = "prefer_manual"
if not hasattr(self, "entry"):
self.entry = None
if not hasattr(self, "subtitle_files"):
self.subtitle_files = []
if not hasattr(self, "chapter_files"):
self.chapter_files = []
_PERSISTED_DOWNLOAD_FIELDS = (
"id",
"title",
"url",
"quality",
"download_type",
"codec",
"format",
"folder",
"custom_name_prefix",
"playlist_item_limit",
"split_by_chapters",
"chapter_template",
"subtitle_language",
"subtitle_mode",
"status",
"timestamp",
"error",
"msg",
"filename",
"size",
"chapter_files",
)
_COMPACT_ENTRY_EXTRA_KEYS = frozenset(("n_entries", "__last_playlist_index"))
def _compact_persisted_entry(entry: Any) -> Optional[dict[str, Any]]:
if not isinstance(entry, dict):
return None
compact = {
key: value
for key, value in entry.items()
if key.startswith("playlist") or key.startswith("channel") or key in _COMPACT_ENTRY_EXTRA_KEYS
}
return compact or None
def _download_info_to_record(
info: DownloadInfo,
*,
include_entry: bool,
) -> dict[str, Any]:
record: dict[str, Any] = {}
for key in _PERSISTED_DOWNLOAD_FIELDS:
if hasattr(info, key):
value = getattr(info, key)
if value is not None:
record[key] = to_json_compatible(value)
if include_entry:
compact_entry = _compact_persisted_entry(getattr(info, "entry", None))
if compact_entry is not None:
record["entry"] = to_json_compatible(compact_entry)
return record
def _download_info_from_record(record: dict[str, Any]) -> DownloadInfo:
info = DownloadInfo.__new__(DownloadInfo)
info.__setstate__({key: from_json_compatible(value) for key, value in record.items()})
if not hasattr(info, "msg"):
info.msg = None
if not hasattr(info, "percent"):
info.percent = None
if not hasattr(info, "speed"):
info.speed = None
if not hasattr(info, "eta"):
info.eta = None
if not hasattr(info, "status"):
info.status = "pending"
if not hasattr(info, "size"):
info.size = None
if not hasattr(info, "error"):
info.error = None
return info
class Download:
manager = None
@@ -475,11 +595,9 @@ class PersistentQueue:
pdir = os.path.dirname(path)
if not os.path.isdir(pdir):
os.mkdir(pdir)
with shelve.open(path, 'c'):
pass
self.path = path
self.repair()
self.legacy_path = path
self.path = f"{path}.json"
self.store = AtomicJsonStore(self.path, kind=f"persistent_queue:{name}")
self.dict = OrderedDict()
def load(self):
@@ -496,20 +614,91 @@ class PersistentQueue:
return self.dict.items()
def saved_items(self):
with shelve.open(self.path, 'r') as shelf:
return sorted(shelf.items(), key=lambda item: item[1].timestamp)
items = [
(item["key"], _download_info_from_record(item["info"]))
for item in self._load_state_items()
]
return sorted(items, key=lambda item: item[1].timestamp)
def _should_persist_entry(self) -> bool:
return self.identifier != "completed"
def _serialize_items(self):
return [
{
"key": key,
"info": _download_info_to_record(
download.info,
include_entry=self._should_persist_entry(),
),
}
for key, download in self.dict.items()
]
def _save_dict(self):
self.store.save({"items": self._serialize_items()})
def _load_state_items(self):
payload = self.store.load()
if payload is not None:
items = payload.get("items")
if isinstance(items, list):
compact_items = [
{
"key": item["key"],
"info": _download_info_to_record(
_download_info_from_record(item["info"]),
include_entry=self._should_persist_entry(),
),
}
for item in items
if isinstance(item, dict) and "key" in item and "info" in item
]
if payload.get("schema_version") != self.store.schema_version or compact_items != items:
self.store.save({"items": compact_items})
return compact_items
log.warning("PersistentQueue:%s state file did not contain an items list", self.identifier)
return []
legacy_items = read_legacy_shelf(self.legacy_path)
if legacy_items is None:
return []
items = [
{
"key": key,
"info": _download_info_to_record(
value,
include_entry=self._should_persist_entry(),
),
}
for key, value in sorted(legacy_items, key=lambda item: item[1].timestamp)
]
self.store.save({"items": items})
return items
def put(self, value):
key = value.info.url
old = self.dict.get(key)
self.dict[key] = value
with shelve.open(self.path, 'w') as shelf:
shelf[key] = value.info
try:
self._save_dict()
except Exception:
if old is None:
del self.dict[key]
else:
self.dict[key] = old
raise
def delete(self, key):
if key in self.dict:
old = self.dict[key]
del self.dict[key]
with shelve.open(self.path, 'w') as shelf:
shelf.pop(key, None)
try:
self._save_dict()
except Exception:
self.dict[key] = old
raise
def next(self):
k, v = next(iter(self.dict.items()))
@@ -518,90 +707,6 @@ class PersistentQueue:
def empty(self):
return not bool(self.dict)
def repair(self):
# check DB format
type_check = subprocess.run(
["file", self.path],
capture_output=True,
text=True
)
db_type = type_check.stdout.lower()
# create backup (<queue>.old)
try:
shutil.copy2(self.path, f"{self.path}.old")
except Exception as e:
# if we cannot backup then its not safe to attempt a repair
# since it could be due to a filesystem error
log.debug(f"PersistentQueue:{self.identifier} backup failed, skipping repair")
return
if "gnu dbm" in db_type:
# perform gdbm repair
log_prefix = f"PersistentQueue:{self.identifier} repair (dbm/file)"
log.debug(f"{log_prefix} started")
try:
result = subprocess.run(
["gdbmtool", self.path],
input="recover verbose summary\n",
text=True,
capture_output=True,
timeout=60
)
log.debug(f"{log_prefix} {result.stdout}")
if result.stderr:
log.debug(f"{log_prefix} failed: {result.stderr}")
except FileNotFoundError:
log.debug(f"{log_prefix} failed: 'gdbmtool' was not found")
# perform null key cleanup
log_prefix = f"PersistentQueue:{self.identifier} repair (null keys)"
log.debug(f"{log_prefix} started")
deleted = 0
try:
with dbm.open(self.path, "w") as db:
for key in list(db.keys()):
if len(key) > 0 and all(b == 0x00 for b in key):
log.debug(f"{log_prefix} deleting key of length {len(key)} (all NUL bytes)")
del db[key]
deleted += 1
log.debug(f"{log_prefix} done - deleted {deleted} key(s)")
except dbm.error:
log.debug(f"{log_prefix} failed: db type is dbm.gnu, but the module is not available (dbm.error; module support may be missing or the file may be corrupted)")
elif "sqlite" in db_type:
# perform sqlite3 recovery
log_prefix = f"PersistentQueue:{self.identifier} repair (sqlite3/file)"
log.debug(f"{log_prefix} started")
try:
recover_proc = subprocess.Popen(
["sqlite3", self.path, ".recover"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
run_result = subprocess.run(
["sqlite3", f"{self.path}.tmp"],
stdin=recover_proc.stdout,
capture_output=True,
text=True,
timeout=60,
)
if recover_proc.stdout is not None:
recover_proc.stdout.close()
recover_stderr = recover_proc.stderr.read() if recover_proc.stderr is not None else ""
recover_proc.wait(timeout=60)
if run_result.stderr or recover_stderr:
error_text = " ".join(part for part in [recover_stderr.strip(), run_result.stderr.strip()] if part)
log.debug(f"{log_prefix} failed: {error_text}")
else:
shutil.move(f"{self.path}.tmp", self.path)
log.debug(f"{log_prefix}{run_result.stdout or ' was successful, no output'}")
except FileNotFoundError:
log.debug(f"{log_prefix} failed: 'sqlite3' was not found")
except subprocess.TimeoutExpired:
log.debug(f"{log_prefix} failed: sqlite recovery timed out")
class DownloadQueue:
def __init__(self, config, notifier):
self.config = config
@@ -715,15 +820,13 @@ class DownloadQueue:
if entry is not None and entry.get('playlist_index') is not None:
if len(self.config.OUTPUT_TEMPLATE_PLAYLIST):
output = self.config.OUTPUT_TEMPLATE_PLAYLIST
for property, value in entry.items():
if property.startswith("playlist"):
output = _outtmpl_substitute_field(output, property, _sanitize_path_component(value))
sanitized = {k: _sanitize_path_component(v) for k, v in entry.items()}
output = _resolve_outtmpl_fields(output, sanitized, ('playlist',))
if entry is not None and entry.get('channel_index') is not None:
if len(self.config.OUTPUT_TEMPLATE_CHANNEL):
output = self.config.OUTPUT_TEMPLATE_CHANNEL
for property, value in entry.items():
if property.startswith("channel"):
output = _outtmpl_substitute_field(output, property, _sanitize_path_component(value))
sanitized = {k: _sanitize_path_component(v) for k, v in entry.items()}
output = _resolve_outtmpl_fields(output, sanitized, ('channel',))
ytdl_options = dict(self.config.YTDL_OPTIONS)
playlist_item_limit = getattr(dl, 'playlist_item_limit', 0)
if playlist_item_limit > 0:
@@ -793,8 +896,9 @@ class DownloadQueue:
# Convert generator to list if needed (for len() and slicing operations)
if isinstance(entries, types.GeneratorType):
entries = list(entries)
log.info(f'{etype} detected with {len(entries)} entries')
index_digits = len(str(len(entries)))
total_entries = len(entries)
log.info(f'{etype} detected with {total_entries} entries')
index_digits = len(str(total_entries))
results = []
if playlist_item_limit > 0:
log.info(f'Item limit is set. Processing only first {playlist_item_limit} entries')
@@ -806,6 +910,12 @@ class DownloadQueue:
etr["_type"] = "video"
etr[etype] = entry.get("id") or entry.get("channel_id") or entry.get("channel")
etr[f"{etype}_index"] = '{{0:0{0:d}d}}'.format(index_digits).format(index)
etr[f"{etype}_count"] = total_entries
etr[f"{etype}_autonumber"] = index
# n_entries: standard yt-dlp field for total count (used by template engine)
# __last_playlist_index: yt-dlp internal field for auto-padding autonumber
etr["n_entries"] = total_entries
etr["__last_playlist_index"] = total_entries
for property in ("id", "title", "uploader", "uploader_id"):
if property in entry:
etr[f"{etype}_{property}"] = entry[property]
@@ -914,6 +1024,42 @@ class DownloadQueue:
_add_gen,
)
async def add_entry(
self,
entry,
download_type,
codec,
format,
quality,
folder,
custom_name_prefix,
playlist_item_limit,
auto_start=True,
split_by_chapters=False,
chapter_template=None,
subtitle_language="en",
subtitle_mode="prefer_manual",
):
normalized_entry = copy.deepcopy(entry) if isinstance(entry, dict) else entry
already = set()
return await self.__add_entry(
normalized_entry,
download_type,
codec,
format,
quality,
folder,
custom_name_prefix,
playlist_item_limit,
auto_start,
split_by_chapters,
chapter_template,
subtitle_language,
subtitle_mode,
already,
None,
)
async def start_pending(self, ids):
for id in ids:
if not self.pending.exists(id):
+14 -14
View File
@@ -23,21 +23,21 @@
},
"private": true,
"dependencies": {
"@angular/animations": "^21.2.5",
"@angular/common": "^21.2.5",
"@angular/compiler": "^21.2.5",
"@angular/core": "^21.2.5",
"@angular/forms": "^21.2.5",
"@angular/platform-browser": "^21.2.5",
"@angular/platform-browser-dynamic": "^21.2.5",
"@angular/service-worker": "^21.2.5",
"@angular/animations": "^21.2.6",
"@angular/common": "^21.2.6",
"@angular/compiler": "^21.2.6",
"@angular/core": "^21.2.6",
"@angular/forms": "^21.2.6",
"@angular/platform-browser": "^21.2.6",
"@angular/platform-browser-dynamic": "^21.2.6",
"@angular/service-worker": "^21.2.6",
"@fortawesome/angular-fontawesome": "~4.0.0",
"@fortawesome/fontawesome-svg-core": "^7.2.0",
"@fortawesome/free-brands-svg-icons": "^7.2.0",
"@fortawesome/free-regular-svg-icons": "^7.2.0",
"@fortawesome/free-solid-svg-icons": "^7.2.0",
"@ng-bootstrap/ng-bootstrap": "^20.0.0",
"@ng-select/ng-select": "^21.5.2",
"@ng-select/ng-select": "^21.7.0",
"@popperjs/core": "^2.11.8",
"bootstrap": "^5.3.8",
"ngx-cookie-service": "^21.3.1",
@@ -48,16 +48,16 @@
},
"devDependencies": {
"@angular-eslint/builder": "21.1.0",
"@angular/build": "^21.2.3",
"@angular/cli": "^21.2.3",
"@angular/compiler-cli": "^21.2.5",
"@angular/localize": "^21.2.5",
"@angular/build": "^21.2.5",
"@angular/cli": "^21.2.5",
"@angular/compiler-cli": "^21.2.6",
"@angular/localize": "^21.2.6",
"@eslint/js": "^9.39.4",
"angular-eslint": "21.1.0",
"eslint": "^9.39.4",
"jsdom": "^27.4.0",
"typescript": "~5.9.3",
"typescript-eslint": "8.47.0",
"vitest": "^4.1.0"
"vitest": "^4.1.2"
}
}
+493 -482
View File
File diff suppressed because it is too large Load Diff
+200 -31
View File
@@ -89,15 +89,7 @@
<!-- Main URL Input with Download Button -->
<div class="row mb-4">
<div class="col">
<div class="input-group input-group-lg shadow-sm">
<input type="text"
autocomplete="off"
spellcheck="false"
class="form-control form-control-lg"
placeholder="Enter video, channel, or playlist URL"
name="addUrl"
[(ngModel)]="addUrl"
[disabled]="addInProgress || downloads.loading">
<ng-template #urlBarActions>
@if (addInProgress && cancelRequested) {
<button class="btn btn-warning btn-lg px-3" type="button" disabled>
<span class="spinner-border spinner-border-sm me-1" role="status"></span>
@@ -115,13 +107,54 @@
title="Cancel adding URL">
<fa-icon [icon]="faTimesCircle" class="me-1" /> Cancel
</button>
} @else if (subscribeInProgress) {
<button class="btn btn-primary btn-lg px-4" type="button" disabled>
Download
</button>
<button class="btn btn-outline-secondary btn-lg px-3" type="button" disabled>
<span class="spinner-border spinner-border-sm me-2" role="status"></span>
Subscribing...
</button>
} @else {
<button class="btn btn-primary btn-lg px-4" type="submit"
(click)="addDownload()"
[disabled]="downloads.loading">
Download
</button>
<button class="btn btn-outline-secondary btn-lg px-3" type="button"
(click)="addSubscription()"
[disabled]="downloads.loading">
Subscribe
</button>
}
</ng-template>
<!-- Narrow viewports: full-width field, then Bootstrap btn-group (no faux input-group strip) -->
<div class="vstack gap-2 d-md-none">
<input type="text"
autocomplete="off"
spellcheck="false"
class="form-control form-control-lg"
placeholder="Enter video, channel, or playlist URL"
[(ngModel)]="addUrl"
[ngModelOptions]="{standalone: true}"
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
<div class="btn-group w-100" role="group" aria-label="Download or subscribe">
<ng-container [ngTemplateOutlet]="urlBarActions" />
</div>
</div>
<!-- md and up: standard input-group so Bootstrap handles fused borders -->
<div class="input-group input-group-lg shadow-sm d-none d-md-flex">
<input type="text"
autocomplete="off"
spellcheck="false"
class="form-control form-control-lg"
placeholder="Enter video, channel, or playlist URL"
[(ngModel)]="addUrl"
[ngModelOptions]="{standalone: true}"
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
<ng-container [ngTemplateOutlet]="urlBarActions" />
</div>
</div>
</div>
@@ -136,7 +169,7 @@
name="downloadType"
[(ngModel)]="downloadType"
(change)="downloadTypeChanged()"
[disabled]="addInProgress || downloads.loading">
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
@for (type of downloadTypes; track type.id) {
<option [ngValue]="type.id">{{ type.text }}</option>
}
@@ -150,7 +183,7 @@
name="codec"
[(ngModel)]="codec"
(change)="codecChanged()"
[disabled]="addInProgress || downloads.loading">
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
@for (vc of videoCodecs; track vc.id) {
<option [ngValue]="vc.id">{{ vc.text }}</option>
}
@@ -164,7 +197,7 @@
name="format"
[(ngModel)]="format"
(change)="formatChanged()"
[disabled]="addInProgress || downloads.loading">
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
@for (f of formatOptions; track f.id) {
<option [ngValue]="f.id">{{ f.text }}</option>
}
@@ -178,7 +211,7 @@
name="quality"
[(ngModel)]="quality"
(change)="qualityChanged()"
[disabled]="addInProgress || downloads.loading || !showQualitySelector()">
[disabled]="addInProgress || subscribeInProgress || downloads.loading || !showQualitySelector()">
@for (q of qualities; track q.id) {
<option [ngValue]="q.id">{{ q.text }}</option>
}
@@ -193,7 +226,7 @@
name="downloadType"
[(ngModel)]="downloadType"
(change)="downloadTypeChanged()"
[disabled]="addInProgress || downloads.loading">
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
@for (type of downloadTypes; track type.id) {
<option [ngValue]="type.id">{{ type.text }}</option>
}
@@ -207,7 +240,7 @@
name="format"
[(ngModel)]="format"
(change)="formatChanged()"
[disabled]="addInProgress || downloads.loading">
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
@for (f of formatOptions; track f.id) {
<option [ngValue]="f.id">{{ f.text }}</option>
}
@@ -221,7 +254,7 @@
name="quality"
[(ngModel)]="quality"
(change)="qualityChanged()"
[disabled]="addInProgress || downloads.loading">
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
@for (q of qualities; track q.id) {
<option [ngValue]="q.id">{{ q.text }}</option>
}
@@ -229,28 +262,29 @@
</div>
</div>
} @else if (downloadType === 'captions') {
<div class="col-md-3">
<!-- 4× col-md-3 is too tight at ~768px (long addons wrap the 4th field); 2×2 mdlg, one row lg+ -->
<div class="col-12 col-md-6 col-lg-3">
<div class="input-group">
<span class="input-group-text">Type</span>
<select class="form-select"
name="downloadType"
[(ngModel)]="downloadType"
(change)="downloadTypeChanged()"
[disabled]="addInProgress || downloads.loading">
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
@for (type of downloadTypes; track type.id) {
<option [ngValue]="type.id">{{ type.text }}</option>
}
</select>
</div>
</div>
<div class="col-md-3">
<div class="col-12 col-md-6 col-lg-3">
<div class="input-group">
<span class="input-group-text">Format</span>
<select class="form-select"
name="format"
[(ngModel)]="format"
(change)="formatChanged()"
[disabled]="addInProgress || downloads.loading"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="Subtitle output format for captions mode">
@for (f of formatOptions; track f.id) {
<option [ngValue]="f.id">{{ f.text }}</option>
@@ -258,7 +292,7 @@
</select>
</div>
</div>
<div class="col-md-3">
<div class="col-12 col-md-6 col-lg-3">
<div class="input-group">
<span class="input-group-text">Language</span>
<input class="form-control"
@@ -267,7 +301,7 @@
name="subtitleLanguage"
[(ngModel)]="subtitleLanguage"
(change)="subtitleLanguageChanged()"
[disabled]="addInProgress || downloads.loading"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
placeholder="e.g. en, es, zh-Hans"
ngbTooltip="Subtitle language (you can type any language code)">
<datalist id="subtitleLanguageOptions">
@@ -277,14 +311,14 @@
</datalist>
</div>
</div>
<div class="col-md-3">
<div class="col-12 col-md-6 col-lg-3">
<div class="input-group">
<span class="input-group-text">Subtitle Source</span>
<select class="form-select"
name="subtitleMode"
[(ngModel)]="subtitleMode"
(change)="subtitleModeChanged()"
[disabled]="addInProgress || downloads.loading"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="Choose manual, auto, or fallback preference for captions mode">
@for (mode of subtitleModes; track mode.id) {
<option [ngValue]="mode.id">{{ mode.text }}</option>
@@ -300,7 +334,7 @@
name="downloadType"
[(ngModel)]="downloadType"
(change)="downloadTypeChanged()"
[disabled]="addInProgress || downloads.loading">
[disabled]="addInProgress || subscribeInProgress || downloads.loading">
@for (type of downloadTypes; track type.id) {
<option [ngValue]="type.id">{{ type.text }}</option>
}
@@ -345,7 +379,7 @@
name="autoStart"
[(ngModel)]="autoStart"
(change)="autoStartChanged()"
[disabled]="addInProgress || downloads.loading"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="Automatically start downloads when added">
<option [ngValue]="true">Yes</option>
<option [ngValue]="false">No</option>
@@ -362,7 +396,7 @@
addTagText="Create directory"
bindLabel="folder"
[(ngModel)]="folder"
[disabled]="addInProgress || downloads.loading"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
[virtualScroll]="true"
[clearable]="true"
[loading]="downloads.loading"
@@ -381,7 +415,7 @@
placeholder="Default"
name="customNamePrefix"
[(ngModel)]="customNamePrefix"
[disabled]="addInProgress || downloads.loading"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="Add a prefix to downloaded filenames">
</div>
</div>
@@ -395,17 +429,31 @@
name="playlistItemLimit"
(keydown)="isNumber($event)"
[(ngModel)]="playlistItemLimit"
[disabled]="addInProgress || downloads.loading"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="Maximum number of items to download from a playlist or channel (0 = no limit)">
</div>
</div>
<div class="col-md-6">
<div class="input-group">
<span class="input-group-text">Subscription Check (min)</span>
<input type="number"
min="1"
class="form-control"
name="checkIntervalMinutes"
(keydown)="isNumber($event)"
[(ngModel)]="checkIntervalMinutes"
(ngModelChange)="checkIntervalChanged()"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="How often to poll subscriptions for new videos">
</div>
</div>
<div class="col-12">
<div class="row g-2 align-items-center">
<div class="col-auto">
<div class="form-check form-switch">
<input class="form-check-input" type="checkbox" role="switch" id="checkbox-split-chapters"
name="splitByChapters" [(ngModel)]="splitByChapters" (change)="splitByChaptersChanged()"
[disabled]="addInProgress || downloads.loading"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="Split video into separate files by chapters">
<label class="form-check-label" for="checkbox-split-chapters">Split by chapters</label>
</div>
@@ -415,7 +463,7 @@
<div class="input-group">
<span class="input-group-text">Template</span>
<input type="text" class="form-control" name="chapterTemplate" [(ngModel)]="chapterTemplate"
(change)="chapterTemplateChanged()" [disabled]="addInProgress || downloads.loading"
(change)="chapterTemplateChanged()" [disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="Output template for chapter files">
</div>
</div>
@@ -745,6 +793,127 @@
</tbody>
</table>
</div>
<div class="metube-section-header">Subscriptions</div>
<div class="px-2 py-3 border-bottom">
@if (checkingAllSubscriptions) {
<button type="button" class="btn btn-link text-decoration-none px-0 me-4" disabled>
<span class="spinner-border spinner-border-sm me-2" role="status" aria-hidden="true"></span>Check all now
</button>
} @else {
<button type="button" class="btn btn-link text-decoration-none px-0 me-4"
(click)="checkAllSubscriptions()"
[disabled]="downloads.loading || cachedSubs.length === 0 || checkingSelectedSubscriptions">
<fa-icon [icon]="faRedoAlt" />&nbsp; Check all now
</button>
}
@if (checkingSelectedSubscriptions) {
<button type="button" class="btn btn-link text-decoration-none px-0 me-4" disabled>
<span class="spinner-border spinner-border-sm me-2" role="status" aria-hidden="true"></span>Check selected
</button>
} @else {
<button type="button" class="btn btn-link text-decoration-none px-0 me-4"
(click)="checkSelectedSubscriptions()"
[disabled]="downloads.loading || selectedSubscriptionIds.size === 0 || checkingAllSubscriptions">
<fa-icon [icon]="faRedoAlt" />&nbsp; Check selected
</button>
}
<button type="button" class="btn btn-link text-decoration-none px-0 me-4"
(click)="deleteSelectedSubscriptions()"
[disabled]="downloads.loading || selectedSubscriptionIds.size === 0">
<fa-icon [icon]="faTrashAlt" />&nbsp; Delete selected
</button>
</div>
<div class="overflow-auto">
<table class="table">
<thead>
<tr>
<th scope="col" style="width: 1rem;">
<input type="checkbox" class="form-check-input"
[checked]="allSubsSelected()"
(change)="toggleSubMaster($event)"
[disabled]="downloads.loading || cachedSubs.length === 0"
aria-label="Select all subscriptions" />
</th>
<th scope="col">Name</th>
<th scope="col">URL</th>
<th scope="col" class="text-nowrap">Interval (min)</th>
<th scope="col" class="text-nowrap">Last checked</th>
<th scope="col">Status</th>
<th scope="col" style="width: 8rem;"></th>
</tr>
</thead>
<tbody>
@for (entry of cachedSubs; track entry[0]) {
<tr>
<td>
<input type="checkbox" class="form-check-input"
[checked]="isSubSelected(entry[0])"
(change)="toggleSubSelected(entry[0])"
[disabled]="downloads.loading"
[attr.aria-label]="'Select subscription ' + entry[1].name" />
</td>
<td>{{ entry[1].name }}</td>
<td class="text-break"><a [href]="entry[1].url" target="_blank" rel="noopener">{{ entry[1].url }}</a></td>
<td>{{ entry[1].check_interval_minutes }}</td>
<td class="text-nowrap">
@if (entry[1].last_checked !== null) {
<span>{{ entry[1].last_checked! * 1000 | date:'yyyy-MM-dd HH:mm:ss' }}</span>
} @else {
<span class="text-muted"></span>
}
</td>
<td>
@if (entry[1].error) {
<span class="text-danger small">{{ entry[1].error }}</span>
} @else if (entry[1].enabled) {
<span class="text-success">Active</span>
} @else {
<span class="text-secondary">Paused</span>
}
</td>
<td>
<div class="d-flex flex-wrap gap-1">
@if (isSubscriptionChecking(entry[0])) {
<button type="button" class="btn btn-link btn-sm p-0 me-2"
disabled
[attr.aria-label]="'Checking ' + entry[1].name"
ngbTooltip="Checking now">
<span class="spinner-border spinner-border-sm" role="status" aria-hidden="true"></span>
</button>
} @else {
<button type="button" class="btn btn-link btn-sm p-0 me-2"
(click)="checkSubscriptionNow(entry[0])"
[disabled]="downloads.loading"
[attr.aria-label]="'Check now ' + entry[1].name"
ngbTooltip="Check now">
<fa-icon [icon]="faRedoAlt" />
</button>
}
<button type="button" class="btn btn-link btn-sm p-0 me-2"
(click)="toggleSubscriptionEnabled(entry[1])"
[disabled]="downloads.loading"
[attr.aria-label]="(entry[1].enabled ? 'Pause ' : 'Resume ') + entry[1].name"
[ngbTooltip]="entry[1].enabled ? 'Pause' : 'Resume'">
@if (entry[1].enabled) {
<fa-icon [icon]="faPause" />
} @else {
<fa-icon [icon]="faPlay" />
}
</button>
<button type="button" class="btn btn-link btn-sm p-0 text-danger"
(click)="deleteSubscription(entry[0])"
[disabled]="downloads.loading"
[attr.aria-label]="'Delete subscription ' + entry[1].name">
<fa-icon [icon]="faTrashAlt" />
</button>
</div>
</td>
</tr>
}
</tbody>
</table>
</div>
</main><!-- /.container -->
<footer class="footer navbar-dark bg-dark py-3 mt-5">
+274 -10
View File
@@ -1,16 +1,18 @@
import { AsyncPipe, DatePipe, KeyValuePipe } from '@angular/common';
import { AsyncPipe, DatePipe, KeyValuePipe, NgTemplateOutlet } from '@angular/common';
import { HttpClient } from '@angular/common/http';
import { AfterViewInit, ChangeDetectionStrategy, ChangeDetectorRef, Component, DestroyRef, ElementRef, viewChild, inject, OnDestroy, OnInit } from '@angular/core';
import { Observable, map, distinctUntilChanged } from 'rxjs';
import { Observable, Subscription, map, distinctUntilChanged, finalize } from 'rxjs';
import { FormsModule } from '@angular/forms';
import { takeUntilDestroyed } from '@angular/core/rxjs-interop';
import { FontAwesomeModule } from '@fortawesome/angular-fontawesome';
import { NgbModule } from '@ng-bootstrap/ng-bootstrap';
import { NgSelectModule } from '@ng-select/ng-select';
import { faTrashAlt, faCheckCircle, faTimesCircle, faRedoAlt, faSun, faMoon, faCheck, faCircleHalfStroke, faDownload, faExternalLinkAlt, faFileImport, faFileExport, faCopy, faClock, faTachometerAlt, faSortAmountDown, faSortAmountUp, faChevronRight, faChevronDown, faUpload } from '@fortawesome/free-solid-svg-icons';
import { faTrashAlt, faCheckCircle, faTimesCircle, faRedoAlt, faSun, faMoon, faCheck, faCircleHalfStroke, faDownload, faExternalLinkAlt, faFileImport, faFileExport, faCopy, faClock, faTachometerAlt, faSortAmountDown, faSortAmountUp, faChevronRight, faChevronDown, faUpload, faPause, faPlay } from '@fortawesome/free-solid-svg-icons';
import { faGithub } from '@fortawesome/free-brands-svg-icons';
import { CookieService } from 'ngx-cookie-service';
import { AddDownloadPayload, DownloadsService } from './services/downloads.service';
import { SubscriptionsService } from './services/subscriptions.service';
import { SubscriptionRow } from './interfaces/subscription';
import { Themes } from './theme';
import {
Download,
@@ -36,6 +38,7 @@ import { SelectAllCheckboxComponent, ItemCheckboxComponent } from './components/
changeDetection: ChangeDetectionStrategy.OnPush,
imports: [
FormsModule,
NgTemplateOutlet,
KeyValuePipe,
AsyncPipe,
DatePipe,
@@ -53,6 +56,7 @@ import { SelectAllCheckboxComponent, ItemCheckboxComponent } from './components/
})
export class App implements AfterViewInit, OnInit, OnDestroy {
downloads = inject(DownloadsService);
subscriptionsSvc = inject(SubscriptionsService);
private cookieService = inject(CookieService);
private http = inject(HttpClient);
private cdr = inject(ChangeDetectorRef);
@@ -81,6 +85,13 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
subtitleMode: string;
addInProgress = false;
cancelRequested = false;
subscribeInProgress = false;
checkIntervalMinutes = 60;
cachedSubs: [string, SubscriptionRow][] = [];
selectedSubscriptionIds = new Set<string>();
checkingSubscriptionIds = new Set<string>();
checkingAllSubscriptions = false;
checkingSelectedSubscriptions = false;
hasCookies = false;
cookieUploadInProgress = false;
themes: Theme[] = Themes;
@@ -101,6 +112,7 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
cachedSortedDone: [string, Download][] = [];
lastCopiedErrorId: string | null = null;
private previousDownloadType = 'video';
private addRequestSub?: Subscription;
private selectionsByType: Record<string, {
codec: string;
format: string;
@@ -155,6 +167,8 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
faChevronRight = faChevronRight;
faChevronDown = faChevronDown;
faUpload = faUpload;
faPause = faPause;
faPlay = faPlay;
subtitleLanguages = [
{ id: 'en', text: 'English' },
{ id: 'ar', text: 'Arabic' },
@@ -238,6 +252,10 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
this.saveSelection(this.downloadType);
this.sortAscending = this.cookieService.get('metube_sort_ascending') === 'true';
const ci = parseInt(this.cookieService.get('metube_check_interval') || '', 10);
if (!Number.isNaN(ci) && ci >= 1) {
this.checkIntervalMinutes = ci;
}
this.activeTheme = this.getPreferredTheme(this.cookieService);
// Subscribe to download updates
@@ -255,6 +273,11 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
this.updateMetrics();
this.cdr.markForCheck();
});
this.subscriptionsSvc.subscriptionsChanged.pipe(takeUntilDestroyed(this.destroyRef)).subscribe(() => {
this.rebuildCachedSubs();
this.cdr.markForCheck();
});
}
ngOnInit() {
@@ -286,6 +309,7 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
}
ngOnDestroy() {
this.addRequestSub?.unsubscribe();
this.colorSchemeMediaQuery.removeEventListener('change', this.onColorSchemeChanged);
}
@@ -380,11 +404,239 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
if (!this.chapterTemplate) {
this.chapterTemplate = config['OUTPUT_TEMPLATE_CHAPTER'];
}
if (!this.cookieService.check('metube_check_interval')) {
const dci = parseInt(String(config['SUBSCRIPTION_DEFAULT_CHECK_INTERVAL'] ?? 60), 10);
if (!Number.isNaN(dci) && dci >= 1) {
this.checkIntervalMinutes = dci;
}
}
this.cdr.markForCheck();
}
});
}
private rebuildCachedSubs() {
this.cachedSubs = Array.from(this.subscriptionsSvc.subscriptions.entries());
const validIds = new Set(this.cachedSubs.map(([id]) => id));
for (const id of [...this.selectedSubscriptionIds]) {
if (!validIds.has(id)) {
this.selectedSubscriptionIds.delete(id);
}
}
}
checkIntervalChanged() {
this.cookieService.set('metube_check_interval', String(this.checkIntervalMinutes), {
expires: this.settingsCookieExpiryDays,
});
}
private getStatusError(res: unknown): string | null {
const status = res as { status?: string; msg?: string };
return status?.status === 'error' ? status.msg || null : null;
}
private refreshSubscriptionsWithAlert() {
this.subscriptionsSvc.refreshList().pipe(takeUntilDestroyed(this.destroyRef)).subscribe((refreshRes) => {
const error = this.getStatusError(refreshRes);
if (error) {
alert(error || 'Refresh subscriptions failed');
return;
}
this.cdr.markForCheck();
});
}
isSubSelected(id: string): boolean {
return this.selectedSubscriptionIds.has(id);
}
toggleSubSelected(id: string) {
if (this.selectedSubscriptionIds.has(id)) {
this.selectedSubscriptionIds.delete(id);
} else {
this.selectedSubscriptionIds.add(id);
}
this.cdr.markForCheck();
}
toggleSubMaster(event: Event) {
const checked = (event.target as HTMLInputElement).checked;
this.selectedSubscriptionIds.clear();
if (checked) {
for (const [id] of this.cachedSubs) {
this.selectedSubscriptionIds.add(id);
}
}
this.cdr.markForCheck();
}
allSubsSelected(): boolean {
if (this.cachedSubs.length === 0) {
return false;
}
return this.cachedSubs.every(([id]) => this.selectedSubscriptionIds.has(id));
}
addSubscription() {
if (this.subscribeInProgress) {
return;
}
const payload = this.buildAddPayload();
if (!payload.url?.trim()) {
alert('Please enter a URL');
return;
}
if (payload.splitByChapters && !payload.chapterTemplate.includes('%(section_number)')) {
alert('Chapter template must include %(section_number)');
return;
}
this.subscribeInProgress = true;
this.subscriptionsSvc
.subscribe({
...payload,
checkIntervalMinutes: this.checkIntervalMinutes,
})
.pipe(
takeUntilDestroyed(this.destroyRef),
finalize(() => {
this.subscribeInProgress = false;
this.cdr.markForCheck();
}),
)
.subscribe({
next: (res) => {
const r = res as { status?: string; msg?: string };
if (r.status === 'error') {
alert(r.msg || 'Subscribe failed');
} else {
this.addUrl = '';
}
},
});
}
deleteSubscription(id: string) {
this.subscriptionsSvc.delete([id]).subscribe((res) => {
const error = this.getStatusError(res);
if (error) {
alert(error || 'Delete subscription failed');
return;
}
this.selectedSubscriptionIds.delete(id);
this.cdr.markForCheck();
});
}
deleteSelectedSubscriptions() {
const ids = Array.from(this.selectedSubscriptionIds);
if (!ids.length) {
return;
}
this.subscriptionsSvc.delete(ids).subscribe((res) => {
const error = this.getStatusError(res);
if (error) {
alert(error || 'Delete subscriptions failed');
return;
}
this.selectedSubscriptionIds.clear();
this.cdr.markForCheck();
});
}
checkSubscriptionNow(id: string) {
if (this.checkingSubscriptionIds.has(id)) {
return;
}
this.checkingSubscriptionIds.add(id);
this.cdr.markForCheck();
this.subscriptionsSvc
.checkNow([id])
.pipe(
takeUntilDestroyed(this.destroyRef),
finalize(() => {
this.checkingSubscriptionIds.delete(id);
this.cdr.markForCheck();
}),
)
.subscribe((res) => {
const error = this.getStatusError(res);
if (error) {
alert(error || 'Subscription check failed');
return;
}
this.refreshSubscriptionsWithAlert();
});
}
isSubscriptionChecking(id: string): boolean {
return this.checkingSubscriptionIds.has(id);
}
private runBulkSubscriptionCheck(ids: string[] | undefined, mode: 'all' | 'selected') {
const targetIds = ids ?? this.cachedSubs.filter(([, row]) => row.enabled).map(([id]) => id);
if (!targetIds.length) {
return;
}
const checkedIds = new Set(targetIds);
for (const id of checkedIds) {
this.checkingSubscriptionIds.add(id);
}
if (mode === 'all') {
this.checkingAllSubscriptions = true;
} else {
this.checkingSelectedSubscriptions = true;
}
this.cdr.markForCheck();
this.subscriptionsSvc
.checkNow(ids)
.pipe(
takeUntilDestroyed(this.destroyRef),
finalize(() => {
for (const id of checkedIds) {
this.checkingSubscriptionIds.delete(id);
}
if (mode === 'all') {
this.checkingAllSubscriptions = false;
} else {
this.checkingSelectedSubscriptions = false;
}
this.cdr.markForCheck();
}),
)
.subscribe((res) => {
const error = this.getStatusError(res);
if (error) {
alert(error || 'Subscription check failed');
return;
}
this.refreshSubscriptionsWithAlert();
});
}
checkSelectedSubscriptions() {
const ids = Array.from(this.selectedSubscriptionIds);
if (!ids.length) {
return;
}
this.runBulkSubscriptionCheck(ids, 'selected');
}
checkAllSubscriptions() {
this.runBulkSubscriptionCheck(undefined, 'all');
}
toggleSubscriptionEnabled(row: SubscriptionRow) {
this.subscriptionsSvc.update(row.id, { enabled: !row.enabled }).subscribe((res) => {
const error = this.getStatusError(res);
if (error) {
alert(error || 'Update subscription failed');
}
});
}
getPreferredTheme(cookieService: CookieService) {
let theme = 'auto';
if (cookieService.check('metube_theme')) {
@@ -474,13 +726,13 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
}
queueSelectionChanged(checked: number) {
this.queueDelSelected().nativeElement.disabled = checked == 0;
this.queueDownloadSelected().nativeElement.disabled = checked == 0;
this.queueDelSelected().nativeElement.disabled = checked === 0;
this.queueDownloadSelected().nativeElement.disabled = checked === 0;
}
doneSelectionChanged(checked: number) {
this.doneDelSelected().nativeElement.disabled = checked == 0;
this.doneDownloadSelected().nativeElement.disabled = checked == 0;
this.doneDelSelected().nativeElement.disabled = checked === 0;
this.doneDownloadSelected().nativeElement.disabled = checked === 0;
}
private updateDoneActionButtons() {
@@ -657,26 +909,38 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
console.debug('Downloading:', payload);
this.addInProgress = true;
this.cancelRequested = false;
this.downloads.add(payload).subscribe((status: Status) => {
this.addRequestSub?.unsubscribe();
this.addRequestSub = this.downloads.add(payload).subscribe((status: Status) => {
if (status.status === 'error' && !this.cancelRequested) {
alert(`Error adding URL: ${status.msg}`);
} else if (status.status !== 'error') {
this.addUrl = '';
}
this.addInProgress = false;
this.cancelRequested = false;
this.resetAddState();
});
}
cancelAdding() {
this.cancelRequested = true;
this.downloads.cancelAdd().subscribe({
next: () => {
this.addRequestSub?.unsubscribe();
this.resetAddState();
},
error: (err) => {
this.cancelRequested = false;
console.error('Failed to cancel adding:', err?.message || err);
}
});
}
private resetAddState() {
this.addRequestSub = undefined;
this.addInProgress = false;
this.cancelRequested = false;
this.cdr.markForCheck();
}
downloadItemByKey(id: string) {
this.downloads.startById([id]).subscribe();
}
@@ -33,7 +33,7 @@ export class SelectAllCheckboxComponent {
return;
let checked = 0;
this.list().forEach(item => { if(item.checked) checked++ });
this.selected = checked > 0 && checked == this.list().size;
this.selected = checked > 0 && checked === this.list().size;
masterCheckbox.nativeElement.indeterminate = checked > 0 && checked < this.list().size;
this.changed.emit(checked);
}
+1 -1
View File
@@ -6,4 +6,4 @@ export * from './download';
export * from './checkable';
export * from './format';
export * from './formats';
export * from './subscription';
+15
View File
@@ -0,0 +1,15 @@
export interface SubscriptionRow {
id: string;
name: string;
url: string;
enabled: boolean;
check_interval_minutes: number;
download_type: string;
codec: string;
format: string;
quality: string;
folder: string;
last_checked: number | null;
seen_count: number;
error: string | null;
}
@@ -0,0 +1,128 @@
import { DestroyRef, inject, Injectable } from '@angular/core';
import { HttpClient, HttpErrorResponse } from '@angular/common/http';
import { of, Subject } from 'rxjs';
import { catchError, tap } from 'rxjs/operators';
import { takeUntilDestroyed } from '@angular/core/rxjs-interop';
import { MeTubeSocket } from './metube-socket.service';
import { SubscriptionRow } from '../interfaces/subscription';
import { Status } from '../interfaces';
import { AddDownloadPayload } from './downloads.service';
export interface SubscribePayload extends AddDownloadPayload {
checkIntervalMinutes: number;
}
@Injectable({
providedIn: 'root',
})
export class SubscriptionsService {
private http = inject(HttpClient);
private socket = inject(MeTubeSocket);
private destroyRef = inject(DestroyRef);
subscriptions = new Map<string, SubscriptionRow>();
subscriptionsChanged = new Subject<void>();
private publishList(rows: SubscriptionRow[]) {
this.subscriptions.clear();
for (const row of rows) {
this.subscriptions.set(row.id, row);
}
this.subscriptionsChanged.next();
}
constructor() {
this.socket
.fromEvent('subscriptions_all')
.pipe(takeUntilDestroyed(this.destroyRef))
.subscribe((strdata: string) => {
const data: SubscriptionRow[] = JSON.parse(strdata);
this.publishList(data);
});
this.socket
.fromEvent('subscription_added')
.pipe(takeUntilDestroyed(this.destroyRef))
.subscribe((strdata: string) => {
const row: SubscriptionRow = JSON.parse(strdata);
this.subscriptions.set(row.id, row);
this.subscriptionsChanged.next();
});
this.socket
.fromEvent('subscription_updated')
.pipe(takeUntilDestroyed(this.destroyRef))
.subscribe((strdata: string) => {
const row: SubscriptionRow = JSON.parse(strdata);
this.subscriptions.set(row.id, row);
this.subscriptionsChanged.next();
});
this.socket
.fromEvent('subscription_removed')
.pipe(takeUntilDestroyed(this.destroyRef))
.subscribe((strdata: string) => {
const id: string = JSON.parse(strdata);
this.subscriptions.delete(id);
this.subscriptionsChanged.next();
});
}
handleHTTPError(error: HttpErrorResponse) {
const msg =
error.error instanceof ErrorEvent
? error.error.message
: typeof error.error === 'string'
? error.error
: error.error?.msg || error.message || 'Request failed';
return of({ status: 'error' as const, msg });
}
subscribe(payload: SubscribePayload) {
return this.http
.post<Status>('subscribe', {
url: payload.url,
download_type: payload.downloadType,
codec: payload.codec,
quality: payload.quality,
format: payload.format,
folder: payload.folder,
custom_name_prefix: payload.customNamePrefix,
playlist_item_limit: payload.playlistItemLimit,
auto_start: payload.autoStart,
split_by_chapters: payload.splitByChapters,
chapter_template: payload.chapterTemplate,
subtitle_language: payload.subtitleLanguage,
subtitle_mode: payload.subtitleMode,
check_interval_minutes: payload.checkIntervalMinutes,
})
.pipe(catchError((err) => this.handleHTTPError(err)));
}
delete(ids: string[]) {
return this.http.post('subscriptions/delete', { ids }).pipe(catchError((err) => this.handleHTTPError(err)));
}
update(id: string, changes: Partial<Pick<SubscriptionRow, 'enabled' | 'check_interval_minutes' | 'name'>>) {
return this.http
.post('subscriptions/update', { id, ...changes })
.pipe(catchError((err) => this.handleHTTPError(err)));
}
checkNow(ids?: string[]) {
return this.http
.post('subscriptions/check', ids?.length ? { ids } : {})
.pipe(catchError((err) => this.handleHTTPError(err)));
}
fetchList() {
return this.http.get<SubscriptionRow[]>('subscriptions').pipe(catchError(() => of([])));
}
refreshList() {
return this.http.get<SubscriptionRow[]>('subscriptions').pipe(
tap((rows) => this.publishList(rows)),
catchError((err) => this.handleHTTPError(err)),
);
}
}
Generated
+69 -69
View File
@@ -13,7 +13,7 @@ wheels = [
[[package]]
name = "aiohttp"
version = "3.13.3"
version = "3.13.5"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohappyeyeballs" },
@@ -24,59 +24,59 @@ dependencies = [
{ name = "propcache" },
{ name = "yarl" },
]
sdist = { url = "https://files.pythonhosted.org/packages/50/42/32cf8e7704ceb4481406eb87161349abb46a57fee3f008ba9cb610968646/aiohttp-3.13.3.tar.gz", hash = "sha256:a949eee43d3782f2daae4f4a2819b2cb9b0c5d3b7f7a927067cc84dafdbb9f88", size = 7844556, upload-time = "2026-01-03T17:33:05.204Z" }
sdist = { url = "https://files.pythonhosted.org/packages/77/9a/152096d4808df8e4268befa55fba462f440f14beab85e8ad9bf990516918/aiohttp-3.13.5.tar.gz", hash = "sha256:9d98cc980ecc96be6eb4c1994ce35d28d8b1f5e5208a23b421187d1209dbb7d1", size = 7858271, upload-time = "2026-03-31T22:01:03.343Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/97/8a/12ca489246ca1faaf5432844adbfce7ff2cc4997733e0af120869345643a/aiohttp-3.13.3-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:5dff64413671b0d3e7d5918ea490bdccb97a4ad29b3f311ed423200b2203e01c", size = 734190, upload-time = "2026-01-03T17:30:45.832Z" },
{ url = "https://files.pythonhosted.org/packages/32/08/de43984c74ed1fca5c014808963cc83cb00d7bb06af228f132d33862ca76/aiohttp-3.13.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:87b9aab6d6ed88235aa2970294f496ff1a1f9adcd724d800e9b952395a80ffd9", size = 491783, upload-time = "2026-01-03T17:30:47.466Z" },
{ url = "https://files.pythonhosted.org/packages/17/f8/8dd2cf6112a5a76f81f81a5130c57ca829d101ad583ce57f889179accdda/aiohttp-3.13.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:425c126c0dc43861e22cb1c14ba4c8e45d09516d0a3ae0a3f7494b79f5f233a3", size = 490704, upload-time = "2026-01-03T17:30:49.373Z" },
{ url = "https://files.pythonhosted.org/packages/6d/40/a46b03ca03936f832bc7eaa47cfbb1ad012ba1be4790122ee4f4f8cba074/aiohttp-3.13.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7f9120f7093c2a32d9647abcaf21e6ad275b4fbec5b55969f978b1a97c7c86bf", size = 1720652, upload-time = "2026-01-03T17:30:50.974Z" },
{ url = "https://files.pythonhosted.org/packages/f7/7e/917fe18e3607af92657e4285498f500dca797ff8c918bd7d90b05abf6c2a/aiohttp-3.13.3-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:697753042d57f4bf7122cab985bf15d0cef23c770864580f5af4f52023a56bd6", size = 1692014, upload-time = "2026-01-03T17:30:52.729Z" },
{ url = "https://files.pythonhosted.org/packages/71/b6/cefa4cbc00d315d68973b671cf105b21a609c12b82d52e5d0c9ae61d2a09/aiohttp-3.13.3-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:6de499a1a44e7de70735d0b39f67c8f25eb3d91eb3103be99ca0fa882cdd987d", size = 1759777, upload-time = "2026-01-03T17:30:54.537Z" },
{ url = "https://files.pythonhosted.org/packages/fb/e3/e06ee07b45e59e6d81498b591fc589629be1553abb2a82ce33efe2a7b068/aiohttp-3.13.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:37239e9f9a7ea9ac5bf6b92b0260b01f8a22281996da609206a84df860bc1261", size = 1861276, upload-time = "2026-01-03T17:30:56.512Z" },
{ url = "https://files.pythonhosted.org/packages/7c/24/75d274228acf35ceeb2850b8ce04de9dd7355ff7a0b49d607ee60c29c518/aiohttp-3.13.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f76c1e3fe7d7c8afad7ed193f89a292e1999608170dcc9751a7462a87dfd5bc0", size = 1743131, upload-time = "2026-01-03T17:30:58.256Z" },
{ url = "https://files.pythonhosted.org/packages/04/98/3d21dde21889b17ca2eea54fdcff21b27b93f45b7bb94ca029c31ab59dc3/aiohttp-3.13.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:fc290605db2a917f6e81b0e1e0796469871f5af381ce15c604a3c5c7e51cb730", size = 1556863, upload-time = "2026-01-03T17:31:00.445Z" },
{ url = "https://files.pythonhosted.org/packages/9e/84/da0c3ab1192eaf64782b03971ab4055b475d0db07b17eff925e8c93b3aa5/aiohttp-3.13.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4021b51936308aeea0367b8f006dc999ca02bc118a0cc78c303f50a2ff6afb91", size = 1682793, upload-time = "2026-01-03T17:31:03.024Z" },
{ url = "https://files.pythonhosted.org/packages/ff/0f/5802ada182f575afa02cbd0ec5180d7e13a402afb7c2c03a9aa5e5d49060/aiohttp-3.13.3-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:49a03727c1bba9a97d3e93c9f93ca03a57300f484b6e935463099841261195d3", size = 1716676, upload-time = "2026-01-03T17:31:04.842Z" },
{ url = "https://files.pythonhosted.org/packages/3f/8c/714d53bd8b5a4560667f7bbbb06b20c2382f9c7847d198370ec6526af39c/aiohttp-3.13.3-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:3d9908a48eb7416dc1f4524e69f1d32e5d90e3981e4e37eb0aa1cd18f9cfa2a4", size = 1733217, upload-time = "2026-01-03T17:31:06.868Z" },
{ url = "https://files.pythonhosted.org/packages/7d/79/e2176f46d2e963facea939f5be2d26368ce543622be6f00a12844d3c991f/aiohttp-3.13.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:2712039939ec963c237286113c68dbad80a82a4281543f3abf766d9d73228998", size = 1552303, upload-time = "2026-01-03T17:31:08.958Z" },
{ url = "https://files.pythonhosted.org/packages/ab/6a/28ed4dea1759916090587d1fe57087b03e6c784a642b85ef48217b0277ae/aiohttp-3.13.3-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:7bfdc049127717581866fa4708791220970ce291c23e28ccf3922c700740fdc0", size = 1763673, upload-time = "2026-01-03T17:31:10.676Z" },
{ url = "https://files.pythonhosted.org/packages/e8/35/4a3daeb8b9fab49240d21c04d50732313295e4bd813a465d840236dd0ce1/aiohttp-3.13.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8057c98e0c8472d8846b9c79f56766bcc57e3e8ac7bfd510482332366c56c591", size = 1721120, upload-time = "2026-01-03T17:31:12.575Z" },
{ url = "https://files.pythonhosted.org/packages/bc/9f/d643bb3c5fb99547323e635e251c609fbbc660d983144cfebec529e09264/aiohttp-3.13.3-cp313-cp313-win32.whl", hash = "sha256:1449ceddcdbcf2e0446957863af03ebaaa03f94c090f945411b61269e2cb5daf", size = 427383, upload-time = "2026-01-03T17:31:14.382Z" },
{ url = "https://files.pythonhosted.org/packages/4e/f1/ab0395f8a79933577cdd996dd2f9aa6014af9535f65dddcf88204682fe62/aiohttp-3.13.3-cp313-cp313-win_amd64.whl", hash = "sha256:693781c45a4033d31d4187d2436f5ac701e7bbfe5df40d917736108c1cc7436e", size = 453899, upload-time = "2026-01-03T17:31:15.958Z" },
{ url = "https://files.pythonhosted.org/packages/99/36/5b6514a9f5d66f4e2597e40dea2e3db271e023eb7a5d22defe96ba560996/aiohttp-3.13.3-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:ea37047c6b367fd4bd632bff8077449b8fa034b69e812a18e0132a00fae6e808", size = 737238, upload-time = "2026-01-03T17:31:17.909Z" },
{ url = "https://files.pythonhosted.org/packages/f7/49/459327f0d5bcd8c6c9ca69e60fdeebc3622861e696490d8674a6d0cb90a6/aiohttp-3.13.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:6fc0e2337d1a4c3e6acafda6a78a39d4c14caea625124817420abceed36e2415", size = 492292, upload-time = "2026-01-03T17:31:19.919Z" },
{ url = "https://files.pythonhosted.org/packages/e8/0b/b97660c5fd05d3495b4eb27f2d0ef18dc1dc4eff7511a9bf371397ff0264/aiohttp-3.13.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c685f2d80bb67ca8c3837823ad76196b3694b0159d232206d1e461d3d434666f", size = 493021, upload-time = "2026-01-03T17:31:21.636Z" },
{ url = "https://files.pythonhosted.org/packages/54/d4/438efabdf74e30aeceb890c3290bbaa449780583b1270b00661126b8aae4/aiohttp-3.13.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:48e377758516d262bde50c2584fc6c578af272559c409eecbdd2bae1601184d6", size = 1717263, upload-time = "2026-01-03T17:31:23.296Z" },
{ url = "https://files.pythonhosted.org/packages/71/f2/7bddc7fd612367d1459c5bcf598a9e8f7092d6580d98de0e057eb42697ad/aiohttp-3.13.3-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:34749271508078b261c4abb1767d42b8d0c0cc9449c73a4df494777dc55f0687", size = 1669107, upload-time = "2026-01-03T17:31:25.334Z" },
{ url = "https://files.pythonhosted.org/packages/00/5a/1aeaecca40e22560f97610a329e0e5efef5e0b5afdf9f857f0d93839ab2e/aiohttp-3.13.3-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:82611aeec80eb144416956ec85b6ca45a64d76429c1ed46ae1b5f86c6e0c9a26", size = 1760196, upload-time = "2026-01-03T17:31:27.394Z" },
{ url = "https://files.pythonhosted.org/packages/f8/f8/0ff6992bea7bd560fc510ea1c815f87eedd745fe035589c71ce05612a19a/aiohttp-3.13.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:2fff83cfc93f18f215896e3a190e8e5cb413ce01553901aca925176e7568963a", size = 1843591, upload-time = "2026-01-03T17:31:29.238Z" },
{ url = "https://files.pythonhosted.org/packages/e3/d1/e30e537a15f53485b61f5be525f2157da719819e8377298502aebac45536/aiohttp-3.13.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bbe7d4cecacb439e2e2a8a1a7b935c25b812af7a5fd26503a66dadf428e79ec1", size = 1720277, upload-time = "2026-01-03T17:31:31.053Z" },
{ url = "https://files.pythonhosted.org/packages/84/45/23f4c451d8192f553d38d838831ebbc156907ea6e05557f39563101b7717/aiohttp-3.13.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b928f30fe49574253644b1ca44b1b8adbd903aa0da4b9054a6c20fc7f4092a25", size = 1548575, upload-time = "2026-01-03T17:31:32.87Z" },
{ url = "https://files.pythonhosted.org/packages/6a/ed/0a42b127a43712eda7807e7892c083eadfaf8429ca8fb619662a530a3aab/aiohttp-3.13.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7b5e8fe4de30df199155baaf64f2fcd604f4c678ed20910db8e2c66dc4b11603", size = 1679455, upload-time = "2026-01-03T17:31:34.76Z" },
{ url = "https://files.pythonhosted.org/packages/2e/b5/c05f0c2b4b4fe2c9d55e73b6d3ed4fd6c9dc2684b1d81cbdf77e7fad9adb/aiohttp-3.13.3-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:8542f41a62bcc58fc7f11cf7c90e0ec324ce44950003feb70640fc2a9092c32a", size = 1687417, upload-time = "2026-01-03T17:31:36.699Z" },
{ url = "https://files.pythonhosted.org/packages/c9/6b/915bc5dad66aef602b9e459b5a973529304d4e89ca86999d9d75d80cbd0b/aiohttp-3.13.3-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:5e1d8c8b8f1d91cd08d8f4a3c2b067bfca6ec043d3ff36de0f3a715feeedf926", size = 1729968, upload-time = "2026-01-03T17:31:38.622Z" },
{ url = "https://files.pythonhosted.org/packages/11/3b/e84581290a9520024a08640b63d07673057aec5ca548177a82026187ba73/aiohttp-3.13.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:90455115e5da1c3c51ab619ac57f877da8fd6d73c05aacd125c5ae9819582aba", size = 1545690, upload-time = "2026-01-03T17:31:40.57Z" },
{ url = "https://files.pythonhosted.org/packages/f5/04/0c3655a566c43fd647c81b895dfe361b9f9ad6d58c19309d45cff52d6c3b/aiohttp-3.13.3-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:042e9e0bcb5fba81886c8b4fbb9a09d6b8a00245fd8d88e4d989c1f96c74164c", size = 1746390, upload-time = "2026-01-03T17:31:42.857Z" },
{ url = "https://files.pythonhosted.org/packages/1f/53/71165b26978f719c3419381514c9690bd5980e764a09440a10bb816ea4ab/aiohttp-3.13.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2eb752b102b12a76ca02dff751a801f028b4ffbbc478840b473597fc91a9ed43", size = 1702188, upload-time = "2026-01-03T17:31:44.984Z" },
{ url = "https://files.pythonhosted.org/packages/29/a7/cbe6c9e8e136314fa1980da388a59d2f35f35395948a08b6747baebb6aa6/aiohttp-3.13.3-cp314-cp314-win32.whl", hash = "sha256:b556c85915d8efaed322bf1bdae9486aa0f3f764195a0fb6ee962e5c71ef5ce1", size = 433126, upload-time = "2026-01-03T17:31:47.463Z" },
{ url = "https://files.pythonhosted.org/packages/de/56/982704adea7d3b16614fc5936014e9af85c0e34b58f9046655817f04306e/aiohttp-3.13.3-cp314-cp314-win_amd64.whl", hash = "sha256:9bf9f7a65e7aa20dd764151fb3d616c81088f91f8df39c3893a536e279b4b984", size = 459128, upload-time = "2026-01-03T17:31:49.2Z" },
{ url = "https://files.pythonhosted.org/packages/6c/2a/3c79b638a9c3d4658d345339d22070241ea341ed4e07b5ac60fb0f418003/aiohttp-3.13.3-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:05861afbbec40650d8a07ea324367cb93e9e8cc7762e04dd4405df99fa65159c", size = 769512, upload-time = "2026-01-03T17:31:51.134Z" },
{ url = "https://files.pythonhosted.org/packages/29/b9/3e5014d46c0ab0db8707e0ac2711ed28c4da0218c358a4e7c17bae0d8722/aiohttp-3.13.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:2fc82186fadc4a8316768d61f3722c230e2c1dcab4200d52d2ebdf2482e47592", size = 506444, upload-time = "2026-01-03T17:31:52.85Z" },
{ url = "https://files.pythonhosted.org/packages/90/03/c1d4ef9a054e151cd7839cdc497f2638f00b93cbe8043983986630d7a80c/aiohttp-3.13.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:0add0900ff220d1d5c5ebbf99ed88b0c1bbf87aa7e4262300ed1376a6b13414f", size = 510798, upload-time = "2026-01-03T17:31:54.91Z" },
{ url = "https://files.pythonhosted.org/packages/ea/76/8c1e5abbfe8e127c893fe7ead569148a4d5a799f7cf958d8c09f3eedf097/aiohttp-3.13.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:568f416a4072fbfae453dcf9a99194bbb8bdeab718e08ee13dfa2ba0e4bebf29", size = 1868835, upload-time = "2026-01-03T17:31:56.733Z" },
{ url = "https://files.pythonhosted.org/packages/8e/ac/984c5a6f74c363b01ff97adc96a3976d9c98940b8969a1881575b279ac5d/aiohttp-3.13.3-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:add1da70de90a2569c5e15249ff76a631ccacfe198375eead4aadf3b8dc849dc", size = 1720486, upload-time = "2026-01-03T17:31:58.65Z" },
{ url = "https://files.pythonhosted.org/packages/b2/9a/b7039c5f099c4eb632138728828b33428585031a1e658d693d41d07d89d1/aiohttp-3.13.3-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:10b47b7ba335d2e9b1239fa571131a87e2d8ec96b333e68b2a305e7a98b0bae2", size = 1847951, upload-time = "2026-01-03T17:32:00.989Z" },
{ url = "https://files.pythonhosted.org/packages/3c/02/3bec2b9a1ba3c19ff89a43a19324202b8eb187ca1e928d8bdac9bbdddebd/aiohttp-3.13.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3dd4dce1c718e38081c8f35f323209d4c1df7d4db4bab1b5c88a6b4d12b74587", size = 1941001, upload-time = "2026-01-03T17:32:03.122Z" },
{ url = "https://files.pythonhosted.org/packages/37/df/d879401cedeef27ac4717f6426c8c36c3091c6e9f08a9178cc87549c537f/aiohttp-3.13.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:34bac00a67a812570d4a460447e1e9e06fae622946955f939051e7cc895cfab8", size = 1797246, upload-time = "2026-01-03T17:32:05.255Z" },
{ url = "https://files.pythonhosted.org/packages/8d/15/be122de1f67e6953add23335c8ece6d314ab67c8bebb3f181063010795a7/aiohttp-3.13.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a19884d2ee70b06d9204b2727a7b9f983d0c684c650254679e716b0b77920632", size = 1627131, upload-time = "2026-01-03T17:32:07.607Z" },
{ url = "https://files.pythonhosted.org/packages/12/12/70eedcac9134cfa3219ab7af31ea56bc877395b1ac30d65b1bc4b27d0438/aiohttp-3.13.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:5f8ca7f2bb6ba8348a3614c7918cc4bb73268c5ac2a207576b7afea19d3d9f64", size = 1795196, upload-time = "2026-01-03T17:32:09.59Z" },
{ url = "https://files.pythonhosted.org/packages/32/11/b30e1b1cd1f3054af86ebe60df96989c6a414dd87e27ad16950eee420bea/aiohttp-3.13.3-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:b0d95340658b9d2f11d9697f59b3814a9d3bb4b7a7c20b131df4bcef464037c0", size = 1782841, upload-time = "2026-01-03T17:32:11.445Z" },
{ url = "https://files.pythonhosted.org/packages/88/0d/d98a9367b38912384a17e287850f5695c528cff0f14f791ce8ee2e4f7796/aiohttp-3.13.3-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:a1e53262fd202e4b40b70c3aff944a8155059beedc8a89bba9dc1f9ef06a1b56", size = 1795193, upload-time = "2026-01-03T17:32:13.705Z" },
{ url = "https://files.pythonhosted.org/packages/43/a5/a2dfd1f5ff5581632c7f6a30e1744deda03808974f94f6534241ef60c751/aiohttp-3.13.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:d60ac9663f44168038586cab2157e122e46bdef09e9368b37f2d82d354c23f72", size = 1621979, upload-time = "2026-01-03T17:32:15.965Z" },
{ url = "https://files.pythonhosted.org/packages/fa/f0/12973c382ae7c1cccbc4417e129c5bf54c374dfb85af70893646e1f0e749/aiohttp-3.13.3-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:90751b8eed69435bac9ff4e3d2f6b3af1f57e37ecb0fbeee59c0174c9e2d41df", size = 1822193, upload-time = "2026-01-03T17:32:18.219Z" },
{ url = "https://files.pythonhosted.org/packages/3c/5f/24155e30ba7f8c96918af1350eb0663e2430aad9e001c0489d89cd708ab1/aiohttp-3.13.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:fc353029f176fd2b3ec6cfc71be166aba1936fe5d73dd1992ce289ca6647a9aa", size = 1769801, upload-time = "2026-01-03T17:32:20.25Z" },
{ url = "https://files.pythonhosted.org/packages/eb/f8/7314031ff5c10e6ece114da79b338ec17eeff3a079e53151f7e9f43c4723/aiohttp-3.13.3-cp314-cp314t-win32.whl", hash = "sha256:2e41b18a58da1e474a057b3d35248d8320029f61d70a37629535b16a0c8f3767", size = 466523, upload-time = "2026-01-03T17:32:22.215Z" },
{ url = "https://files.pythonhosted.org/packages/b4/63/278a98c715ae467624eafe375542d8ba9b4383a016df8fdefe0ae28382a7/aiohttp-3.13.3-cp314-cp314t-win_amd64.whl", hash = "sha256:44531a36aa2264a1860089ffd4dce7baf875ee5a6079d5fb42e261c704ef7344", size = 499694, upload-time = "2026-01-03T17:32:24.546Z" },
{ url = "https://files.pythonhosted.org/packages/78/e9/d76bf503005709e390122d34e15256b88f7008e246c4bdbe915cd4f1adce/aiohttp-3.13.5-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:a5029cc80718bbd545123cd8fe5d15025eccaaaace5d0eeec6bd556ad6163d61", size = 742930, upload-time = "2026-03-31T21:58:13.155Z" },
{ url = "https://files.pythonhosted.org/packages/57/00/4b7b70223deaebd9bb85984d01a764b0d7bd6526fcdc73cca83bcbe7243e/aiohttp-3.13.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:4bb6bf5811620003614076bdc807ef3b5e38244f9d25ca5fe888eaccea2a9832", size = 496927, upload-time = "2026-03-31T21:58:15.073Z" },
{ url = "https://files.pythonhosted.org/packages/9c/f5/0fb20fb49f8efdcdce6cd8127604ad2c503e754a8f139f5e02b01626523f/aiohttp-3.13.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a84792f8631bf5a94e52d9cc881c0b824ab42717165a5579c760b830d9392ac9", size = 497141, upload-time = "2026-03-31T21:58:17.009Z" },
{ url = "https://files.pythonhosted.org/packages/3b/86/b7c870053e36a94e8951b803cb5b909bfbc9b90ca941527f5fcafbf6b0fa/aiohttp-3.13.5-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:57653eac22c6a4c13eb22ecf4d673d64a12f266e72785ab1c8b8e5940d0e8090", size = 1732476, upload-time = "2026-03-31T21:58:18.925Z" },
{ url = "https://files.pythonhosted.org/packages/b5/e5/4e161f84f98d80c03a238671b4136e6530453d65262867d989bbe78244d0/aiohttp-3.13.5-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:e5e5f7debc7a57af53fdf5c5009f9391d9f4c12867049d509bf7bb164a6e295b", size = 1706507, upload-time = "2026-03-31T21:58:21.094Z" },
{ url = "https://files.pythonhosted.org/packages/d4/56/ea11a9f01518bd5a2a2fcee869d248c4b8a0cfa0bb13401574fa31adf4d4/aiohttp-3.13.5-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c719f65bebcdf6716f10e9eff80d27567f7892d8988c06de12bbbd39307c6e3a", size = 1773465, upload-time = "2026-03-31T21:58:23.159Z" },
{ url = "https://files.pythonhosted.org/packages/eb/40/333ca27fb74b0383f17c90570c748f7582501507307350a79d9f9f3c6eb1/aiohttp-3.13.5-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d97f93fdae594d886c5a866636397e2bcab146fd7a132fd6bb9ce182224452f8", size = 1873523, upload-time = "2026-03-31T21:58:25.59Z" },
{ url = "https://files.pythonhosted.org/packages/f0/d2/e2f77eef1acb7111405433c707dc735e63f67a56e176e72e9e7a2cd3f493/aiohttp-3.13.5-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3df334e39d4c2f899a914f1dba283c1aadc311790733f705182998c6f7cae665", size = 1754113, upload-time = "2026-03-31T21:58:27.624Z" },
{ url = "https://files.pythonhosted.org/packages/fb/56/3f653d7f53c89669301ec9e42c95233e2a0c0a6dd051269e6e678db4fdb0/aiohttp-3.13.5-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:fe6970addfea9e5e081401bcbadf865d2b6da045472f58af08427e108d618540", size = 1562351, upload-time = "2026-03-31T21:58:29.918Z" },
{ url = "https://files.pythonhosted.org/packages/ec/a6/9b3e91eb8ae791cce4ee736da02211c85c6f835f1bdfac0594a8a3b7018c/aiohttp-3.13.5-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7becdf835feff2f4f335d7477f121af787e3504b48b449ff737afb35869ba7bb", size = 1693205, upload-time = "2026-03-31T21:58:32.214Z" },
{ url = "https://files.pythonhosted.org/packages/98/fc/bfb437a99a2fcebd6b6eaec609571954de2ed424f01c352f4b5504371dd3/aiohttp-3.13.5-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:676e5651705ad5d8a70aeb8eb6936c436d8ebbd56e63436cb7dd9bb36d2a9a46", size = 1730618, upload-time = "2026-03-31T21:58:34.728Z" },
{ url = "https://files.pythonhosted.org/packages/e4/b6/c8534862126191a034f68153194c389addc285a0f1347d85096d349bbc15/aiohttp-3.13.5-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:9b16c653d38eb1a611cc898c41e76859ca27f119d25b53c12875fd0474ae31a8", size = 1745185, upload-time = "2026-03-31T21:58:36.909Z" },
{ url = "https://files.pythonhosted.org/packages/0b/93/4ca8ee2ef5236e2707e0fd5fecb10ce214aee1ff4ab307af9c558bda3b37/aiohttp-3.13.5-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:999802d5fa0389f58decd24b537c54aa63c01c3219ce17d1214cbda3c2b22d2d", size = 1557311, upload-time = "2026-03-31T21:58:39.38Z" },
{ url = "https://files.pythonhosted.org/packages/57/ae/76177b15f18c5f5d094f19901d284025db28eccc5ae374d1d254181d33f4/aiohttp-3.13.5-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:ec707059ee75732b1ba130ed5f9580fe10ff75180c812bc267ded039db5128c6", size = 1773147, upload-time = "2026-03-31T21:58:41.476Z" },
{ url = "https://files.pythonhosted.org/packages/01/a4/62f05a0a98d88af59d93b7fcac564e5f18f513cb7471696ac286db970d6a/aiohttp-3.13.5-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:2d6d44a5b48132053c2f6cd5c8cb14bc67e99a63594e336b0f2af81e94d5530c", size = 1730356, upload-time = "2026-03-31T21:58:44.049Z" },
{ url = "https://files.pythonhosted.org/packages/e4/85/fc8601f59dfa8c9523808281f2da571f8b4699685f9809a228adcc90838d/aiohttp-3.13.5-cp313-cp313-win32.whl", hash = "sha256:329f292ed14d38a6c4c435e465f48bebb47479fd676a0411936cc371643225cc", size = 432637, upload-time = "2026-03-31T21:58:46.167Z" },
{ url = "https://files.pythonhosted.org/packages/c0/1b/ac685a8882896acf0f6b31d689e3792199cfe7aba37969fa91da63a7fa27/aiohttp-3.13.5-cp313-cp313-win_amd64.whl", hash = "sha256:69f571de7500e0557801c0b51f4780482c0ec5fe2ac851af5a92cfce1af1cb83", size = 458896, upload-time = "2026-03-31T21:58:48.119Z" },
{ url = "https://files.pythonhosted.org/packages/5d/ce/46572759afc859e867a5bc8ec3487315869013f59281ce61764f76d879de/aiohttp-3.13.5-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:eb4639f32fd4a9904ab8fb45bf3383ba71137f3d9d4ba25b3b3f3109977c5b8c", size = 745721, upload-time = "2026-03-31T21:58:50.229Z" },
{ url = "https://files.pythonhosted.org/packages/13/fe/8a2efd7626dbe6049b2ef8ace18ffda8a4dfcbe1bcff3ac30c0c7575c20b/aiohttp-3.13.5-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:7e5dc4311bd5ac493886c63cbf76ab579dbe4641268e7c74e48e774c74b6f2be", size = 497663, upload-time = "2026-03-31T21:58:52.232Z" },
{ url = "https://files.pythonhosted.org/packages/9b/91/cc8cc78a111826c54743d88651e1687008133c37e5ee615fee9b57990fac/aiohttp-3.13.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:756c3c304d394977519824449600adaf2be0ccee76d206ee339c5e76b70ded25", size = 499094, upload-time = "2026-03-31T21:58:54.566Z" },
{ url = "https://files.pythonhosted.org/packages/0a/33/a8362cb15cf16a3af7e86ed11962d5cd7d59b449202dc576cdc731310bde/aiohttp-3.13.5-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ecc26751323224cf8186efcf7fbcbc30f4e1d8c7970659daf25ad995e4032a56", size = 1726701, upload-time = "2026-03-31T21:58:56.864Z" },
{ url = "https://files.pythonhosted.org/packages/45/0c/c091ac5c3a17114bd76cbf85d674650969ddf93387876cf67f754204bd77/aiohttp-3.13.5-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:10a75acfcf794edf9d8db50e5a7ec5fc818b2a8d3f591ce93bc7b1210df016d2", size = 1683360, upload-time = "2026-03-31T21:58:59.072Z" },
{ url = "https://files.pythonhosted.org/packages/23/73/bcee1c2b79bc275e964d1446c55c54441a461938e70267c86afaae6fba27/aiohttp-3.13.5-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:0f7a18f258d124cd678c5fe072fe4432a4d5232b0657fca7c1847f599233c83a", size = 1773023, upload-time = "2026-03-31T21:59:01.776Z" },
{ url = "https://files.pythonhosted.org/packages/c7/ef/720e639df03004fee2d869f771799d8c23046dec47d5b81e396c7cda583a/aiohttp-3.13.5-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:df6104c009713d3a89621096f3e3e88cc323fd269dbd7c20afe18535094320be", size = 1853795, upload-time = "2026-03-31T21:59:04.568Z" },
{ url = "https://files.pythonhosted.org/packages/bd/c9/989f4034fb46841208de7aeeac2c6d8300745ab4f28c42f629ba77c2d916/aiohttp-3.13.5-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:241a94f7de7c0c3b616627aaad530fe2cb620084a8b144d3be7b6ecfe95bae3b", size = 1730405, upload-time = "2026-03-31T21:59:07.221Z" },
{ url = "https://files.pythonhosted.org/packages/ce/75/ee1fd286ca7dc599d824b5651dad7b3be7ff8d9a7e7b3fe9820d9180f7db/aiohttp-3.13.5-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c974fb66180e58709b6fc402846f13791240d180b74de81d23913abe48e96d94", size = 1558082, upload-time = "2026-03-31T21:59:09.484Z" },
{ url = "https://files.pythonhosted.org/packages/c3/20/1e9e6650dfc436340116b7aa89ff8cb2bbdf0abc11dfaceaad8f74273a10/aiohttp-3.13.5-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:6e27ea05d184afac78aabbac667450c75e54e35f62238d44463131bd3f96753d", size = 1692346, upload-time = "2026-03-31T21:59:12.068Z" },
{ url = "https://files.pythonhosted.org/packages/d8/40/8ebc6658d48ea630ac7903912fe0dd4e262f0e16825aa4c833c56c9f1f56/aiohttp-3.13.5-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:a79a6d399cef33a11b6f004c67bb07741d91f2be01b8d712d52c75711b1e07c7", size = 1698891, upload-time = "2026-03-31T21:59:14.552Z" },
{ url = "https://files.pythonhosted.org/packages/d8/78/ea0ae5ec8ba7a5c10bdd6e318f1ba5e76fcde17db8275188772afc7917a4/aiohttp-3.13.5-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:c632ce9c0b534fbe25b52c974515ed674937c5b99f549a92127c85f771a78772", size = 1742113, upload-time = "2026-03-31T21:59:17.068Z" },
{ url = "https://files.pythonhosted.org/packages/8a/66/9d308ed71e3f2491be1acb8769d96c6f0c47d92099f3bc9119cada27b357/aiohttp-3.13.5-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:fceedde51fbd67ee2bcc8c0b33d0126cc8b51ef3bbde2f86662bd6d5a6f10ec5", size = 1553088, upload-time = "2026-03-31T21:59:19.541Z" },
{ url = "https://files.pythonhosted.org/packages/da/a6/6cc25ed8dfc6e00c90f5c6d126a98e2cf28957ad06fa1036bd34b6f24a2c/aiohttp-3.13.5-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:f92995dfec9420bb69ae629abf422e516923ba79ba4403bc750d94fb4a6c68c1", size = 1757976, upload-time = "2026-03-31T21:59:22.311Z" },
{ url = "https://files.pythonhosted.org/packages/c1/2b/cce5b0ffe0de99c83e5e36d8f828e4161e415660a9f3e58339d07cce3006/aiohttp-3.13.5-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:20ae0ff08b1f2c8788d6fb85afcb798654ae6ba0b747575f8562de738078457b", size = 1712444, upload-time = "2026-03-31T21:59:24.635Z" },
{ url = "https://files.pythonhosted.org/packages/6c/cf/9e1795b4160c58d29421eafd1a69c6ce351e2f7c8d3c6b7e4ca44aea1a5b/aiohttp-3.13.5-cp314-cp314-win32.whl", hash = "sha256:b20df693de16f42b2472a9c485e1c948ee55524786a0a34345511afdd22246f3", size = 438128, upload-time = "2026-03-31T21:59:27.291Z" },
{ url = "https://files.pythonhosted.org/packages/22/4d/eaedff67fc805aeba4ba746aec891b4b24cebb1a7d078084b6300f79d063/aiohttp-3.13.5-cp314-cp314-win_amd64.whl", hash = "sha256:f85c6f327bf0b8c29da7d93b1cabb6363fb5e4e160a32fa241ed2dce21b73162", size = 464029, upload-time = "2026-03-31T21:59:29.429Z" },
{ url = "https://files.pythonhosted.org/packages/79/11/c27d9332ee20d68dd164dc12a6ecdef2e2e35ecc97ed6cf0d2442844624b/aiohttp-3.13.5-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:1efb06900858bb618ff5cee184ae2de5828896c448403d51fb633f09e109be0a", size = 778758, upload-time = "2026-03-31T21:59:31.547Z" },
{ url = "https://files.pythonhosted.org/packages/04/fb/377aead2e0a3ba5f09b7624f702a964bdf4f08b5b6728a9799830c80041e/aiohttp-3.13.5-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:fee86b7c4bd29bdaf0d53d14739b08a106fdda809ca5fe032a15f52fae5fe254", size = 512883, upload-time = "2026-03-31T21:59:34.098Z" },
{ url = "https://files.pythonhosted.org/packages/bb/a6/aa109a33671f7a5d3bd78b46da9d852797c5e665bfda7d6b373f56bff2ec/aiohttp-3.13.5-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:20058e23909b9e65f9da62b396b77dfa95965cbe840f8def6e572538b1d32e36", size = 516668, upload-time = "2026-03-31T21:59:36.497Z" },
{ url = "https://files.pythonhosted.org/packages/79/b3/ca078f9f2fa9563c36fb8ef89053ea2bb146d6f792c5104574d49d8acb63/aiohttp-3.13.5-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8cf20a8d6868cb15a73cab329ffc07291ba8c22b1b88176026106ae39aa6df0f", size = 1883461, upload-time = "2026-03-31T21:59:38.723Z" },
{ url = "https://files.pythonhosted.org/packages/b7/e3/a7ad633ca1ca497b852233a3cce6906a56c3225fb6d9217b5e5e60b7419d/aiohttp-3.13.5-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:330f5da04c987f1d5bdb8ae189137c77139f36bd1cb23779ca1a354a4b027800", size = 1747661, upload-time = "2026-03-31T21:59:41.187Z" },
{ url = "https://files.pythonhosted.org/packages/33/b9/cd6fe579bed34a906d3d783fe60f2fa297ef55b27bb4538438ee49d4dc41/aiohttp-3.13.5-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:6f1cbf0c7926d315c3c26c2da41fd2b5d2fe01ac0e157b78caefc51a782196cf", size = 1863800, upload-time = "2026-03-31T21:59:43.84Z" },
{ url = "https://files.pythonhosted.org/packages/c0/3f/2c1e2f5144cefa889c8afd5cf431994c32f3b29da9961698ff4e3811b79a/aiohttp-3.13.5-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:53fc049ed6390d05423ba33103ded7281fe897cf97878f369a527070bd95795b", size = 1958382, upload-time = "2026-03-31T21:59:46.187Z" },
{ url = "https://files.pythonhosted.org/packages/66/1d/f31ec3f1013723b3babe3609e7f119c2c2fb6ef33da90061a705ef3e1bc8/aiohttp-3.13.5-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:898703aa2667e3c5ca4c54ca36cd73f58b7a38ef87a5606414799ebce4d3fd3a", size = 1803724, upload-time = "2026-03-31T21:59:48.656Z" },
{ url = "https://files.pythonhosted.org/packages/0e/b4/57712dfc6f1542f067daa81eb61da282fab3e6f1966fca25db06c4fc62d5/aiohttp-3.13.5-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:0494a01ca9584eea1e5fbd6d748e61ecff218c51b576ee1999c23db7066417d8", size = 1640027, upload-time = "2026-03-31T21:59:51.284Z" },
{ url = "https://files.pythonhosted.org/packages/25/3c/734c878fb43ec083d8e31bf029daae1beafeae582d1b35da234739e82ee7/aiohttp-3.13.5-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:6cf81fe010b8c17b09495cbd15c1d35afbc8fb405c0c9cf4738e5ae3af1d65be", size = 1806644, upload-time = "2026-03-31T21:59:53.753Z" },
{ url = "https://files.pythonhosted.org/packages/20/a5/f671e5cbec1c21d044ff3078223f949748f3a7f86b14e34a365d74a5d21f/aiohttp-3.13.5-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:c564dd5f09ddc9d8f2c2d0a301cd30a79a2cc1b46dd1a73bef8f0038863d016b", size = 1791630, upload-time = "2026-03-31T21:59:56.239Z" },
{ url = "https://files.pythonhosted.org/packages/0b/63/fb8d0ad63a0b8a99be97deac8c04dacf0785721c158bdf23d679a87aa99e/aiohttp-3.13.5-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:2994be9f6e51046c4f864598fd9abeb4fba6e88f0b2152422c9666dcd4aea9c6", size = 1809403, upload-time = "2026-03-31T21:59:59.103Z" },
{ url = "https://files.pythonhosted.org/packages/59/0c/bfed7f30662fcf12206481c2aac57dedee43fe1c49275e85b3a1e1742294/aiohttp-3.13.5-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:157826e2fa245d2ef46c83ea8a5faf77ca19355d278d425c29fda0beb3318037", size = 1634924, upload-time = "2026-03-31T22:00:02.116Z" },
{ url = "https://files.pythonhosted.org/packages/17/d6/fd518d668a09fd5a3319ae5e984d4d80b9a4b3df4e21c52f02251ef5a32e/aiohttp-3.13.5-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:a8aca50daa9493e9e13c0f566201a9006f080e7c50e5e90d0b06f53146a54500", size = 1836119, upload-time = "2026-03-31T22:00:04.756Z" },
{ url = "https://files.pythonhosted.org/packages/78/b7/15fb7a9d52e112a25b621c67b69c167805cb1f2ab8f1708a5c490d1b52fe/aiohttp-3.13.5-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:3b13560160d07e047a93f23aaa30718606493036253d5430887514715b67c9d9", size = 1772072, upload-time = "2026-03-31T22:00:07.494Z" },
{ url = "https://files.pythonhosted.org/packages/7e/df/57ba7f0c4a553fc2bd8b6321df236870ec6fd64a2a473a8a13d4f733214e/aiohttp-3.13.5-cp314-cp314t-win32.whl", hash = "sha256:9a0f4474b6ea6818b41f82172d799e4b3d29e22c2c520ce4357856fced9af2f8", size = 471819, upload-time = "2026-03-31T22:00:10.277Z" },
{ url = "https://files.pythonhosted.org/packages/62/29/2f8418269e46454a26171bfdd6a055d74febf32234e474930f2f60a17145/aiohttp-3.13.5-cp314-cp314t-win_amd64.whl", hash = "sha256:18a2f6c1182c51baa1d28d68fea51513cb2a76612f038853c0ad3c145423d3d9", size = 505441, upload-time = "2026-03-31T22:00:12.791Z" },
]
[[package]]
@@ -93,14 +93,14 @@ wheels = [
[[package]]
name = "anyio"
version = "4.12.1"
version = "4.13.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "idna" },
]
sdist = { url = "https://files.pythonhosted.org/packages/96/f0/5eb65b2bb0d09ac6776f2eb54adee6abe8228ea05b20a5ad0e4945de8aac/anyio-4.12.1.tar.gz", hash = "sha256:41cfcc3a4c85d3f05c932da7c26d0201ac36f72abd4435ba90d0464a3ffed703", size = 228685, upload-time = "2026-01-06T11:45:21.246Z" }
sdist = { url = "https://files.pythonhosted.org/packages/19/14/2c5dd9f512b66549ae92767a9c7b330ae88e1932ca57876909410251fe13/anyio-4.13.0.tar.gz", hash = "sha256:334b70e641fd2221c1505b3890c69882fe4a2df910cba14d97019b90b24439dc", size = 231622, upload-time = "2026-03-24T12:59:09.671Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/38/0e/27be9fdef66e72d64c0cdc3cc2823101b80585f8119b5c112c2e8f5f7dab/anyio-4.12.1-py3-none-any.whl", hash = "sha256:d405828884fc140aa80a3c667b8beed277f1dfedec42ba031bd6ac3db606ab6c", size = 113592, upload-time = "2026-01-06T11:45:19.497Z" },
{ url = "https://files.pythonhosted.org/packages/da/42/e921fccf5015463e32a3cf6ee7f980a6ed0f395ceeaa45060b61d86486c2/anyio-4.13.0-py3-none-any.whl", hash = "sha256:08b310f9e24a9594186fd75b4f73f4a4152069e3853f1ed8bfbf58369f4ad708", size = 114353, upload-time = "2026-03-24T12:59:08.246Z" },
]
[[package]]
@@ -324,15 +324,15 @@ wheels = [
[[package]]
name = "deno"
version = "2.7.7"
version = "2.7.10"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/87/b4/e893908807648b8c499a085cf47c9ca6418a060b0f12e73f128478ada409/deno-2.7.7.tar.gz", hash = "sha256:5798bba73f89ddf50fa33044c8a44fe708fb19ab77b3ef98d02f4124e760fb65", size = 8166, upload-time = "2026-03-19T13:57:09.905Z" }
sdist = { url = "https://files.pythonhosted.org/packages/e8/01/c03ed7db9adbd02a45de56037e2b685adc730775e8878229881ed907458d/deno-2.7.10.tar.gz", hash = "sha256:ea30a61f98c9a57b80f80a525a1d4687e36b7fcca133f813439c8431489e703b", size = 8165, upload-time = "2026-03-31T15:12:00.299Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/02/08/362f834c64798033ca56a02a1a4e8feca653b9b767aab4a854069ba8c801/deno-2.7.7-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:70be65294ee575b2e5ee66b587c459500984b1df17505fd6f5e7bffad402de0f", size = 46934365, upload-time = "2026-03-19T13:56:54.324Z" },
{ url = "https://files.pythonhosted.org/packages/4a/3f/cdbe9daa33e997f26610ee7f554e51ba2c8fd7a18abcbc9c6069e6386164/deno-2.7.7-py3-none-macosx_11_0_arm64.whl", hash = "sha256:65641b2dd299e3a4aae4f080d4e32d632bbcf44d77f72f97f61aa7b68ded4747", size = 43831345, upload-time = "2026-03-19T13:56:57.565Z" },
{ url = "https://files.pythonhosted.org/packages/25/e7/5f63b2a64fc2f7a7ce6c73e9e847c41034283890e6edec0b2791518b7edd/deno-2.7.7-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:cc90d761472df285a8709483d3615fbd2faf4bbc162530196b5a112e4a561016", size = 47571993, upload-time = "2026-03-19T13:57:00.833Z" },
{ url = "https://files.pythonhosted.org/packages/e7/f2/68f4bb53de09970744f905628cff011bd6964f2f00f263140dcc9412a7b5/deno-2.7.7-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:8ced70363e30a7e3f27f614ffd46d69ccf1dd57633f0df6a3c6375ed2c803aa7", size = 49577613, upload-time = "2026-03-19T13:57:03.766Z" },
{ url = "https://files.pythonhosted.org/packages/0e/db/2fa6239c0d4df46ef6f3f43d55133aeda6cdd6668c6044d275548a95da24/deno-2.7.7-py3-none-win_amd64.whl", hash = "sha256:e614f666c169ade86a3a089a15a32b9a2002d1ad3294f1fbc8a1bd50c2bac4ab", size = 48802184, upload-time = "2026-03-19T13:57:07.328Z" },
{ url = "https://files.pythonhosted.org/packages/39/4b/a28d8c7ff5d797f52098dcbce91b3ff8394bdbd0dd07cb4c87b032ead539/deno-2.7.10-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:4e361633c1ce6ec439d312911ec230e4e060c4e5ca957e8f58823129af511b13", size = 47849281, upload-time = "2026-03-31T15:11:43.879Z" },
{ url = "https://files.pythonhosted.org/packages/b5/da/d572cf9f195aaf317a0e222af10f8adf3f3acfd114f80fec78c110fb66e6/deno-2.7.10-py3-none-macosx_11_0_arm64.whl", hash = "sha256:6c4c03e583c4c4d5647ec97690038a4b7c00a7ad076949b5ce26203857b1c85a", size = 44608218, upload-time = "2026-03-31T15:11:47.664Z" },
{ url = "https://files.pythonhosted.org/packages/e6/da/f77fd4852d84063728d618b9f4c088b31d27a999b6dfc3dd1f9623dd56a9/deno-2.7.10-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:b415a36b63e3c5c478180a5cc0e9f517095005086144dbe52b34268b397c404b", size = 48384632, upload-time = "2026-03-31T15:11:50.816Z" },
{ url = "https://files.pythonhosted.org/packages/dc/13/a41c3aba09103cd31ddbe2fc8fc98df5118df7b23d4cee248926367c6469/deno-2.7.10-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:1f4aab0be642692205df91e39eff1db774d6b4c5d8dfcff0669d014fd0c80cba", size = 50420236, upload-time = "2026-03-31T15:11:54.139Z" },
{ url = "https://files.pythonhosted.org/packages/c7/a9/423f671846107bed51c405bbd1e32782f0b39edd5d075e4f806b8eea77f5/deno-2.7.10-py3-none-win_amd64.whl", hash = "sha256:3c2ee1773cf48b0fe9e74d23da3b6f9b685240e90db81ce6f5c8c0922c08b992", size = 49403842, upload-time = "2026-03-31T15:11:57.728Z" },
]
[[package]]
@@ -728,11 +728,11 @@ wheels = [
[[package]]
name = "pygments"
version = "2.19.2"
version = "2.20.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/b0/77/a5b8c569bf593b0140bde72ea885a803b82086995367bf2037de0159d924/pygments-2.19.2.tar.gz", hash = "sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887", size = 4968631, upload-time = "2025-06-21T13:39:12.283Z" }
sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl", hash = "sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b", size = 1225217, upload-time = "2025-06-21T13:39:07.939Z" },
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
]
[[package]]
@@ -822,7 +822,7 @@ wheels = [
[[package]]
name = "requests"
version = "2.32.5"
version = "2.33.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "certifi" },
@@ -830,9 +830,9 @@ dependencies = [
{ name = "idna" },
{ name = "urllib3" },
]
sdist = { url = "https://files.pythonhosted.org/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" }
sdist = { url = "https://files.pythonhosted.org/packages/5f/a4/98b9c7c6428a668bf7e42ebb7c79d576a1c3c1e3ae2d47e674b468388871/requests-2.33.1.tar.gz", hash = "sha256:18817f8c57c6263968bc123d237e3b8b08ac046f5456bd1e307ee8f4250d3517", size = 134120, upload-time = "2026-03-30T16:09:15.531Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" },
{ url = "https://files.pythonhosted.org/packages/d7/8e/7540e8a2036f79a125c1d2ebadf69ed7901608859186c856fa0388ef4197/requests-2.33.1-py3-none-any.whl", hash = "sha256:4e6d1ef462f3626a1f0a0a9c42dd93c63bad33f9f1c1937509b8c5c8718ab56a", size = 64947, upload-time = "2026-03-30T16:09:13.83Z" },
]
[[package]]