Compare commits

...

25 Commits

Author SHA1 Message Date
Alex Shnitman 5d96a581b9 allow filtering out members-only videos in subscriptions (closes #971) 2026-04-28 22:02:05 +03:00
Alex Shnitman 4f83174d05 implement time-clipped downloads (closes #969, replaces #907) 2026-04-26 23:07:50 +03:00
Alex Shnitman 91ee8312bf title filter for subscriptions (closes #968) 2026-04-26 22:51:48 +03:00
dependabot[bot] d89a5ddbe5 Bump aquasecurity/trivy-action in the github-actions group
Bumps the github-actions group with 1 update: [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action).


Updates `aquasecurity/trivy-action` from 0.35.0 to 0.36.0
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](https://github.com/aquasecurity/trivy-action/compare/0.35.0...v0.36.0)

---
updated-dependencies:
- dependency-name: aquasecurity/trivy-action
  dependency-version: 0.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-26 16:12:39 +00:00
Alex Shnitman abb9492d21 upgrade dependencies 2026-04-21 16:20:39 +03:00
Alex Shnitman 23de9824f0 cr fixes 2026-04-21 16:13:58 +03:00
rdiaz738 0ea934c08f Updated import and fixed race condition 2026-04-20 17:24:16 -07:00
Alex Shnitman e9f979b349 fix yt-dlp options overrides (closes #958) 2026-04-18 08:46:29 +03:00
Alex Shnitman ab42325db5 upgrade dependencies 2026-04-16 22:30:42 +03:00
Alex Shnitman 1a32eba474 fix PUBLIC_HOST_URL without a trailing slash (closes #959) 2026-04-16 22:08:08 +03:00
Alex Shnitman 29ccc42409 don't run workflow on README changes 2026-04-13 20:49:10 +03:00
Alex Shnitman f2d71cbe2e add more CORS details 2026-04-13 20:45:20 +03:00
Alex Shnitman 03f71fd257 fix asterisk CORS_ALLOWED_ORIGINS, mentioned in #955 2026-04-13 19:02:27 +03:00
Alex Shnitman 210c607c53 fix pnpm build 2026-04-12 23:07:22 +03:00
dependabot[bot] 381896901a Bump softprops/action-gh-release from 2 to 3 in the github-actions group
Bumps the github-actions group with 1 update: [softprops/action-gh-release](https://github.com/softprops/action-gh-release).


Updates `softprops/action-gh-release` from 2 to 3
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/v2...v3)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-12 16:12:38 +00:00
Alex Shnitman 4330d3b6c6 fix yt-dlp options examples 2026-04-10 14:06:08 +03:00
Alex Shnitman 06c4a2c4a8 update documentation 2026-04-10 08:38:32 +03:00
Alex Shnitman 388aeb180d Merge branch 'bgervan/master' 2026-04-10 08:10:00 +03:00
Alex Shnitman aa60420ead document CORS_ALLOWED_ORIGINS variable 2026-04-10 08:09:20 +03:00
Benjamin Gervan a6e8617ad8 Don't mark live streams as seen 2026-04-10 06:41:45 +02:00
az10b 0072d3488a Fix permissive CORS policy that allows cross-origin attacks
The on_prepare handler unconditionally reflected the Origin request
header into Access-Control-Allow-Origin, and Socket.IO was configured
with cors_allowed_origins='*'. This allowed any website to make
authenticated cross-origin requests to all API endpoints, enabling
cross-origin download initiation, cookie overwrite, and data deletion.

Replace the blanket origin reflection with an explicit allowlist via
the CORS_ALLOWED_ORIGINS environment variable. When unset, cross-origin
requests are denied by default. Users who need cross-origin access can
set CORS_ALLOWED_ORIGINS to a comma-separated list of trusted origins.
2026-04-09 19:45:51 -05:00
Alex Shnitman 0b3645aea1 upgrade dependencies 2026-04-09 21:00:26 +03:00
Alex Shnitman 2c838e3d3d Merge branch 'dependabot/github_actions/github-actions-7530ffc9b9' of https://github.com/alexta69/metube into McSwindler/master 2026-04-09 20:59:13 +03:00
McSwindler d38d7bd1b1 fix: handle playlists that don't supply video ids 2026-04-09 10:15:11 -05:00
dependabot[bot] b7709d3536 Bump astral-sh/setup-uv from 6 to 7 in the github-actions group
Bumps the github-actions group with 1 update: [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv).


Updates `astral-sh/setup-uv` from 6 to 7
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/v6...v7)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-05 16:12:43 +00:00
23 changed files with 2290 additions and 736 deletions
+5 -3
View File
@@ -4,6 +4,8 @@ on:
push:
branches:
- 'master'
paths-ignore:
- '**.md'
jobs:
quality-checks:
@@ -32,7 +34,7 @@ jobs:
env:
CI: true
- name: Install uv
uses: astral-sh/setup-uv@v6
uses: astral-sh/setup-uv@v7
- name: Install Python dependencies
run: uv sync --frozen --group dev
- name: Run backend smoke checks
@@ -40,7 +42,7 @@ jobs:
- name: Run backend tests
run: uv run pytest app/tests/
- name: Run Trivy filesystem scan
uses: aquasecurity/trivy-action@0.35.0
uses: aquasecurity/trivy-action@v0.36.0
with:
scan-type: fs
scan-ref: .
@@ -209,7 +211,7 @@ jobs:
git push origin ":refs/tags/$TAG_NAME" || true
fi
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
tag_name: ${{ steps.date.outputs.date }}
name: Release ${{ steps.date.outputs.date }}
+60
View File
@@ -0,0 +1,60 @@
# Agent Guidelines
## README.md size constraint
The README.md is synced to Docker Hub, which has a **25,000 character limit**.
Any change to README.md **must** keep the file under 25,000 characters (`wc -c README.md`).
If an addition would exceed the limit, trim existing prose elsewhere — prefer tightening verbose descriptions over removing sections.
## Tech stack
- **Backend:** Python 3.13+, aiohttp, python-socketio 5.x, yt-dlp
- **Frontend:** Angular 21, TypeScript, Bootstrap 5, SASS, ngx-socket-io
- **Package managers:** uv (Python), pnpm (frontend)
- **Container:** Multi-stage Docker (Node builder + Python runtime), multi-arch (amd64/arm64)
## Build & test commands
```bash
# Frontend (run from ui/)
pnpm install --frozen-lockfile
pnpm run lint
pnpm run build
pnpm exec ng test --watch=false
# Backend (run from repo root)
uv sync --frozen --group dev
python -m compileall app
uv run pytest app/tests/
```
All of these run in CI (`.github/workflows/main.yml`) on every push to master and must pass.
## Code style
Follow `.editorconfig`:
- Python: 4-space indent
- Everything else (TypeScript, YAML, JSON, HTML): 2-space indent
- UTF-8, LF line endings, trim trailing whitespace, final newline
Frontend additionally uses ESLint (`ui/eslint.config.js`) and Prettier (config in `ui/package.json`: `printWidth=100`, `singleQuote=true`).
## Project structure
```
app/main.py — HTTP server, Socket.IO events, REST API routes, Config class
app/ytdl.py — Download queue logic, yt-dlp integration
app/subscriptions.py — Channel/playlist subscription manager
app/state_store.py — JSON-based persistent storage with atomic writes
app/dl_formats.py — Video/audio codec/quality mapping
app/tests/ — pytest tests (asyncio_mode=auto)
ui/src/app/ — Angular standalone components (no NgModules)
```
## Key conventions
- Backend configuration lives in the `Config` class in `app/main.py` with env-var defaults in `_DEFAULTS`. New env vars go there.
- Real-time communication uses Socket.IO events, not REST polling.
- Frontend uses standalone Angular components with `inject()` for DI, RxJS Subjects for state, and `takeUntilDestroyed()` for cleanup.
- State is persisted as JSON files via `AtomicJsonStore` in `app/state_store.py`.
- No pre-commit hooks — linting and tests are enforced in CI only.
+21 -30
View File
@@ -82,6 +82,7 @@ Certain values can be set via environment variables, using the `-e` parameter on
* __HTTPS__: Use `https` instead of `http` (__CERTFILE__ and __KEYFILE__ required). Defaults to `false`.
* __CERTFILE__: HTTPS certificate file path.
* __KEYFILE__: HTTPS key file path.
* __CORS_ALLOWED_ORIGINS__: Comma-separated list of origins permitted to make cross-origin requests to the MeTube API. When unset or empty, all cross-origin requests are denied. Set to `*` to allow all origins. This must be configured for [browser extensions](#-browser-extensions), [bookmarklets](#-bookmarklet), and any other browser-based tools that contact MeTube from a different origin. For browser extensions use `*` (see below); for bookmarklets you can list specific sites, e.g. `https://www.youtube.com,https://www.vimeo.com`.
* __ROBOTS_TXT__: A path to a `robots.txt` file mounted in the container.
### 🏠 Basic Setup
@@ -103,11 +104,13 @@ MeTube lets you customize how [yt-dlp](https://github.com/yt-dlp/yt-dlp) behaves
When a download starts, these layers are combined in order. If the same option appears in more than one layer, the more specific one wins: per-download overrides beat presets, and presets beat global options.
In JSON presets and overrides, setting an option to **`null`** clears that option for that download (for example, `"download_archive": null` overrides a global archive path so the archive is not used). This follows yt-dlps usual meaning of `None` for that option.
### Option format
yt-dlp options in MeTube are expressed as JSON objects. The keys are yt-dlp API option names, which roughly correspond to command-line flags with dashes replaced by underscores. For example, the command-line flag `--embed-thumbnail` becomes `"embed_thumbnail": true` in JSON.
yt-dlp options in MeTube are expressed as JSON objects. The keys are yt-dlp API option names, which roughly correspond to command-line flags with dashes replaced by underscores. For example, the command-line flag `--write-subs` becomes `"writesubtitles": true` in JSON.
> **Tip:** Some command-line flags don't have a direct single-key equivalent — for instance, `--recode-video` must be expressed via `"postprocessors"`. A full list of available API options can be found [in the yt-dlp source](https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/YoutubeDL.py#L224), and [this conversion script](https://github.com/yt-dlp/yt-dlp/blob/master/devscripts/cli_to_api.py) can help translate command-line flags to their API equivalents.
> **Tip:** Some command-line flags don't have a direct single-key equivalent — for instance, `--embed-thumbnail` and `--recode-video` must be expressed via `"postprocessors"`. A full list of available API options can be found [in the yt-dlp source](https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/YoutubeDL.py#L224), and [this conversion script](https://github.com/yt-dlp/yt-dlp/blob/master/devscripts/cli_to_api.py) can help translate command-line flags to their API equivalents.
### Global options
@@ -117,7 +120,7 @@ Global options form the baseline for every download. There are two ways to defin
```yaml
environment:
- 'YTDL_OPTIONS={"writesubtitles": true, "subtitleslangs": ["en", "de"], "updatetime": false, "embed_thumbnail": true}'
- 'YTDL_OPTIONS={"writesubtitles": true, "subtitleslangs": ["en", "de"], "updatetime": false, "writethumbnail": true}'
```
**Via a JSON file** (`YTDL_OPTIONS_FILE`) — mount a file into the container and point to it:
@@ -136,7 +139,7 @@ where `ytdl-options.json` contains:
"writesubtitles": true,
"subtitleslangs": ["en", "de"],
"updatetime": false,
"embed_thumbnail": true
"writethumbnail": true
}
```
@@ -203,6 +206,8 @@ When a download starts, the final set of yt-dlp options is built in this order:
2. Apply each selected **preset** in order (later presets overwrite earlier ones for conflicting keys).
3. Apply any **per-download overrides** on top (overwrite everything else for conflicting keys).
MeTube always forces its own flat-extract behaviour during the initial metadata fetch (`extract_flat`, `noplaylist`, etc.); presets cannot override those keys for that phase.
**Example:** Suppose your global options set `"writesubtitles": false`, but you select a preset that sets `"writesubtitles": true`. Subtitles will be written for that download because the preset overrides the global setting. If you additionally enter `{"writesubtitles": false}` in the per-download overrides field, that value wins and subtitles will not be written.
### Configuration cookbooks
@@ -225,7 +230,9 @@ In case you need to use your browser's cookies with MeTube, for example to downl
## 🔌 Browser extensions
Browser extensions allow right-clicking videos and sending them directly to MeTube. Please note that if you're on an HTTPS page, your MeTube instance must be behind an HTTPS reverse proxy (see below) for the extensions to work.
Browser extensions allow right-clicking videos and sending them directly to MeTube. If you're on an HTTPS page, your MeTube instance must be behind an HTTPS reverse proxy (see below) for extensions to work.
Since browser extensions make requests from their own origin (`chrome-extension://...` or `moz-extension://...`), you must set `CORS_ALLOWED_ORIGINS=*` for them to work.
__Chrome:__ contributed by [Rpsl](https://github.com/rpsl). You can install it from [Google Chrome Webstore](https://chrome.google.com/webstore/detail/metube-downloader/fbmkmdnlhacefjljljlbhkodfmfkijdh) or use developer mode and install [from sources](https://github.com/Rpsl/metube-browser-extension).
@@ -239,6 +246,8 @@ __Firefox:__ contributed by [nanocortex](https://github.com/nanocortex). You can
[kushfest](https://github.com/kushfest) has created a Chrome bookmarklet for sending the currently open webpage to MeTube. Please note that if you're on an HTTPS page, your MeTube instance must be configured with `HTTPS` as `true` in the environment, or be behind an HTTPS reverse proxy (see below) for the bookmarklet to work.
Since bookmarklets run in the context of the current page (e.g. youtube.com), the requests they make to MeTube are cross-origin. You must add the origins of sites where you use the bookmarklet to the __CORS_ALLOWED_ORIGINS__ environment variable, otherwise the browser will block the requests. For example, to use the bookmarklet on YouTube and Vimeo: `CORS_ALLOWED_ORIGINS=https://www.youtube.com,https://www.vimeo.com`.
GitHub doesn't allow embedding JavaScript as a link, so the bookmarklet has to be created manually by copying the following code to a new bookmark you create on your bookmarks bar. Change the hostname in the URL below to point to your MeTube instance.
```javascript
@@ -251,23 +260,15 @@ javascript:!function(){xhr=new XMLHttpRequest();xhr.open("POST","https://metube.
javascript:(function(){xhr=new XMLHttpRequest();xhr.open("POST","https://metube.domain.com/add");xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function(){if(xhr.status==200){alert("Sent to metube!")}else{alert("Send to metube failed. Check the javascript console for clues.")}}})();
```
The above bookmarklets use `alert()` as a success/failure notification. The following will show a toast message instead:
Chrome:
The above bookmarklets use `alert()` for notifications. This variant shows a toast instead (Chrome — for Firefox, replace the `!function(){...}()` wrapper with `(function(){...})()`):
```javascript
javascript:!function(){function notify(msg) {var sc = document.scrollingElement.scrollTop; var text = document.createElement('span');text.innerHTML=msg;var ts = text.style;ts.all = 'revert';ts.color = '#000';ts.fontFamily = 'Verdana, sans-serif';ts.fontSize = '15px';ts.backgroundColor = 'white';ts.padding = '15px';ts.border = '1px solid gainsboro';ts.boxShadow = '3px 3px 10px';ts.zIndex = '100';document.body.appendChild(text);ts.position = 'absolute'; ts.top = 50 + sc + 'px'; ts.left = (window.innerWidth / 2)-(text.offsetWidth / 2) + 'px'; setTimeout(function () { text.style.visibility = "hidden"; }, 1500);}xhr=new XMLHttpRequest();xhr.open("POST","https://metube.domain.com/add");xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function() { if(xhr.status==200){notify("Sent to metube!")}else {notify("Send to metube failed. Check the javascript console for clues.")}}}();
```
Firefox:
```javascript
javascript:(function(){function notify(msg) {var sc = document.scrollingElement.scrollTop; var text = document.createElement('span');text.innerHTML=msg;var ts = text.style;ts.all = 'revert';ts.color = '#000';ts.fontFamily = 'Verdana, sans-serif';ts.fontSize = '15px';ts.backgroundColor = 'white';ts.padding = '15px';ts.border = '1px solid gainsboro';ts.boxShadow = '3px 3px 10px';ts.zIndex = '100';document.body.appendChild(text);ts.position = 'absolute'; ts.top = 50 + sc + 'px'; ts.left = (window.innerWidth / 2)-(text.offsetWidth / 2) + 'px'; setTimeout(function () { text.style.visibility = "hidden"; }, 1500);}xhr=new XMLHttpRequest();xhr.open("POST","https://metube.domain.com/add");xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function() { if(xhr.status==200){notify("Sent to metube!")}else {notify("Send to metube failed. Check the javascript console for clues.")}}})();
```
## ⚡ Raycast extension
[dotvhs](https://github.com/dotvhs) has created an [extension for Raycast](https://www.raycast.com/dot/metube) that allows adding videos to MeTube directly from Raycast.
[dotvhs](https://github.com/dotvhs) has created an [extension for Raycast](https://www.raycast.com/dot/metube) for adding videos to MeTube directly from Raycast.
## 🔒 HTTPS support, and running behind a reverse proxy
@@ -291,11 +292,9 @@ services:
- KEYFILE=/ssl/key.pem
```
It's also possible to run MeTube behind a reverse proxy, in order to support authentication. HTTPS support can also be added in this way.
MeTube can also run behind a reverse proxy for HTTPS termination or authentication. When serving under a subdirectory, set `URL_PREFIX` accordingly.
When running behind a reverse proxy which remaps the URL (i.e. serves MeTube under a subdirectory and not under root), don't forget to set the URL_PREFIX environment variable to the correct value.
If you're using the [linuxserver/swag](https://docs.linuxserver.io/general/swag) image for your reverse proxying needs (which I can heartily recommend), it already includes ready snippets for proxying MeTube both in [subfolder](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subfolder.conf.sample) and [subdomain](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subdomain.conf.sample) modes under the `nginx/proxy-confs` directory in the configuration volume. It also includes Authelia which can be used for authentication.
The [linuxserver/swag](https://docs.linuxserver.io/general/swag) image includes ready-made snippets for MeTube in [subfolder](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subfolder.conf.sample) and [subdomain](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subdomain.conf.sample) modes, plus Authelia for authentication.
### 🌐 NGINX
@@ -347,28 +346,20 @@ example.com {
## 🔄 Updating yt-dlp
The engine which powers the actual video downloads in MeTube is [yt-dlp](https://github.com/yt-dlp/yt-dlp). Since video sites regularly change their layouts, frequent updates of yt-dlp are required to keep up.
There's an automatic nightly build of MeTube which looks for a new version of yt-dlp, and if one exists, the build pulls it and publishes an updated docker image. Therefore, in order to keep up with the changes, it's recommended that you update your MeTube container regularly with the latest image.
I recommend installing and setting up [watchtower](https://github.com/nicholas-fedor/watchtower) for this purpose.
MeTube is powered by [yt-dlp](https://github.com/yt-dlp/yt-dlp), which requires frequent updates as video sites change their layouts. A nightly build automatically publishes a new Docker image whenever a new yt-dlp version is available, so keep your container up to date — [watchtower](https://github.com/nicholas-fedor/watchtower) works well for this.
## 🔧 Troubleshooting and submitting issues
Before asking a question or submitting an issue for MeTube, please remember that MeTube is only a UI for [yt-dlp](https://github.com/yt-dlp/yt-dlp). Any issues you might be experiencing with authentication to video websites, postprocessing, permissions, other `YTDL_OPTIONS` configurations which seem not to work, or anything else that concerns the workings of the underlying yt-dlp library, need not be opened on the MeTube project. In order to debug and troubleshoot them, it's advised to try using the yt-dlp binary directly first, bypassing the UI, and once that is working, importing the options that worked for you into `YTDL_OPTIONS`.
In order to test with the yt-dlp command directly, you can either download it and run it locally, or for a better simulation of its actual conditions, you can run it within the MeTube container itself. Assuming your MeTube container is called `metube`, run the following on your Docker host to get a shell inside the container:
MeTube is only a UI for [yt-dlp](https://github.com/yt-dlp/yt-dlp). Issues with authentication, postprocessing, permissions, or `YTDL_OPTIONS` should be debugged with yt-dlp directly first — once working, import those options into MeTube. To test inside the container:
```bash
docker exec -ti metube sh
cd /downloads
```
Once there, you can use the yt-dlp command freely.
## 💡 Submitting feature requests
MeTube development relies on code contributions by the community. The program as it currently stands fits my own use cases, and is therefore feature-complete as far as I'm concerned. If your use cases are different and require additional features, please feel free to submit PRs that implement those features. It's advisable to create an issue first to discuss the planned implementation, because in an effort to reduce bloat, some PRs may not be accepted. However, note that opening a feature request when you don't intend to implement the feature will rarely result in the request being fulfilled.
MeTube development relies on community contributions. If you need additional features, please submit a PR. Create an issue first to discuss the implementation — some PRs may not be accepted to reduce bloat. Feature requests without an accompanying PR are unlikely to be fulfilled.
## 🛠️ Building and running locally
+178 -5
View File
@@ -14,10 +14,11 @@ import logging
import json
import pathlib
import re
from urllib.parse import parse_qs, urlencode, urlparse, urlunparse
from watchfiles import DefaultFilter, Change, awatch
from ytdl import DownloadQueueNotifier, DownloadQueue, Download
from subscriptions import SubscriptionManager, SubscriptionNotifier, SubscriptionInfo
from subscriptions import SubscriptionManager, SubscriptionNotifier, SubscriptionInfo, coerce_optional_bool
from yt_dlp.version import __version__ as yt_dlp_version
log = logging.getLogger('main')
@@ -60,6 +61,7 @@ class Config:
'YTDL_OPTIONS_PRESETS': '{}',
'YTDL_OPTIONS_PRESETS_FILE': '',
'ALLOW_YTDL_OPTIONS_OVERRIDES': 'false',
'CORS_ALLOWED_ORIGINS': '',
'ROBOTS_TXT': '',
'HOST': '0.0.0.0',
'PORT': '8081',
@@ -91,6 +93,11 @@ class Config:
if not self.URL_PREFIX.endswith('/'):
self.URL_PREFIX += '/'
for attr in ('PUBLIC_HOST_URL', 'PUBLIC_HOST_AUDIO_URL'):
val = getattr(self, attr)
if val and not val.endswith('/'):
setattr(self, attr, val + '/')
# Convert relative addresses to absolute addresses to prevent the failure of file address comparison
if self.YTDL_OPTIONS_FILE and self.YTDL_OPTIONS_FILE.startswith('.'):
self.YTDL_OPTIONS_FILE = str(Path(self.YTDL_OPTIONS_FILE).resolve())
@@ -223,7 +230,8 @@ class ObjectSerializer(json.JSONEncoder):
serializer = ObjectSerializer()
app = web.Application()
sio = socketio.AsyncServer(cors_allowed_origins='*')
_cors_origins = [o.strip() for o in config.CORS_ALLOWED_ORIGINS.split(',') if o.strip()] if config.CORS_ALLOWED_ORIGINS else []
sio = socketio.AsyncServer(cors_allowed_origins=_cors_origins if _cors_origins else [])
sio.attach(app, socketio_path=config.URL_PREFIX + 'socket.io')
routes = web.RouteTableDef()
VALID_SUBTITLE_FORMATS = {'srt', 'txt', 'vtt', 'ttml', 'sbv', 'scc', 'dfxp'}
@@ -253,6 +261,115 @@ def _parse_ytdl_options_overrides(value, *, enabled: bool) -> dict:
return value
_YOUTUBE_T_COMPACT_RE = re.compile(
r'^(?:(\d+)h)?(?:(\d+)m)?(?:(\d+)(?:s)?)?$',
re.IGNORECASE,
)
def _parse_youtube_t_compact(value: str) -> float | None:
"""Parse YouTube-style ``t`` values: ``885``, ``885s``, ``14m45s``, ``1h2m3s``."""
v = value.strip()
if not v:
return None
if re.fullmatch(r'-?\d+(\.\d+)?', v):
sec = float(v)
return sec if sec >= 0 else None
m = _YOUTUBE_T_COMPACT_RE.match(v)
if m and any(m.groups()):
hours = int(m.group(1) or 0)
minutes = int(m.group(2) or 0)
seconds = int(m.group(3) or 0)
total = hours * 3600 + minutes * 60 + seconds
return float(total) if total >= 0 else None
return None
def _parse_clock_timestamp(s: str) -> float:
"""Parse ``MM:SS``, ``H:MM:SS``, or single segment as seconds (with optional decimals)."""
part = s.strip()
if not part:
raise ValueError('empty timestamp')
segments = part.split(':')
if len(segments) > 3:
raise ValueError('too many segments')
try:
nums = [float(x) for x in segments]
except ValueError as exc:
raise ValueError('invalid number') from exc
if any(x < 0 for x in nums):
raise ValueError('negative segment')
if len(segments) == 1:
return nums[0]
if len(segments) == 2:
return nums[0] * 60 + nums[1]
return nums[0] * 3600 + nums[1] * 60 + nums[2]
def _parse_clip_timestamp_value(value) -> float:
"""Coerce a clip boundary from JSON to seconds (non-negative)."""
if isinstance(value, bool):
raise web.HTTPBadRequest(reason='clip timestamp must be a number or string')
if isinstance(value, (int, float)):
if value < 0:
raise web.HTTPBadRequest(reason='clip timestamp must be non-negative')
return float(value)
s = str(value).strip()
if not s:
raise web.HTTPBadRequest(reason='clip timestamp cannot be empty')
if ':' in s:
try:
return _parse_clock_timestamp(s)
except ValueError as exc:
raise web.HTTPBadRequest(reason='invalid clip timestamp format') from exc
compact = _parse_youtube_t_compact(s)
if compact is not None:
return compact
raise web.HTTPBadRequest(reason='invalid clip timestamp format')
def _optional_clip_field(raw) -> float | None:
if raw is None:
return None
if isinstance(raw, str) and not raw.strip():
return None
return _parse_clip_timestamp_value(raw)
def _clip_field_provided_in_post(raw) -> bool:
if raw is None:
return False
if isinstance(raw, str) and not raw.strip():
return False
return True
def _extract_t_query_from_url(url: str) -> tuple[str, float | None]:
"""If ``t=`` is present and parseable, return URL without ``t`` and start seconds."""
try:
parsed = urlparse(url)
params = parse_qs(parsed.query)
except Exception:
return url, None
t_values = params.get('t')
if not t_values:
return url, None
start = _parse_youtube_t_compact(t_values[0])
if start is None:
return url, None
filtered = {k: v for k, v in params.items() if k != 't'}
new_query = urlencode(filtered, doseq=True)
cleaned = urlunparse((
parsed.scheme,
parsed.netloc,
parsed.path,
parsed.params,
new_query,
parsed.fragment,
))
return cleaned, float(start)
def _parse_ytdl_options_presets(post: dict) -> list[str]:
"""Normalize preset names from add/subscribe body; supports list or legacy singular string."""
raw = post.get('ytdl_options_presets')
@@ -535,6 +652,39 @@ def parse_download_options(post: dict) -> dict:
except (TypeError, ValueError) as exc:
raise web.HTTPBadRequest(reason='playlist_item_limit must be an integer') from exc
clip_start_raw = post.get('clip_start')
clip_end_raw = post.get('clip_end')
clip_start: float | None
clip_end: float | None
if download_type in ('captions', 'thumbnail'):
if _clip_field_provided_in_post(clip_start_raw) or _clip_field_provided_in_post(clip_end_raw):
raise web.HTTPBadRequest(
reason='clip_start and clip_end are only supported for video and audio downloads',
)
clip_start = None
clip_end = None
else:
cleaned_url, url_t = _extract_t_query_from_url(url)
if url_t is not None:
url = cleaned_url
explicit_start = _optional_clip_field(clip_start_raw)
explicit_end = _optional_clip_field(clip_end_raw)
explicit_start_provided = _clip_field_provided_in_post(clip_start_raw)
explicit_end_provided = _clip_field_provided_in_post(clip_end_raw)
if explicit_start_provided:
clip_start = explicit_start
elif explicit_end_provided:
clip_start = 0.0
elif url_t is not None:
clip_start = url_t
else:
clip_start = None
clip_end = explicit_end
if clip_end is not None and clip_start is None:
clip_start = 0.0
if clip_start is not None and clip_end is not None and clip_end <= clip_start:
raise web.HTTPBadRequest(reason='clip_end must be greater than clip_start')
return {
'url': url,
'download_type': download_type,
@@ -551,6 +701,8 @@ def parse_download_options(post: dict) -> dict:
'subtitle_mode': subtitle_mode,
'ytdl_options_presets': ytdl_options_presets,
'ytdl_options_overrides': ytdl_options_overrides,
'clip_start': clip_start,
'clip_end': clip_end,
}
@@ -587,6 +739,8 @@ async def add(request):
o['subtitle_mode'],
o['ytdl_options_presets'],
o['ytdl_options_overrides'],
o['clip_start'],
o['clip_end'],
)
return web.Response(text=serializer.encode(status))
@@ -620,6 +774,17 @@ async def subscribe(request):
raise web.HTTPBadRequest(reason='check_interval_minutes must be an integer') from exc
if cic < 1:
raise web.HTTPBadRequest(reason='check_interval_minutes must be at least 1')
if o.get('clip_start') is not None or o.get('clip_end') is not None:
raise web.HTTPBadRequest(reason='clip options are not supported for subscriptions')
try:
skip_subscriber_only = coerce_optional_bool(
post.get('skip_subscriber_only'),
default=False,
field_name='skip_subscriber_only',
)
except ValueError as exc:
raise web.HTTPBadRequest(reason=str(exc)) from exc
result = await submgr.add_subscription(
o['url'],
@@ -638,6 +803,8 @@ async def subscribe(request):
subtitle_mode=o['subtitle_mode'],
ytdl_options_presets=o['ytdl_options_presets'],
ytdl_options_overrides=o['ytdl_options_overrides'],
title_regex=post.get('title_regex'),
skip_subscriber_only=skip_subscriber_only,
)
return web.Response(text=serializer.encode(result))
@@ -653,7 +820,12 @@ async def subscriptions_update(request):
sub_id = post.get('id')
if not sub_id:
raise web.HTTPBadRequest(reason='missing subscription id')
changes = {k: v for k, v in post.items() if k != 'id' and k in ('enabled', 'check_interval_minutes', 'name')}
changes = {
k: v
for k, v in post.items()
if k != 'id'
and k in ('enabled', 'check_interval_minutes', 'name', 'title_regex', 'skip_subscriber_only')
}
if not changes:
raise web.HTTPBadRequest(reason='no valid fields to update')
log.info("Subscription update requested for %s: %s", sub_id, sorted(changes.keys()))
@@ -912,8 +1084,9 @@ app.router.add_route('OPTIONS', config.URL_PREFIX + 'upload-cookies', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'delete-cookies', add_cors)
async def on_prepare(request, response):
if 'Origin' in request.headers:
response.headers['Access-Control-Allow-Origin'] = request.headers['Origin']
origin = request.headers.get('Origin')
if origin and _cors_origins and ('*' in _cors_origins or origin in _cors_origins):
response.headers['Access-Control-Allow-Origin'] = origin
response.headers['Access-Control-Allow-Headers'] = 'Content-Type'
app.on_response_prepare.append(on_prepare)
+129 -6
View File
@@ -6,6 +6,7 @@ import asyncio
import copy
import logging
import os
import re
import time
import types
import uuid
@@ -126,6 +127,21 @@ def _entry_id(entry: dict) -> Optional[str]:
return url
def _is_subscriber_only_entry(entry: dict) -> bool:
"""True when yt-dlp marks the entry as channel member-only (subscriber_only availability)."""
return str(entry.get("availability") or "") == "subscriber_only"
def coerce_optional_bool(value: Any, *, default: bool = False, field_name: str = "value") -> bool:
"""Parse optional JSON booleans for subscription settings."""
if value is None:
return default
try:
return _coerce_bool(value)
except ValueError as exc:
raise ValueError(f"{field_name} must be a boolean") from exc
@dataclass
class SubscriptionInfo:
id: str
@@ -147,6 +163,8 @@ class SubscriptionInfo:
subtitle_mode: str = "prefer_manual"
ytdl_options_presets: list[str] = field(default_factory=list)
ytdl_options_overrides: dict[str, Any] = field(default_factory=dict)
title_regex: str = ""
skip_subscriber_only: bool = False
last_checked: Optional[float] = None
seen_ids: list[str] = field(default_factory=list)
error: Optional[str] = None
@@ -167,6 +185,8 @@ class SubscriptionInfo:
"format": self.format,
"quality": self.quality,
"folder": self.folder,
"title_regex": self.title_regex,
"skip_subscriber_only": self.skip_subscriber_only,
"last_checked": self.last_checked,
"seen_count": len(self.seen_ids),
"error": self.error,
@@ -194,6 +214,8 @@ def _subscription_to_record(sub: SubscriptionInfo) -> dict[str, Any]:
"subtitle_mode": sub.subtitle_mode,
"ytdl_options_presets": list(sub.ytdl_options_presets),
"ytdl_options_overrides": sub.ytdl_options_overrides,
"title_regex": sub.title_regex,
"skip_subscriber_only": sub.skip_subscriber_only,
"last_checked": sub.last_checked,
"seen_ids": list(sub.seen_ids),
"error": sub.error,
@@ -231,6 +253,22 @@ def _subscription_from_record(record: Any) -> Optional[SubscriptionInfo]:
return None
def _normalize_title_regex_value(value: Any) -> str:
if value is None:
return ""
if isinstance(value, str):
return value.strip()
return str(value).strip()
def validate_title_regex(value: Any) -> str:
"""Return stored title regex string; non-empty values must compile (re.error on failure)."""
s = _normalize_title_regex_value(value)
if s:
re.compile(s)
return s
def _coerce_bool(value: Any) -> bool:
"""Accept JSON booleans and common string forms used by API clients."""
if isinstance(value, bool):
@@ -359,6 +397,8 @@ class SubscriptionManager:
if not eid or not vurl:
continue
queue_entry = dict(ent)
if "id" not in queue_entry:
queue_entry["id"] = eid
queue_entry["_type"] = "video"
queue_entry["webpage_url"] = vurl
result = await self.dqueue.add_entry(
@@ -446,10 +486,24 @@ class SubscriptionManager:
subtitle_mode: str,
ytdl_options_presets: Optional[list[str]] = None,
ytdl_options_overrides: Optional[dict[str, Any]] = None,
title_regex: Any = None,
skip_subscriber_only: Any = None,
) -> dict:
url = self._normalize_url(url)
if not url:
return {"status": "error", "msg": "Missing URL"}
try:
title_regex_stored = validate_title_regex(title_regex)
except re.error as exc:
return {"status": "error", "msg": f"Invalid title_regex: {exc}"}
try:
skip_so = coerce_optional_bool(
skip_subscriber_only,
default=False,
field_name="skip_subscriber_only",
)
except ValueError as exc:
return {"status": "error", "msg": str(exc)}
async with self._lock:
if url in self._url_index or url in self._pending_urls:
@@ -481,6 +535,8 @@ class SubscriptionManager:
seen_entries = [ent for ent in entries if _is_media_entry(ent)]
all_ids: list[str] = []
for ent in seen_entries:
if ent.get("live_status") == "is_upcoming":
continue # Don't mark scheduled streams as seen; queue them when they go live
eid = _entry_id(ent)
if eid:
all_ids.append(eid)
@@ -505,6 +561,8 @@ class SubscriptionManager:
subtitle_mode=subtitle_mode,
ytdl_options_presets=list(ytdl_options_presets or []),
ytdl_options_overrides=dict(ytdl_options_overrides or {}),
title_regex=title_regex_stored,
skip_subscriber_only=skip_so,
last_checked=time.time(),
seen_ids=list(dict.fromkeys(all_ids)),
error=None,
@@ -551,6 +609,25 @@ class SubscriptionManager:
return {"status": "ok"}
async def update_subscription(self, sub_id: str, changes: dict) -> dict:
validated_tr: Optional[str] = None
if "title_regex" in changes:
try:
validated_tr = validate_title_regex(changes["title_regex"])
except re.error as exc:
return {"status": "error", "msg": f"Invalid title_regex: {exc}"}
skip_so_set = False
validated_skip_so = False
if "skip_subscriber_only" in changes:
try:
validated_skip_so = coerce_optional_bool(
changes["skip_subscriber_only"],
field_name="skip_subscriber_only",
)
skip_so_set = True
except ValueError as exc:
return {"status": "error", "msg": str(exc)}
async with self._lock:
sub = self._subs.get(sub_id)
if not sub:
@@ -564,6 +641,10 @@ class SubscriptionManager:
sub.check_interval_minutes = max(1, int(changes["check_interval_minutes"]))
if "name" in changes and changes["name"]:
sub.name = str(changes["name"])
if validated_tr is not None:
sub.title_regex = validated_tr
if skip_so_set:
sub.skip_subscriber_only = validated_skip_so
try:
self._save_locked()
@@ -655,18 +736,54 @@ class SubscriptionManager:
dl_submode = cur.subtitle_mode
dl_ytdl_presets = list(cur.ytdl_options_presets)
dl_ytdl_overrides = dict(cur.ytdl_options_overrides)
dl_title_regex = cur.title_regex or ""
dl_skip_subscriber_only = bool(cur.skip_subscriber_only)
new_entries: list[dict] = []
new_ids: list[str] = []
for ent in entries:
eid = _entry_id(ent)
if not eid or eid in seen:
if not eid:
continue
if eid in seen and ent.get("live_status") != "is_live":
continue
new_entries.append(ent)
new_ids.append(eid)
pattern_re: Optional[re.Pattern[str]] = None
if dl_title_regex:
try:
pattern_re = re.compile(dl_title_regex)
except re.error:
log.warning(
"Invalid stored title_regex on subscription %s, ignoring filter",
sub.name,
)
queue_entries: list[dict] = []
filtered_ids: list[str] = []
for ent in new_entries:
eid = _entry_id(ent)
if pattern_re is not None:
title = str(ent.get("title") or "")
if not pattern_re.search(title):
if eid:
filtered_ids.append(eid)
continue
queue_entries.append(ent)
subscriber_filtered_ids: list[str] = []
if dl_skip_subscriber_only:
kept_entries: list[dict] = []
for ent in queue_entries:
eid = _entry_id(ent)
if _is_subscriber_only_entry(ent):
if eid:
subscriber_filtered_ids.append(eid)
continue
kept_entries.append(ent)
queue_entries = kept_entries
queued_ids, queue_errors = await self._queue_subscription_entries(
new_entries,
queue_entries,
download_type=dl_type,
codec=dl_codec,
format=dl_format,
@@ -683,14 +800,20 @@ class SubscriptionManager:
ytdl_options_overrides=dl_ytdl_overrides,
)
log.info(
"Subscription check finished for %s: %d new, %d queued, %d failed",
"Subscription check finished for %s: %d new, %d filtered, %d subscriber_skipped, %d queued, %d failed",
sub.name,
len(new_entries),
len(filtered_ids),
len(subscriber_filtered_ids),
len(queued_ids),
len(queue_errors),
)
merged = list(dict.fromkeys(queued_ids + seen_ids_snapshot))
merged = list(
dict.fromkeys(
queued_ids + filtered_ids + subscriber_filtered_ids + seen_ids_snapshot
)
)
max_seen = int(getattr(self.config, "SUBSCRIPTION_MAX_SEEN_IDS", 50000))
if len(merged) > max_seen:
merged = merged[:max_seen]
+27
View File
@@ -279,3 +279,30 @@ async def test_add_legacy_format_migrated(mock_dqueue):
call = mock_dqueue.add.await_args
assert call is not None
assert call.args[1] == "audio"
@pytest.mark.asyncio
async def test_add_passes_clip_bounds_to_queue(mock_dqueue):
req = _json_request(
_valid_video_add_body(clip_start="2:26", clip_end="3:24"),
)
resp = await main.add(req)
assert resp.status == 200
call = mock_dqueue.add.await_args
assert call is not None
assert call.args[15] == pytest.approx(146.0)
assert call.args[16] == pytest.approx(204.0)
@pytest.mark.asyncio
async def test_subscribe_rejects_clip_options(mock_dqueue, monkeypatch):
monkeypatch.setattr(main.submgr, "add_subscription", AsyncMock())
req = _json_request(
{
**_valid_video_add_body(clip_start="10"),
"check_interval_minutes": 60,
}
)
with pytest.raises(web.HTTPBadRequest):
await main.subscribe(req)
main.submgr.add_subscription.assert_not_awaited()
+41
View File
@@ -23,6 +23,47 @@ class ConfigTests(unittest.TestCase):
c = Config()
self.assertEqual(c.URL_PREFIX, "foo/")
def test_public_host_url_gets_trailing_slash(self):
with patch.dict(
os.environ,
_base_env(PUBLIC_HOST_URL="https://ytdl.example.com"),
clear=False,
):
c = Config()
self.assertEqual(c.PUBLIC_HOST_URL, "https://ytdl.example.com/")
def test_public_host_audio_url_gets_trailing_slash(self):
with patch.dict(
os.environ,
_base_env(PUBLIC_HOST_AUDIO_URL="https://audio.example.com"),
clear=False,
):
c = Config()
self.assertEqual(c.PUBLIC_HOST_AUDIO_URL, "https://audio.example.com/")
def test_public_host_url_empty_stays_empty(self):
with patch.dict(
os.environ,
_base_env(PUBLIC_HOST_URL="", PUBLIC_HOST_AUDIO_URL=""),
clear=False,
):
c = Config()
self.assertEqual(c.PUBLIC_HOST_URL, "")
self.assertEqual(c.PUBLIC_HOST_AUDIO_URL, "")
def test_public_host_url_already_slashed_unchanged(self):
with patch.dict(
os.environ,
_base_env(
PUBLIC_HOST_URL="https://ytdl.example.com/",
PUBLIC_HOST_AUDIO_URL="https://audio.example.com/",
),
clear=False,
):
c = Config()
self.assertEqual(c.PUBLIC_HOST_URL, "https://ytdl.example.com/")
self.assertEqual(c.PUBLIC_HOST_AUDIO_URL, "https://audio.example.com/")
def test_ytdl_options_json_loaded(self):
opts = {"quiet": True, "no_warnings": True}
with patch.dict(
+171 -4
View File
@@ -56,7 +56,7 @@ def test_get_returns_tuple_of_lists(dq_env):
async def test_add_single_video_goes_to_pending_when_auto_start_false(dq_env):
notifier = AsyncMock()
def fake_extract(self, url):
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
return {
"_type": "video",
"id": "vid1",
@@ -86,7 +86,7 @@ async def test_add_single_video_goes_to_pending_when_auto_start_false(dq_env):
async def test_cancel_removes_from_pending(dq_env):
notifier = AsyncMock()
def fake_extract(self, url):
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
return {
"_type": "video",
"id": "vid1",
@@ -114,11 +114,54 @@ async def test_cancel_removes_from_pending(dq_env):
notifier.canceled.assert_awaited()
@pytest.mark.asyncio
async def test_cancel_before_start_marks_download_canceled(dq_env):
"""Regression test for the race condition where cancel() arrives after the
download has been placed in the queue and ``__start_download`` has been
scheduled via ``asyncio.create_task`` but has not yet executed. Without the
fix, the pending task would run ``download.start()`` despite the user
cancelling, because its ``download.canceled`` guard was never flipped."""
notifier = AsyncMock()
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
return {
"_type": "video",
"id": "vid1",
"title": "Test Video",
"url": url,
"webpage_url": url,
}
dq = DownloadQueue(dq_env, notifier)
url = "https://example.com/race"
start_mock = AsyncMock()
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", fake_extract), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", start_mock):
await dq.add(
url,
"video",
"auto",
"any",
"best",
"",
"",
0,
auto_start=True,
)
assert dq.queue.exists(url)
download = dq.queue.get(url)
assert download.canceled is False
await dq.cancel([url])
assert not dq.queue.exists(url)
assert download.canceled is True
notifier.canceled.assert_awaited_with(url)
@pytest.mark.asyncio
async def test_start_pending_moves_to_queue(dq_env):
notifier = AsyncMock()
def fake_extract(self, url):
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
return {
"_type": "video",
"id": "vid1",
@@ -187,7 +230,7 @@ async def test_add_merges_global_preset_and_override_options(dq_env):
"Preset B": {"writesubtitles": False, "ratelimit": 1000},
}
def fake_extract(self, url):
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
return {
"_type": "video",
"id": "vid2",
@@ -219,3 +262,127 @@ async def test_add_merges_global_preset_and_override_options(dq_env):
assert queued.ytdl_opts["ratelimit"] == 1000
assert queued.ytdl_opts["proxy"] == "http://override"
assert queued.ytdl_opts["embed_thumbnail"] is True
@pytest.mark.asyncio
async def test_extract_info_preset_null_download_archive_overrides_global(dq_env):
"""Preset download_archive:null must apply during extract_info (global archive otherwise wins first)."""
dq_env.YTDL_OPTIONS = {"download_archive": "/tmp/archive.txt"}
dq_env.YTDL_OPTIONS_PRESETS = {"NoArchive": {"download_archive": None}}
captured_params: list = []
class FakeYoutubeDL:
def __init__(self, params=None):
captured_params.append(params)
def extract_info(self, url, download=False):
return {
"_type": "video",
"id": "vid-archive",
"title": "Archive Test",
"url": url,
"webpage_url": url,
}
notifier = AsyncMock()
dq = DownloadQueue(dq_env, notifier)
with patch("ytdl.yt_dlp.YoutubeDL", FakeYoutubeDL):
result = await dq.add(
"https://example.com/archive-test",
"video",
"auto",
"any",
"best",
"",
"",
0,
auto_start=False,
ytdl_options_presets=["NoArchive"],
)
assert result["status"] == "ok"
assert len(captured_params) == 1
extract_params = captured_params[0]
assert extract_params.get("download_archive") is None
assert extract_params["extract_flat"] is True
assert extract_params["noplaylist"] is True
@pytest.mark.asyncio
async def test_extract_info_metube_extract_keys_win_over_preset(dq_env):
"""MeTube's flat-extract settings must not be overridden by presets."""
dq_env.YTDL_OPTIONS = {}
dq_env.YTDL_OPTIONS_PRESETS = {
"TryOverride": {"extract_flat": False, "noplaylist": False},
}
captured_params: list = []
class FakeYoutubeDL:
def __init__(self, params=None):
captured_params.append(params)
def extract_info(self, url, download=False):
return {
"_type": "video",
"id": "vid-flat",
"title": "Flat Test",
"url": url,
"webpage_url": url,
}
notifier = AsyncMock()
dq = DownloadQueue(dq_env, notifier)
with patch("ytdl.yt_dlp.YoutubeDL", FakeYoutubeDL):
result = await dq.add(
"https://example.com/flat-test",
"video",
"auto",
"any",
"best",
"",
"",
0,
auto_start=False,
ytdl_options_presets=["TryOverride"],
)
assert result["status"] == "ok"
assert captured_params[0]["extract_flat"] is True
assert captured_params[0]["noplaylist"] is True
@pytest.mark.asyncio
async def test_add_sets_clip_bounds_on_download_info(dq_env):
notifier = AsyncMock()
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
return {
"_type": "video",
"id": "vid1",
"title": "Test Video",
"url": url,
"webpage_url": url,
}
dq = DownloadQueue(dq_env, notifier)
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", fake_extract):
result = await dq.add(
"https://example.com/clip",
"video",
"auto",
"any",
"best",
"",
"",
0,
auto_start=False,
clip_start=10.0,
clip_end=99.5,
)
assert result["status"] == "ok"
download = dq.pending.get("https://example.com/clip")
assert download.info.clip_start == 10.0
assert download.info.clip_end == 99.5
+74
View File
@@ -205,6 +205,80 @@ class ParseDownloadOptionsTests(unittest.TestCase):
finally:
main.config.YTDL_OPTIONS_PRESETS = previous
def test_clip_start_end_seconds_and_clock(self):
parsed = main.parse_download_options({
"url": "https://example.com/watch?v=1",
"download_type": "video",
"codec": "auto",
"format": "any",
"quality": "best",
"clip_start": "2:26",
"clip_end": "3:24",
})
self.assertEqual(parsed["clip_start"], 146.0)
self.assertEqual(parsed["clip_end"], 204.0)
def test_clip_url_t_param_strips_query_and_sets_start(self):
parsed = main.parse_download_options({
"url": "https://example.com/watch?v=1&t=855s",
"download_type": "video",
"codec": "auto",
"format": "any",
"quality": "best",
})
self.assertEqual(parsed["url"], "https://example.com/watch?v=1")
self.assertEqual(parsed["clip_start"], 855.0)
self.assertIsNone(parsed["clip_end"])
def test_clip_explicit_start_wins_over_url_t(self):
parsed = main.parse_download_options({
"url": "https://example.com/watch?v=1&t=100",
"download_type": "video",
"codec": "auto",
"format": "any",
"quality": "best",
"clip_start": "50",
})
self.assertEqual(parsed["url"], "https://example.com/watch?v=1")
self.assertEqual(parsed["clip_start"], 50.0)
self.assertIsNone(parsed["clip_end"])
def test_clip_end_only_sets_start_zero_and_strips_url_t(self):
parsed = main.parse_download_options({
"url": "https://example.com/watch?v=1&t=999",
"download_type": "video",
"codec": "auto",
"format": "any",
"quality": "best",
"clip_end": "60",
})
self.assertEqual(parsed["url"], "https://example.com/watch?v=1")
self.assertEqual(parsed["clip_start"], 0.0)
self.assertEqual(parsed["clip_end"], 60.0)
def test_clip_rejects_end_before_start(self):
with self.assertRaises(main.web.HTTPBadRequest):
main.parse_download_options({
"url": "https://example.com/watch?v=1",
"download_type": "video",
"codec": "auto",
"format": "any",
"quality": "best",
"clip_start": "100",
"clip_end": "50",
})
def test_clip_rejected_for_captions(self):
with self.assertRaises(main.web.HTTPBadRequest):
main.parse_download_options({
"url": "https://example.com/watch?v=1",
"download_type": "captions",
"codec": "auto",
"format": "srt",
"quality": "best",
"clip_start": "1",
})
if __name__ == "__main__":
unittest.main()
+508 -1
View File
@@ -28,7 +28,12 @@ sys.modules.setdefault("yt_dlp", fake_yt_dlp)
sys.modules.setdefault("yt_dlp.networking", fake_networking)
sys.modules.setdefault("yt_dlp.networking.impersonate", fake_impersonate)
from subscriptions import SubscriptionManager, extract_flat_playlist
from subscriptions import (
SubscriptionManager,
_is_subscriber_only_entry,
coerce_optional_bool,
extract_flat_playlist,
)
class _Config:
@@ -75,6 +80,20 @@ def _create_legacy_shelf(path: str, record) -> None:
shelf["sub-1"] = record
class SubscriberOnlyHelperTests(unittest.TestCase):
def test_is_subscriber_only_detects_availability(self):
self.assertTrue(_is_subscriber_only_entry({"availability": "subscriber_only"}))
self.assertFalse(_is_subscriber_only_entry({"availability": None}))
self.assertFalse(_is_subscriber_only_entry({}))
def test_coerce_optional_bool_defaults_and_fields(self):
self.assertFalse(coerce_optional_bool(None, default=False))
self.assertTrue(coerce_optional_bool(True))
self.assertFalse(coerce_optional_bool(False))
with self.assertRaises(ValueError):
coerce_optional_bool("maybe", field_name="skip_subscriber_only")
class SubscriptionPersistenceTests(unittest.IsolatedAsyncioTestCase):
def test_load_imports_legacy_subscription_shelf(self):
with tempfile.TemporaryDirectory() as tmp:
@@ -386,6 +405,108 @@ class SubscriptionPersistenceTests(unittest.IsolatedAsyncioTestCase):
self.assertEqual(sub.seen_ids[:2], ["v2", "v1"])
self.assertEqual([entry["webpage_url"] for entry, _, _ in queue.entries], ["https://example.com/v2"])
async def test_check_now_queues_subscriber_only_when_skip_disabled(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
side_effect=[
(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
(
{"_type": "channel", "title": "Channel"},
[
{
"id": "v2",
"title": "Members",
"webpage_url": "https://example.com/v2",
"availability": "subscriber_only",
},
{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"},
],
),
],
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
skip_subscriber_only=False,
)
self.assertFalse(mgr.list_all()[0].skip_subscriber_only)
await mgr.check_now([result["subscription"]["id"]])
sub = mgr.list_all()[0]
self.assertIsNone(sub.error)
self.assertEqual(sub.seen_ids[:2], ["v2", "v1"])
self.assertEqual([entry["webpage_url"] for entry, _, _ in queue.entries], ["https://example.com/v2"])
async def test_check_now_skips_subscriber_only_when_skip_enabled(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
side_effect=[
(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
(
{"_type": "channel", "title": "Channel"},
[
{
"id": "v2",
"title": "Members",
"webpage_url": "https://example.com/v2",
"availability": "subscriber_only",
},
{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"},
],
),
],
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
skip_subscriber_only=True,
)
self.assertTrue(mgr.list_all()[0].skip_subscriber_only)
await mgr.check_now([result["subscription"]["id"]])
sub = mgr.list_all()[0]
self.assertIsNone(sub.error)
self.assertEqual(sub.seen_ids[:2], ["v2", "v1"])
self.assertEqual(queue.entries, [])
async def test_update_subscription_parses_string_false_enabled(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
@@ -453,6 +574,392 @@ class SubscriptionPersistenceTests(unittest.IsolatedAsyncioTestCase):
with self.assertRaises(ValueError):
await mgr.update_subscription(sub_id, {"enabled": "maybe"})
async def test_add_subscription_rejects_invalid_title_regex(self):
with tempfile.TemporaryDirectory() as tmp:
mgr = SubscriptionManager(_Config(tmp), _Queue(), _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
title_regex="[",
)
self.assertEqual(result["status"], "error")
self.assertIn("title_regex", result["msg"].lower())
self.assertEqual(mgr.list_all(), [])
async def test_add_subscription_stores_and_exposes_title_regex(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
title_regex="EPISODE",
)
self.assertEqual(result["status"], "ok")
self.assertEqual(result["subscription"]["title_regex"], "EPISODE")
self.assertEqual(mgr.list_all()[0].title_regex, "EPISODE")
async def test_check_now_title_regex_queues_only_matches_and_marks_unmatched_seen(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
side_effect=[
(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "Old", "webpage_url": "https://example.com/v1"}],
),
(
{"_type": "channel", "title": "Channel"},
[
{
"id": "v2",
"title": "Minecraft | EPISODE 1",
"webpage_url": "https://example.com/v2",
},
{
"id": "v3",
"title": "Unrelated IRL",
"webpage_url": "https://example.com/v3",
},
{
"id": "v1",
"title": "Old",
"webpage_url": "https://example.com/v1",
},
],
),
],
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
title_regex="EPISODE",
)
await mgr.check_now([result["subscription"]["id"]])
self.assertEqual([e["webpage_url"] for e, _, _ in queue.entries], ["https://example.com/v2"])
sub = mgr.list_all()[0]
self.assertEqual(sub.seen_ids[:3], ["v2", "v3", "v1"])
async def test_check_now_title_regex_queue_failure_keeps_matched_id_unseen(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
side_effect=[
(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "Old", "webpage_url": "https://example.com/v1"}],
),
(
{"_type": "channel", "title": "Channel"},
[
{
"id": "v2",
"title": "Show | EPISODE 1",
"webpage_url": "https://example.com/v2",
},
{
"id": "v3",
"title": "Other",
"webpage_url": "https://example.com/v3",
},
],
),
],
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
title_regex="EPISODE",
)
queue.fail = True
await mgr.check_now([result["subscription"]["id"]])
sub = mgr.list_all()[0]
self.assertEqual(sub.error, "queue failed")
self.assertEqual(set(sub.seen_ids), {"v1", "v3"})
self.assertNotIn("v2", sub.seen_ids)
async def test_update_subscription_rejects_invalid_title_regex(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
sub_id = result["subscription"]["id"]
upd = await mgr.update_subscription(sub_id, {"title_regex": "("})
self.assertEqual(upd["status"], "error")
self.assertEqual(mgr.list_all()[0].title_regex, "")
async def test_update_subscription_persists_valid_title_regex(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
sub_id = result["subscription"]["id"]
upd = await mgr.update_subscription(sub_id, {"title_regex": "foo|bar"})
self.assertEqual(upd["status"], "ok")
self.assertEqual(upd["subscription"]["title_regex"], "foo|bar")
self.assertEqual(mgr.list_all()[0].title_regex, "foo|bar")
async def test_update_subscription_skip_subscriber_only(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
sub_id = result["subscription"]["id"]
self.assertFalse(mgr.list_all()[0].skip_subscriber_only)
upd = await mgr.update_subscription(sub_id, {"skip_subscriber_only": True})
self.assertEqual(upd["status"], "ok")
self.assertTrue(upd["subscription"]["skip_subscriber_only"])
self.assertTrue(mgr.list_all()[0].skip_subscriber_only)
async def test_update_subscription_rejects_invalid_skip_subscriber_only(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
mgr = SubscriptionManager(_Config(tmp), queue, _Notifier())
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Channel"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
):
result = await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
sub_id = result["subscription"]["id"]
upd = await mgr.update_subscription(sub_id, {"skip_subscriber_only": "maybe"})
self.assertEqual(upd["status"], "error")
self.assertFalse(mgr.list_all()[0].skip_subscriber_only)
def test_persistence_includes_title_regex(self):
with tempfile.TemporaryDirectory() as tmp:
json_path = os.path.join(tmp, "subscriptions.json")
with open(json_path, "w", encoding="utf-8") as f:
json.dump(
{
"schema_version": 2,
"kind": "subscriptions",
"items": [
{
"id": "sub-1",
"name": "Channel",
"url": "https://example.com/channel",
"enabled": True,
"check_interval_minutes": 60,
"download_type": "video",
"codec": "auto",
"format": "any",
"quality": "best",
"folder": "",
"custom_name_prefix": "",
"auto_start": True,
"playlist_item_limit": 0,
"split_by_chapters": False,
"chapter_template": "",
"subtitle_language": "en",
"subtitle_mode": "prefer_manual",
"ytdl_options_presets": [],
"ytdl_options_overrides": {},
"title_regex": "EPISODE",
"last_checked": None,
"seen_ids": [],
"error": None,
}
],
},
f,
)
mgr = SubscriptionManager(_Config(tmp), _Queue(), _Notifier())
self.assertEqual(mgr.list_all()[0].title_regex, "EPISODE")
self.assertFalse(mgr.list_all()[0].skip_subscriber_only)
def test_persistence_includes_skip_subscriber_only(self):
with tempfile.TemporaryDirectory() as tmp:
json_path = os.path.join(tmp, "subscriptions.json")
with open(json_path, "w", encoding="utf-8") as f:
json.dump(
{
"schema_version": 2,
"kind": "subscriptions",
"items": [
{
"id": "sub-1",
"name": "Channel",
"url": "https://example.com/channel",
"enabled": True,
"check_interval_minutes": 60,
"download_type": "video",
"codec": "auto",
"format": "any",
"quality": "best",
"folder": "",
"custom_name_prefix": "",
"auto_start": True,
"playlist_item_limit": 0,
"split_by_chapters": False,
"chapter_template": "",
"subtitle_language": "en",
"subtitle_mode": "prefer_manual",
"ytdl_options_presets": [],
"ytdl_options_overrides": {},
"title_regex": "",
"skip_subscriber_only": True,
"last_checked": None,
"seen_ids": [],
"error": None,
}
],
},
f,
)
mgr = SubscriptionManager(_Config(tmp), _Queue(), _Notifier())
self.assertTrue(mgr.list_all()[0].skip_subscriber_only)
class ExtractFlatPlaylistTests(unittest.TestCase):
def test_descends_one_level_when_root_entries_are_nested_collections(self):
responses = iter(
+70 -13
View File
@@ -9,6 +9,7 @@ from collections import OrderedDict
import time
import asyncio
import multiprocessing
from functools import partial
import logging
import re
import types
@@ -19,6 +20,7 @@ from yt_dlp.utils import STR_FORMAT_RE_TMPL, STR_FORMAT_TYPES
from dl_formats import get_format, get_opts, AUDIO_FORMATS
from datetime import datetime
from state_store import AtomicJsonStore, from_json_compatible, read_legacy_shelf, to_json_compatible
from subscriptions import _entry_id
log = logging.getLogger('ytdl')
@@ -190,6 +192,8 @@ class DownloadInfo:
subtitle_mode="prefer_manual",
ytdl_options_presets=None,
ytdl_options_overrides=None,
clip_start=None,
clip_end=None,
):
self.id = id if len(custom_name_prefix) == 0 else f'{custom_name_prefix}.{id}'
self.title = title if len(custom_name_prefix) == 0 else f'{custom_name_prefix}.{title}'
@@ -214,6 +218,8 @@ class DownloadInfo:
self.subtitle_mode = subtitle_mode
self.ytdl_options_presets = list(ytdl_options_presets or [])
self.ytdl_options_overrides = dict(ytdl_options_overrides or {})
self.clip_start = clip_start
self.clip_end = clip_end
self.subtitle_files = []
def __setstate__(self, state):
@@ -282,6 +288,10 @@ class DownloadInfo:
self.subtitle_files = []
if not hasattr(self, "chapter_files"):
self.chapter_files = []
if not hasattr(self, "clip_start"):
self.clip_start = None
if not hasattr(self, "clip_end"):
self.clip_end = None
_PERSISTED_DOWNLOAD_FIELDS = (
@@ -301,6 +311,8 @@ _PERSISTED_DOWNLOAD_FIELDS = (
"subtitle_mode",
"ytdl_options_presets",
"ytdl_options_overrides",
"clip_start",
"clip_end",
"status",
"timestamp",
"error",
@@ -471,6 +483,16 @@ class Download:
'force_keyframes': False
})
clip_start = getattr(self.info, 'clip_start', None)
clip_end = getattr(self.info, 'clip_end', None)
if clip_start is not None or clip_end is not None:
start = float(clip_start) if clip_start is not None else 0.0
end = float(clip_end) if clip_end is not None else float('inf')
ytdl_params['download_ranges'] = yt_dlp.utils.download_range_func(
None,
[(start, end)],
)
ret = yt_dlp.YoutubeDL(params=ytdl_params).download([self.info.url])
self.status_queue.put({'status': 'finished' if ret == 0 else 'error'})
log.info(f"Finished download for: {self.info.title}")
@@ -795,9 +817,19 @@ class DownloadQueue:
log.debug(f'Auto-clearing completed download: {url}')
await self.clear([url])
def __extract_info(self, url):
def _build_ytdl_options(self, ytdl_options_presets=None, ytdl_options_overrides=None):
"""Merge global options, presets (in order), and per-download overrides."""
opts = dict(self.config.YTDL_OPTIONS)
for preset_name in ytdl_options_presets or []:
opts.update(self.config.YTDL_OPTIONS_PRESETS.get(preset_name, {}))
opts.update(ytdl_options_overrides or {})
return opts
def __extract_info(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
debug_logging = logging.getLogger().isEnabledFor(logging.DEBUG)
return yt_dlp.YoutubeDL(params={
user_opts = self._build_ytdl_options(ytdl_options_presets, ytdl_options_overrides)
params = {
**user_opts,
'quiet': not debug_logging,
'verbose': debug_logging,
'no_color': True,
@@ -805,9 +837,11 @@ class DownloadQueue:
'ignore_no_formats_error': True,
'noplaylist': True,
'paths': {"home": self.config.DOWNLOAD_DIR, "temp": self.config.TEMP_DIR},
**self.config.YTDL_OPTIONS,
**({'impersonate': yt_dlp.networking.impersonate.ImpersonateTarget.from_str(self.config.YTDL_OPTIONS['impersonate'])} if 'impersonate' in self.config.YTDL_OPTIONS else {}),
}).extract_info(url, download=False)
}
imp = user_opts.get('impersonate')
if imp is not None:
params['impersonate'] = yt_dlp.networking.impersonate.ImpersonateTarget.from_str(imp)
return yt_dlp.YoutubeDL(params=params).extract_info(url, download=False)
def __calc_download_path(self, download_type, folder):
base_directory = self.config.AUDIO_DOWNLOAD_DIR if download_type == 'audio' else self.config.DOWNLOAD_DIR
@@ -843,10 +877,10 @@ class DownloadQueue:
output = self.config.OUTPUT_TEMPLATE_CHANNEL
sanitized = {k: _sanitize_path_component(v) for k, v in entry.items()}
output = _resolve_outtmpl_fields(output, sanitized, ('channel',))
ytdl_options = dict(self.config.YTDL_OPTIONS)
for preset_name in getattr(dl, 'ytdl_options_presets', None) or []:
ytdl_options.update(self.config.YTDL_OPTIONS_PRESETS.get(preset_name, {}))
ytdl_options.update(getattr(dl, 'ytdl_options_overrides', {}) or {})
ytdl_options = self._build_ytdl_options(
getattr(dl, 'ytdl_options_presets', None),
getattr(dl, 'ytdl_options_overrides', {}) or {},
)
playlist_item_limit = getattr(dl, 'playlist_item_limit', 0)
if playlist_item_limit > 0:
log.info(f'playlist limit is set. Processing only first {playlist_item_limit} entries')
@@ -876,6 +910,8 @@ class DownloadQueue:
subtitle_mode,
ytdl_options_presets,
ytdl_options_overrides,
clip_start,
clip_end,
already,
_add_gen=None,
):
@@ -910,6 +946,8 @@ class DownloadQueue:
subtitle_mode,
ytdl_options_presets,
ytdl_options_overrides,
clip_start,
clip_end,
already,
_add_gen,
)
@@ -930,6 +968,8 @@ class DownloadQueue:
if _add_gen is not None and self._add_generation != _add_gen:
log.info(f'Playlist add canceled after processing {len(already)} entries')
return {'status': 'ok', 'msg': f'Canceled - added {len(already)} items before cancel'}
if "id" not in etr:
etr["id"] = _entry_id(etr)
etr["_type"] = "video"
etr[etype] = entry.get("id") or entry.get("channel_id") or entry.get("channel")
etr[f"{etype}_index"] = '{{0:0{0:d}d}}'.format(index_digits).format(index)
@@ -959,6 +999,8 @@ class DownloadQueue:
subtitle_mode,
ytdl_options_presets,
ytdl_options_overrides,
clip_start,
clip_end,
already,
_add_gen,
)
@@ -992,6 +1034,8 @@ class DownloadQueue:
subtitle_mode=subtitle_mode,
ytdl_options_presets=ytdl_options_presets,
ytdl_options_overrides=ytdl_options_overrides,
clip_start=clip_start,
clip_end=clip_end,
)
await self.__add_download(dl, auto_start)
return {'status': 'ok'}
@@ -1014,6 +1058,8 @@ class DownloadQueue:
subtitle_mode="prefer_manual",
ytdl_options_presets=None,
ytdl_options_overrides=None,
clip_start=None,
clip_end=None,
already=None,
_add_gen=None,
):
@@ -1022,7 +1068,7 @@ class DownloadQueue:
log.info(
f'adding {url}: {download_type=} {codec=} {format=} {quality=} {already=} {folder=} {custom_name_prefix=} '
f'{playlist_item_limit=} {auto_start=} {split_by_chapters=} {chapter_template=} '
f'{subtitle_language=} {subtitle_mode=} {ytdl_options_presets=}'
f'{subtitle_language=} {subtitle_mode=} {ytdl_options_presets=} {clip_start=} {clip_end=}'
)
if already is None:
_add_gen = self._add_generation
@@ -1034,7 +1080,10 @@ class DownloadQueue:
else:
already.add(url)
try:
entry = await asyncio.get_running_loop().run_in_executor(None, self.__extract_info, url)
entry = await asyncio.get_running_loop().run_in_executor(
None,
partial(self.__extract_info, url, ytdl_options_presets, ytdl_options_overrides),
)
except yt_dlp.utils.YoutubeDLError as exc:
return {'status': 'error', 'msg': str(exc)}
return await self.__add_entry(
@@ -1053,6 +1102,8 @@ class DownloadQueue:
subtitle_mode,
ytdl_options_presets,
ytdl_options_overrides,
clip_start,
clip_end,
already,
_add_gen,
)
@@ -1074,6 +1125,8 @@ class DownloadQueue:
subtitle_mode="prefer_manual",
ytdl_options_presets=None,
ytdl_options_overrides=None,
clip_start=None,
clip_end=None,
):
if ytdl_options_presets is None:
ytdl_options_presets = []
@@ -1095,6 +1148,8 @@ class DownloadQueue:
subtitle_mode,
ytdl_options_presets,
ytdl_options_overrides,
clip_start,
clip_end,
already,
None,
)
@@ -1121,9 +1176,11 @@ class DownloadQueue:
if not self.queue.exists(id):
log.warning(f'requested cancel for non-existent download {id}')
continue
if self.queue.get(id).started():
self.queue.get(id).cancel()
dl = self.queue.get(id)
if dl.started():
dl.cancel()
else:
dl.canceled = True
self.queue.delete(id)
await self.notifier.canceled(id)
return {'status': 'ok'}
+14 -14
View File
@@ -23,21 +23,21 @@
},
"private": true,
"dependencies": {
"@angular/animations": "^21.2.7",
"@angular/common": "^21.2.7",
"@angular/compiler": "^21.2.7",
"@angular/core": "^21.2.7",
"@angular/forms": "^21.2.7",
"@angular/platform-browser": "^21.2.7",
"@angular/platform-browser-dynamic": "^21.2.7",
"@angular/service-worker": "^21.2.7",
"@angular/animations": "^21.2.9",
"@angular/common": "^21.2.9",
"@angular/compiler": "^21.2.9",
"@angular/core": "^21.2.9",
"@angular/forms": "^21.2.9",
"@angular/platform-browser": "^21.2.9",
"@angular/platform-browser-dynamic": "^21.2.9",
"@angular/service-worker": "^21.2.9",
"@fortawesome/angular-fontawesome": "~4.0.0",
"@fortawesome/fontawesome-svg-core": "^7.2.0",
"@fortawesome/free-brands-svg-icons": "^7.2.0",
"@fortawesome/free-regular-svg-icons": "^7.2.0",
"@fortawesome/free-solid-svg-icons": "^7.2.0",
"@ng-bootstrap/ng-bootstrap": "^20.0.0",
"@ng-select/ng-select": "^21.7.0",
"@ng-select/ng-select": "^21.8.0",
"@popperjs/core": "^2.11.8",
"bootstrap": "^5.3.8",
"ngx-cookie-service": "^21.3.1",
@@ -48,16 +48,16 @@
},
"devDependencies": {
"@angular-eslint/builder": "21.1.0",
"@angular/build": "^21.2.6",
"@angular/cli": "^21.2.6",
"@angular/compiler-cli": "^21.2.7",
"@angular/localize": "^21.2.7",
"@angular/build": "^21.2.7",
"@angular/cli": "^21.2.7",
"@angular/compiler-cli": "^21.2.9",
"@angular/localize": "^21.2.9",
"@eslint/js": "^9.39.4",
"angular-eslint": "21.1.0",
"eslint": "^9.39.4",
"jsdom": "^27.4.0",
"typescript": "~5.9.3",
"typescript-eslint": "8.47.0",
"vitest": "^4.1.2"
"vitest": "^4.1.5"
}
}
+632 -600
View File
File diff suppressed because it is too large Load Diff
+6
View File
@@ -0,0 +1,6 @@
allowBuilds:
'@parcel/watcher': true
core-js: true
esbuild: true
lmdb: true
msgpackr-extract: true
+90
View File
@@ -428,6 +428,34 @@
}
</div>
</div>
@if (downloadType === 'video' || downloadType === 'audio') {
<div class="col-md-6">
<div class="input-group">
<span class="input-group-text">Clip start</span>
<input type="text"
class="form-control"
name="clipStart"
[(ngModel)]="clipStart"
(change)="clipStartChanged()"
placeholder="e.g. 2:26"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="Optional start time (seconds, M:SS, or H:MM:SS). Blank = from start or YouTube &t= in URL.">
</div>
</div>
<div class="col-md-6">
<div class="input-group">
<span class="input-group-text">Clip end</span>
<input type="text"
class="form-control"
name="clipEnd"
[(ngModel)]="clipEnd"
(change)="clipEndChanged()"
placeholder="e.g. 3:24"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="Optional end time. Blank = until end of media.">
</div>
</div>
}
</div>
<!-- Behavior -->
@@ -475,6 +503,27 @@
ngbTooltip="How often to poll subscriptions for new videos">
</div>
</div>
<div class="col-md-6">
<div class="input-group">
<span class="input-group-text">Subscription Title Filter</span>
<input type="text"
class="form-control"
name="titleRegex"
[(ngModel)]="titleRegex"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
placeholder="Optional regex"
ngbTooltip="In subscriptions, only titles matching this Python-style regex are queued. Empty = all. Case-sensitive; use (?i) in the pattern for case-insensitive.">
</div>
</div>
<div class="col-12">
<div class="form-check form-switch">
<input class="form-check-input" type="checkbox" role="switch" id="checkbox-skip-subscriber-only"
name="skipSubscriberOnly" [(ngModel)]="skipSubscriberOnly"
[disabled]="addInProgress || subscribeInProgress || downloads.loading"
ngbTooltip="When enabled, subscription checks skip videos marked members-only by yt-dlp (channel Join). Ignored for one-off downloads." />
<label class="form-check-label" for="checkbox-skip-subscriber-only">Skip members-only subscription videos</label>
</div>
</div>
</div>
<!-- yt-dlp -->
@@ -608,6 +657,19 @@
@if (batchImportStatus) {
<small>{{ batchImportStatus }}</small>
}
@if (batchImportTotal > 0) {
<div class="progress mt-2" style="height: 20px;">
<div class="progress-bar"
[class.bg-danger]="batchImportFailures > 0"
role="progressbar"
[style.width.%]="(batchImportCount / batchImportTotal) * 100"
[attr.aria-valuenow]="batchImportCount"
[attr.aria-valuemin]="0"
[attr.aria-valuemax]="batchImportTotal">
{{ batchImportCount }} / {{ batchImportTotal }}
</div>
</div>
}
</div>
</div>
<div class="modal-footer">
@@ -874,6 +936,8 @@
</th>
<th scope="col">Name</th>
<th scope="col">URL</th>
<th scope="col" class="text-nowrap"
ngbTooltip="Subscriptions only — which new video titles to queue when this feed is checked. Does not affect manual downloads.">Sub. title filter</th>
<th scope="col" class="text-nowrap">Interval (min)</th>
<th scope="col" class="text-nowrap">Last checked</th>
<th scope="col">Status</th>
@@ -892,6 +956,32 @@
</td>
<td>{{ entry[1].name }}</td>
<td class="text-break"><a [href]="entry[1].url" target="_blank" rel="noopener">{{ entry[1].url }}</a></td>
<td>
@if (editingTitleRegexId === entry[0]) {
<div class="d-flex flex-wrap gap-1 align-items-center">
<input type="text"
class="form-control form-control-sm flex-grow-1"
[name]="'subTitleRegex' + entry[0]"
[(ngModel)]="titleRegexEditDraft"
[disabled]="downloads.loading" />
<button type="button" class="btn btn-sm btn-outline-secondary"
(click)="saveTitleRegex(entry[0])"
[disabled]="downloads.loading">Save</button>
<button type="button" class="btn btn-sm btn-outline-secondary"
(click)="cancelEditTitleRegex()"
[disabled]="downloads.loading">Cancel</button>
</div>
} @else {
<div class="d-flex flex-wrap gap-1 align-items-center">
<span class="text-muted small text-break"
[class.text-secondary]="!entry[1].title_regex">{{ entry[1].title_regex || '—' }}</span>
<button type="button" class="btn btn-link btn-sm p-0"
(click)="beginEditTitleRegex(entry[0], entry[1].title_regex)"
[disabled]="downloads.loading"
ngbTooltip="Edit subscription title filter (subscriptions only; not for one-off downloads)">Edit</button>
</div>
}
</td>
<td>{{ entry[1].check_interval_minutes }}</td>
<td class="text-nowrap">
@if (entry[1].last_checked !== null) {
+69 -1
View File
@@ -63,8 +63,10 @@ class DownloadsServiceStub {
class SubscriptionsServiceStub {
subscriptions = new Map();
subscriptionsChanged = new Subject<void>();
subscribeCalls: unknown[] = [];
subscribe() {
subscribe(payload: unknown) {
this.subscribeCalls.push(payload);
return of({ status: 'ok' as const });
}
@@ -72,6 +74,10 @@ class SubscriptionsServiceStub {
return of({});
}
update() {
return of({ status: 'ok' as const });
}
refreshList() {
return of([]);
}
@@ -175,4 +181,66 @@ describe('App', () => {
expect(payload.ytdlOptionsOverrides).toBe('');
});
it('includes titleRegex in subscribe payload', () => {
const fixture = TestBed.createComponent(App);
const app = fixture.componentInstance;
const subs = TestBed.inject(SubscriptionsService) as unknown as SubscriptionsServiceStub;
app.addUrl = 'https://example.com/channel';
app.titleRegex = 'EPISODE';
app.addSubscription();
expect(subs.subscribeCalls.length).toBe(1);
const payload = subs.subscribeCalls[0] as { titleRegex: string; skipSubscriberOnly: boolean };
expect(payload.titleRegex).toBe('EPISODE');
expect(payload.skipSubscriberOnly).toBe(false);
});
it('includes skipSubscriberOnly true when checked', () => {
const fixture = TestBed.createComponent(App);
const app = fixture.componentInstance;
const subs = TestBed.inject(SubscriptionsService) as unknown as SubscriptionsServiceStub;
app.addUrl = 'https://example.com/channel';
app.skipSubscriberOnly = true;
app.addSubscription();
expect(subs.subscribeCalls.length).toBe(1);
const payload = subs.subscribeCalls[0] as { skipSubscriberOnly: boolean };
expect(payload.skipSubscriberOnly).toBe(true);
});
it('omits clip fields from subscribe payload', () => {
const fixture = TestBed.createComponent(App);
const app = fixture.componentInstance;
const subs = TestBed.inject(SubscriptionsService) as unknown as SubscriptionsServiceStub;
app.addUrl = 'https://example.com/channel';
app.clipStart = '1:00';
app.clipEnd = '2:00';
app.addSubscription();
expect(subs.subscribeCalls.length).toBe(1);
const payload = subs.subscribeCalls[0] as Record<string, unknown>;
expect('clipStart' in payload).toBe(false);
expect('clipEnd' in payload).toBe(false);
});
it('buildAddPayload includes clip times', () => {
const fixture = TestBed.createComponent(App);
const app = fixture.componentInstance;
app.clipStart = '0:10';
app.clipEnd = '1:20';
const payload = app['buildAddPayload']();
expect(payload.clipStart).toBe('0:10');
expect(payload.clipEnd).toBe('1:20');
});
it('blocks subscribe with invalid title regex', () => {
const alertSpy = vi.spyOn(window, 'alert').mockImplementation(() => undefined);
const fixture = TestBed.createComponent(App);
const app = fixture.componentInstance;
const subs = TestBed.inject(SubscriptionsService) as unknown as SubscriptionsServiceStub;
app.addUrl = 'https://example.com/channel';
app.titleRegex = '[';
app.addSubscription();
expect(subs.subscribeCalls.length).toBe(0);
expect(alertSpy).toHaveBeenCalledWith('Invalid subscription title filter (regex)');
alertSpy.mockRestore();
});
});
+133 -40
View File
@@ -1,7 +1,7 @@
import { AsyncPipe, DatePipe, KeyValuePipe, NgTemplateOutlet } from '@angular/common';
import { HttpClient } from '@angular/common/http';
import { AfterViewInit, ChangeDetectionStrategy, ChangeDetectorRef, Component, DestroyRef, ElementRef, viewChild, inject, OnDestroy, OnInit } from '@angular/core';
import { Observable, Subscription, map, distinctUntilChanged, finalize } from 'rxjs';
import { Observable, Subject, Subscription, from, map, distinctUntilChanged, finalize, mergeMap, takeUntil, tap } from 'rxjs';
import { FormsModule } from '@angular/forms';
import { takeUntilDestroyed } from '@angular/core/rxjs-interop';
import { FontAwesomeModule } from '@fortawesome/angular-fontawesome';
@@ -81,6 +81,8 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
playlistItemLimit!: number;
splitByChapters: boolean;
chapterTemplate: string;
clipStart = '';
clipEnd = '';
subtitleLanguage: string;
subtitleMode: string;
ytdlOptionsPresets: string[] = [];
@@ -90,6 +92,10 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
cancelRequested = false;
subscribeInProgress = false;
checkIntervalMinutes = 60;
titleRegex = '';
skipSubscriberOnly = false;
editingTitleRegexId: string | null = null;
titleRegexEditDraft = '';
cachedSubs: [string, SubscriptionRow][] = [];
selectedSubscriptionIds = new Set<string>();
checkingSubscriptionIds = new Set<string>();
@@ -104,8 +110,15 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
batchImportModalOpen = false;
batchImportText = '';
batchImportStatus = '';
batchImportCount = 0;
batchImportTotal = 0;
batchImportFailures = 0;
importInProgress = false;
cancelImportFlag = false;
private batchImportCancel$ = new Subject<void>();
// Maximum number of /add requests to have in-flight at once during a batch
// import. Keeps the server from being hit with hundreds of simultaneous
// yt-dlp metadata extractions when a user pastes a huge URL list.
private static readonly BATCH_IMPORT_CONCURRENCY = 4;
ytDlpOptionsUpdateTime: string | null = null;
ytDlpVersion: string | null = null;
metubeVersion: string | null = null;
@@ -232,6 +245,8 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
this.splitByChapters = this.cookieService.get('metube_split_chapters') === 'true';
// Will be set from backend configuration, use empty string as placeholder
this.chapterTemplate = this.cookieService.get('metube_chapter_template') || '';
this.clipStart = this.cookieService.get('metube_clip_start') || '';
this.clipEnd = this.cookieService.get('metube_clip_end') || '';
this.subtitleLanguage = this.cookieService.get('metube_subtitle_language') || 'en';
this.subtitleMode = this.cookieService.get('metube_subtitle_mode') || 'prefer_manual';
this.ytdlOptionsPresets = this.loadYtdlOptionsPresetsFromCookie();
@@ -553,6 +568,15 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
alert('Please enter a URL');
return;
}
const tr = (this.titleRegex || '').trim();
if (tr) {
try {
void RegExp(tr);
} catch {
alert('Invalid subscription title filter (regex)');
return;
}
}
if (payload.splitByChapters && !payload.chapterTemplate.includes('%(section_number)')) {
alert('Chapter template must include %(section_number)');
return;
@@ -560,11 +584,17 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
if (!this.validateYtdlOptionsOverrides(payload.ytdlOptionsOverrides)) {
return;
}
// Subscriptions do not support clip ranges (backend rejects clip fields).
const { clipStart: _clipStart, clipEnd: _clipEnd, ...subscribeBase } = payload;
void _clipStart;
void _clipEnd;
this.subscribeInProgress = true;
this.subscriptionsSvc
.subscribe({
...payload,
...subscribeBase,
checkIntervalMinutes: this.checkIntervalMinutes,
titleRegex: tr,
skipSubscriberOnly: this.skipSubscriberOnly,
})
.pipe(
takeUntilDestroyed(this.destroyRef),
@@ -580,11 +610,45 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
alert(r.msg || 'Subscribe failed');
} else {
this.addUrl = '';
this.titleRegex = '';
this.skipSubscriberOnly = false;
}
},
});
}
beginEditTitleRegex(id: string, current: string | undefined) {
this.editingTitleRegexId = id;
this.titleRegexEditDraft = current ?? '';
this.cdr.markForCheck();
}
cancelEditTitleRegex() {
this.editingTitleRegexId = null;
this.titleRegexEditDraft = '';
this.cdr.markForCheck();
}
saveTitleRegex(id: string) {
const raw = (this.titleRegexEditDraft || '').trim();
if (raw) {
try {
void RegExp(raw);
} catch {
alert('Invalid subscription title filter (regex)');
return;
}
}
this.subscriptionsSvc.update(id, { title_regex: raw }).subscribe((res) => {
const error = this.getStatusError(res);
if (error) {
alert(error || 'Update subscription failed');
return;
}
this.cancelEditTitleRegex();
});
}
deleteSubscription(id: string) {
this.subscriptionsSvc.delete([id]).subscribe((res) => {
const error = this.getStatusError(res);
@@ -754,6 +818,14 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
this.cookieService.set('metube_chapter_template', this.chapterTemplate, { expires: this.settingsCookieExpiryDays });
}
clipStartChanged() {
this.cookieService.set('metube_clip_start', this.clipStart, { expires: this.settingsCookieExpiryDays });
}
clipEndChanged() {
this.cookieService.set('metube_clip_end', this.clipEnd, { expires: this.settingsCookieExpiryDays });
}
subtitleLanguageChanged() {
this.cookieService.set('metube_subtitle_language', this.subtitleLanguage, { expires: this.settingsCookieExpiryDays });
this.saveSelection(this.downloadType);
@@ -980,6 +1052,8 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
ytdlOptionsOverrides: allowYtdlOptionsOverrides
? (overrides.ytdlOptionsOverrides ?? this.ytdlOptionsOverrides)
: '',
clipStart: overrides.clipStart ?? this.clipStart,
clipEnd: overrides.clipEnd ?? this.clipEnd,
};
}
@@ -1053,6 +1127,8 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
? [...download.ytdl_options_presets]
: [],
ytdlOptionsOverrides: download.ytdl_options_overrides ? JSON.stringify(download.ytdl_options_overrides) : '',
clipStart: download.clip_start != null ? String(download.clip_start) : '',
clipEnd: download.clip_end != null ? String(download.clip_end) : '',
});
this.downloads.delById('done', [key]).subscribe();
}
@@ -1173,8 +1249,10 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
this.batchImportModalOpen = true;
this.batchImportText = '';
this.batchImportStatus = '';
this.batchImportCount = 0;
this.batchImportTotal = 0;
this.batchImportFailures = 0;
this.importInProgress = false;
this.cancelImportFlag = false;
setTimeout(() => {
const textarea = document.getElementById('batch-import-textarea');
if (textarea instanceof HTMLTextAreaElement) {
@@ -1200,48 +1278,63 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
return;
}
this.importInProgress = true;
this.cancelImportFlag = false;
this.batchImportStatus = `Starting to import ${urls.length} URLs...`;
let index = 0;
const delayBetween = 1000;
const processNext = () => {
if (this.cancelImportFlag) {
this.batchImportStatus = `Import cancelled after ${index} of ${urls.length} URLs.`;
this.importInProgress = false;
return;
}
if (index >= urls.length) {
this.batchImportStatus = `Finished importing ${urls.length} URLs.`;
this.importInProgress = false;
return;
}
const url = urls[index];
this.batchImportStatus = `Importing URL ${index + 1} of ${urls.length}: ${url}`;
// Pass current selection options to backend
this.downloads.add(this.buildAddPayload({ url }))
.subscribe({
next: (status: Status) => {
this.batchImportCount = 0;
this.batchImportFailures = 0;
this.batchImportTotal = urls.length;
this.updateBatchImportStatus();
from(urls).pipe(
mergeMap(
url => this.downloads.add(this.buildAddPayload({ url })).pipe(
// downloads.add() already catches HTTP errors and emits a single
// Status value, so `tap` (not `finalize`) is the right place to
// count. This avoids incrementing the counter when an in-flight
// request is aborted by cancellation.
tap((status: Status) => {
if (status.status === 'error') {
alert(`Error adding URL ${url}: ${status.msg}`);
this.batchImportFailures++;
console.error(`Error adding URL ${url}: ${status.msg}`);
}
index++;
setTimeout(processNext, delayBetween);
},
error: (err) => {
console.error(`Error importing URL ${url}:`, err);
index++;
setTimeout(processNext, delayBetween);
}
});
};
processNext();
this.batchImportCount++;
this.updateBatchImportStatus();
this.cdr.markForCheck();
}),
),
App.BATCH_IMPORT_CONCURRENCY,
),
takeUntil(this.batchImportCancel$),
takeUntilDestroyed(this.destroyRef),
finalize(() => {
this.importInProgress = false;
this.updateBatchImportStatus(true);
this.cdr.markForCheck();
}),
).subscribe();
}
// Cancel the batch import process
private updateBatchImportStatus(done = false): void {
const parts: string[] = [];
if (done) {
const processed = this.batchImportCount;
if (processed < this.batchImportTotal) {
parts.push(`Import cancelled after ${processed} of ${this.batchImportTotal} URLs.`);
} else {
parts.push(`Finished importing ${this.batchImportTotal} URLs.`);
}
} else {
parts.push(`Importing ${this.batchImportCount} of ${this.batchImportTotal} URLs...`);
}
if (this.batchImportFailures > 0) {
parts.push(`${this.batchImportFailures} failed.`);
}
this.batchImportStatus = parts.join(' ');
}
// Cancel the batch import process: aborts in-flight and pending requests
// immediately via the cancellation Subject wired into the pipeline.
cancelBatchImport(): void {
if (this.importInProgress) {
this.cancelImportFlag = true;
this.batchImportStatus += ' Cancelling...';
this.batchImportCancel$.next();
}
}
+2
View File
@@ -16,6 +16,8 @@ export interface Download {
subtitle_mode?: string;
ytdl_options_presets?: string[];
ytdl_options_overrides?: Record<string, unknown>;
clip_start?: number;
clip_end?: number;
status: string;
msg: string;
percent: number;
+2
View File
@@ -9,6 +9,8 @@ export interface SubscriptionRow {
format: string;
quality: string;
folder: string;
title_regex?: string;
skip_subscriber_only?: boolean;
last_checked: number | null;
seen_count: number;
error: string | null;
@@ -41,6 +41,8 @@ function basePayload(): AddDownloadPayload {
subtitleMode: 'prefer_manual',
ytdlOptionsPresets: [],
ytdlOptionsOverrides: '',
clipStart: '',
clipEnd: '',
};
}
@@ -88,6 +90,24 @@ describe('DownloadsService', () => {
req.flush({ status: 'ok' });
});
it('add() sends clip_start and clip_end when set', () => {
service
.add({
...basePayload(),
clipStart: '1:00',
clipEnd: '2:00',
})
.subscribe();
const req = httpMock.expectOne('add');
expect(req.request.body).toEqual(
expect.objectContaining({
clip_start: '1:00',
clip_end: '2:00',
}),
);
req.flush({ status: 'ok' });
});
it('getPresets() fetches configured preset names', () => {
service.getPresets().subscribe((result) => {
expect(result).toEqual({ presets: ['Preset A'] });
+9 -2
View File
@@ -22,6 +22,8 @@ export interface AddDownloadPayload {
subtitleMode: string;
ytdlOptionsPresets: string[];
ytdlOptionsOverrides: string;
clipStart?: string;
clipEnd?: string;
}
@Injectable({
providedIn: 'root'
@@ -129,7 +131,7 @@ export class DownloadsService {
}
public add(payload: AddDownloadPayload) {
return this.http.post<Status>('add', {
const body: Record<string, unknown> = {
url: payload.url,
download_type: payload.downloadType,
codec: payload.codec,
@@ -145,7 +147,12 @@ export class DownloadsService {
subtitle_mode: payload.subtitleMode,
ytdl_options_presets: payload.ytdlOptionsPresets,
ytdl_options_overrides: payload.ytdlOptionsOverrides,
}).pipe(
};
const cs = payload.clipStart?.trim();
const ce = payload.clipEnd?.trim();
if (cs) body['clip_start'] = cs;
if (ce) body['clip_end'] = ce;
return this.http.post<Status>('add', body).pipe(
catchError(this.handleHTTPError)
);
}
+13 -1
View File
@@ -10,6 +10,8 @@ import { AddDownloadPayload } from './downloads.service';
export interface SubscribePayload extends AddDownloadPayload {
checkIntervalMinutes: number;
titleRegex: string;
skipSubscriberOnly: boolean;
}
@Injectable({
@@ -97,6 +99,8 @@ export class SubscriptionsService {
ytdl_options_presets: payload.ytdlOptionsPresets,
ytdl_options_overrides: payload.ytdlOptionsOverrides,
check_interval_minutes: payload.checkIntervalMinutes,
title_regex: payload.titleRegex,
skip_subscriber_only: payload.skipSubscriberOnly,
})
.pipe(catchError((err) => this.handleHTTPError(err)));
}
@@ -105,7 +109,15 @@ export class SubscriptionsService {
return this.http.post('subscriptions/delete', { ids }).pipe(catchError((err) => this.handleHTTPError(err)));
}
update(id: string, changes: Partial<Pick<SubscriptionRow, 'enabled' | 'check_interval_minutes' | 'name'>>) {
update(
id: string,
changes: Partial<
Pick<
SubscriptionRow,
'enabled' | 'check_interval_minutes' | 'name' | 'title_regex' | 'skip_subscriber_only'
>
>,
) {
return this.http
.post('subscriptions/update', { id, ...changes })
.pipe(catchError((err) => this.handleHTTPError(err)));
Generated
+16 -16
View File
@@ -324,15 +324,15 @@ wheels = [
[[package]]
name = "deno"
version = "2.7.11"
version = "2.7.12"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d7/3f/a0477c72b847c0082ceb8885261eb14fb4addde22e8fb0146c011636979b/deno-2.7.11.tar.gz", hash = "sha256:342a656fca446fadc261ed22af35693b6c34e79129fa2bd387a1e5d39f496a99", size = 8167, upload-time = "2026-04-01T12:48:16.673Z" }
sdist = { url = "https://files.pythonhosted.org/packages/5c/ac/39b45b51e7aaf9cad1de36f91bd58677818a7d7a867fa0721faff43b67c4/deno-2.7.12.tar.gz", hash = "sha256:7ef413693b4a9d86837a6f4991738429908f6f42f6f3ed85254ab0e679438049", size = 8167, upload-time = "2026-04-09T20:39:50.505Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d1/64/c8cb5a9c50135ada59b412b7511852d551d2618e169f48a8e7b8e90a382a/deno-2.7.11-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:76aa633656b07f64cfda3aa94a1e16bc123034ba2fe676d23b179f8326728534", size = 47857669, upload-time = "2026-04-01T12:48:01.947Z" },
{ url = "https://files.pythonhosted.org/packages/27/f3/250216e71e21cfc291ff6eb6503a479410ff927c9f97ebb644463c620692/deno-2.7.11-py3-none-macosx_11_0_arm64.whl", hash = "sha256:a1dd1ceb0c2109b54334a621503e5297d473f5c3e6702992dfa68d9b7cdf1177", size = 44607869, upload-time = "2026-04-01T12:48:05.23Z" },
{ url = "https://files.pythonhosted.org/packages/01/51/b8fcb7d6882d659abd678f48fc5bc89aa0aa10f5a399fca3823680f277ca/deno-2.7.11-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:56578808c891d1daeadc279ac73445a24a83c2c721b37016fd92b1aa9224979b", size = 48394760, upload-time = "2026-04-01T12:48:08.049Z" },
{ url = "https://files.pythonhosted.org/packages/a0/18/91d1ceb2e15b4446fcecdf4be17ed59b02f5b35d8c98c94e7eaadf10591e/deno-2.7.11-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:58eea29d18fe8f80f2a4354d96225a31961ed3fa68336a7fcdd6add331df6acd", size = 50418090, upload-time = "2026-04-01T12:48:10.84Z" },
{ url = "https://files.pythonhosted.org/packages/b6/8d/7a801d85a0a8233d00ea1a381fd32e76c81be9db78ada162e211e7619221/deno-2.7.11-py3-none-win_amd64.whl", hash = "sha256:e2b69676e52543153b82e926f558983f9251db2251474f3225301cf7aca05a2b", size = 49417475, upload-time = "2026-04-01T12:48:14.158Z" },
{ url = "https://files.pythonhosted.org/packages/77/f0/0be717182294e9b61e58cd89b88d1b711c7e9c134e53890474b8d1e95704/deno-2.7.12-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:74677d1be23426b1f16ce33c1a96868bc7554e1d41a12fb4d446017bb1955d1d", size = 47683364, upload-time = "2026-04-09T20:39:35.086Z" },
{ url = "https://files.pythonhosted.org/packages/8d/44/ca1de5368b193c9819c57d505ff8ceb86dc38e2e87856b23c23b58abd234/deno-2.7.12-py3-none-macosx_11_0_arm64.whl", hash = "sha256:acf94fa815ac84a948b196f53a7e088ed126ec2e390a25d92cdfdc8c5d8a851e", size = 44460697, upload-time = "2026-04-09T20:39:38.5Z" },
{ url = "https://files.pythonhosted.org/packages/ce/68/ce111604f5877220e53fb10c5180ffaa484f829f20fcbc7f7115a90afadd/deno-2.7.12-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:ab7c12ee0e10cd232dbb8f7cf5802bfdbcc3462c0b1453726b158d8a27d4185f", size = 48237362, upload-time = "2026-04-09T20:39:41.555Z" },
{ url = "https://files.pythonhosted.org/packages/d7/07/d8ba3284e098128076cd9f84e85d2e62d12fa1e5b8e5c47681983017af1a/deno-2.7.12-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:b396675fef296530c386387990d26d118398452bac47541e739b353b9ff8809f", size = 50247653, upload-time = "2026-04-09T20:39:44.956Z" },
{ url = "https://files.pythonhosted.org/packages/07/b0/c6c215eca2aa54b700a95867901832245518b2f217edfc78e733401bf8b3/deno-2.7.12-py3-none-win_amd64.whl", hash = "sha256:ddeeb03427d7ce0979a395ae2d1aa5ebcddeb97ab0fd9e7dfa56a08e0e5b817c", size = 49200604, upload-time = "2026-04-09T20:39:48.218Z" },
]
[[package]]
@@ -593,20 +593,20 @@ wheels = [
[[package]]
name = "packaging"
version = "26.0"
version = "26.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/65/ee/299d360cdc32edc7d2cf530f3accf79c4fca01e96ffc950d8a52213bd8e4/packaging-26.0.tar.gz", hash = "sha256:00243ae351a257117b6a241061796684b084ed1c516a08c48a3f7e147a9d80b4", size = 143416, upload-time = "2026-01-21T20:50:39.064Z" }
sdist = { url = "https://files.pythonhosted.org/packages/df/de/0d2b39fb4af88a0258f3bac87dfcbb48e73fbdea4a2ed0e2213f9a4c2f9a/packaging-26.1.tar.gz", hash = "sha256:f042152b681c4bfac5cae2742a55e103d27ab2ec0f3d88037136b6bfe7c9c5de", size = 215519, upload-time = "2026-04-14T21:12:49.362Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b7/b9/c538f279a4e237a006a2c98387d081e9eb060d203d8ed34467cc0f0b9b53/packaging-26.0-py3-none-any.whl", hash = "sha256:b36f1fef9334a5588b4166f8bcd26a14e521f2b55e6b9de3aaa80d3ff7a37529", size = 74366, upload-time = "2026-01-21T20:50:37.788Z" },
{ url = "https://files.pythonhosted.org/packages/7a/c2/920ef838e2f0028c8262f16101ec09ebd5969864e5a64c4c05fad0617c56/packaging-26.1-py3-none-any.whl", hash = "sha256:5d9c0669c6285e491e0ced2eee587eaf67b670d94a19e94e3984a481aba6802f", size = 95831, upload-time = "2026-04-14T21:12:47.56Z" },
]
[[package]]
name = "platformdirs"
version = "4.9.4"
version = "4.9.6"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/19/56/8d4c30c8a1d07013911a8fdbd8f89440ef9f08d07a1b50ab8ca8be5a20f9/platformdirs-4.9.4.tar.gz", hash = "sha256:1ec356301b7dc906d83f371c8f487070e99d3ccf9e501686456394622a01a934", size = 28737, upload-time = "2026-03-05T18:34:13.271Z" }
sdist = { url = "https://files.pythonhosted.org/packages/9f/4a/0883b8e3802965322523f0b200ecf33d31f10991d0401162f4b23c698b42/platformdirs-4.9.6.tar.gz", hash = "sha256:3bfa75b0ad0db84096ae777218481852c0ebc6c727b3168c1b9e0118e458cf0a", size = 29400, upload-time = "2026-04-09T00:04:10.812Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/63/d7/97f7e3a6abb67d8080dd406fd4df842c2be0efaf712d1c899c32a075027c/platformdirs-4.9.4-py3-none-any.whl", hash = "sha256:68a9a4619a666ea6439f2ff250c12a853cd1cbd5158d258bd824a7df6be2f868", size = 21216, upload-time = "2026-03-05T18:34:12.172Z" },
{ url = "https://files.pythonhosted.org/packages/75/a6/a0a304dc33b49145b21f4808d763822111e67d1c3a32b524a1baf947b6e1/platformdirs-4.9.6-py3-none-any.whl", hash = "sha256:e61adb1d5e5cb3441b4b7710bea7e4c12250ca49439228cc1021c00dcfac0917", size = 21348, upload-time = "2026-04-09T00:04:09.463Z" },
]
[[package]]
@@ -755,7 +755,7 @@ wheels = [
[[package]]
name = "pytest"
version = "9.0.2"
version = "9.0.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
@@ -764,9 +764,9 @@ dependencies = [
{ name = "pluggy" },
{ name = "pygments" },
]
sdist = { url = "https://files.pythonhosted.org/packages/d1/db/7ef3487e0fb0049ddb5ce41d3a49c235bf9ad299b6a25d5780a89f19230f/pytest-9.0.2.tar.gz", hash = "sha256:75186651a92bd89611d1d9fc20f0b4345fd827c41ccd5c299a868a05d70edf11", size = 1568901, upload-time = "2025-12-06T21:30:51.014Z" }
sdist = { url = "https://files.pythonhosted.org/packages/7d/0d/549bd94f1a0a402dc8cf64563a117c0f3765662e2e668477624baeec44d5/pytest-9.0.3.tar.gz", hash = "sha256:b86ada508af81d19edeb213c681b1d48246c1a91d304c6c81a427674c17eb91c", size = 1572165, upload-time = "2026-04-07T17:16:18.027Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/3b/ab/b3226f0bd7cdcf710fbede2b3548584366da3b19b5021e74f5bde2a8fa3f/pytest-9.0.2-py3-none-any.whl", hash = "sha256:711ffd45bf766d5264d487b917733b453d917afd2b0ad65223959f59089f875b", size = 374801, upload-time = "2025-12-06T21:30:49.154Z" },
{ url = "https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9", size = 375249, upload-time = "2026-04-07T17:16:16.13Z" },
]
[[package]]