mirror of
https://github.com/alexta69/metube.git
synced 2026-06-13 16:40:05 +00:00
Compare commits
10 Commits
2026.04.10
...
2026.04.18
| Author | SHA1 | Date | |
|---|---|---|---|
| e9f979b349 | |||
| ab42325db5 | |||
| 1a32eba474 | |||
| 29ccc42409 | |||
| f2d71cbe2e | |||
| 03f71fd257 | |||
| 210c607c53 | |||
| 381896901a | |||
| 4330d3b6c6 | |||
| 06c4a2c4a8 |
@@ -4,6 +4,8 @@ on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
paths-ignore:
|
||||
- '**.md'
|
||||
|
||||
jobs:
|
||||
quality-checks:
|
||||
@@ -209,7 +211,7 @@ jobs:
|
||||
git push origin ":refs/tags/$TAG_NAME" || true
|
||||
fi
|
||||
- name: Create GitHub Release
|
||||
uses: softprops/action-gh-release@v2
|
||||
uses: softprops/action-gh-release@v3
|
||||
with:
|
||||
tag_name: ${{ steps.date.outputs.date }}
|
||||
name: Release ${{ steps.date.outputs.date }}
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
# Agent Guidelines
|
||||
|
||||
## README.md size constraint
|
||||
|
||||
The README.md is synced to Docker Hub, which has a **25,000 character limit**.
|
||||
Any change to README.md **must** keep the file under 25,000 characters (`wc -c README.md`).
|
||||
If an addition would exceed the limit, trim existing prose elsewhere — prefer tightening verbose descriptions over removing sections.
|
||||
|
||||
## Tech stack
|
||||
|
||||
- **Backend:** Python 3.13+, aiohttp, python-socketio 5.x, yt-dlp
|
||||
- **Frontend:** Angular 21, TypeScript, Bootstrap 5, SASS, ngx-socket-io
|
||||
- **Package managers:** uv (Python), pnpm (frontend)
|
||||
- **Container:** Multi-stage Docker (Node builder + Python runtime), multi-arch (amd64/arm64)
|
||||
|
||||
## Build & test commands
|
||||
|
||||
```bash
|
||||
# Frontend (run from ui/)
|
||||
pnpm install --frozen-lockfile
|
||||
pnpm run lint
|
||||
pnpm run build
|
||||
pnpm exec ng test --watch=false
|
||||
|
||||
# Backend (run from repo root)
|
||||
uv sync --frozen --group dev
|
||||
python -m compileall app
|
||||
uv run pytest app/tests/
|
||||
```
|
||||
|
||||
All of these run in CI (`.github/workflows/main.yml`) on every push to master and must pass.
|
||||
|
||||
## Code style
|
||||
|
||||
Follow `.editorconfig`:
|
||||
- Python: 4-space indent
|
||||
- Everything else (TypeScript, YAML, JSON, HTML): 2-space indent
|
||||
- UTF-8, LF line endings, trim trailing whitespace, final newline
|
||||
|
||||
Frontend additionally uses ESLint (`ui/eslint.config.js`) and Prettier (config in `ui/package.json`: `printWidth=100`, `singleQuote=true`).
|
||||
|
||||
## Project structure
|
||||
|
||||
```
|
||||
app/main.py — HTTP server, Socket.IO events, REST API routes, Config class
|
||||
app/ytdl.py — Download queue logic, yt-dlp integration
|
||||
app/subscriptions.py — Channel/playlist subscription manager
|
||||
app/state_store.py — JSON-based persistent storage with atomic writes
|
||||
app/dl_formats.py — Video/audio codec/quality mapping
|
||||
app/tests/ — pytest tests (asyncio_mode=auto)
|
||||
ui/src/app/ — Angular standalone components (no NgModules)
|
||||
```
|
||||
|
||||
## Key conventions
|
||||
|
||||
- Backend configuration lives in the `Config` class in `app/main.py` with env-var defaults in `_DEFAULTS`. New env vars go there.
|
||||
- Real-time communication uses Socket.IO events, not REST polling.
|
||||
- Frontend uses standalone Angular components with `inject()` for DI, RxJS Subjects for state, and `takeUntilDestroyed()` for cleanup.
|
||||
- State is persisted as JSON files via `AtomicJsonStore` in `app/state_store.py`.
|
||||
- No pre-commit hooks — linting and tests are enforced in CI only.
|
||||
@@ -82,7 +82,7 @@ Certain values can be set via environment variables, using the `-e` parameter on
|
||||
* __HTTPS__: Use `https` instead of `http` (__CERTFILE__ and __KEYFILE__ required). Defaults to `false`.
|
||||
* __CERTFILE__: HTTPS certificate file path.
|
||||
* __KEYFILE__: HTTPS key file path.
|
||||
* __CORS_ALLOWED_ORIGINS__: Comma-separated list of origins permitted to make cross-origin requests to the MeTube API. When unset or empty, all cross-origin requests are denied. This must be configured for [bookmarklets](#-bookmarklet) and any other browser-based tools that contact MeTube from a different origin. Example: `https://www.youtube.com,https://www.vimeo.com`.
|
||||
* __CORS_ALLOWED_ORIGINS__: Comma-separated list of origins permitted to make cross-origin requests to the MeTube API. When unset or empty, all cross-origin requests are denied. Set to `*` to allow all origins. This must be configured for [browser extensions](#-browser-extensions), [bookmarklets](#-bookmarklet), and any other browser-based tools that contact MeTube from a different origin. For browser extensions use `*` (see below); for bookmarklets you can list specific sites, e.g. `https://www.youtube.com,https://www.vimeo.com`.
|
||||
* __ROBOTS_TXT__: A path to a `robots.txt` file mounted in the container.
|
||||
|
||||
### 🏠 Basic Setup
|
||||
@@ -104,11 +104,13 @@ MeTube lets you customize how [yt-dlp](https://github.com/yt-dlp/yt-dlp) behaves
|
||||
|
||||
When a download starts, these layers are combined in order. If the same option appears in more than one layer, the more specific one wins: per-download overrides beat presets, and presets beat global options.
|
||||
|
||||
In JSON presets and overrides, setting an option to **`null`** clears that option for that download (for example, `"download_archive": null` overrides a global archive path so the archive is not used). This follows yt-dlp’s usual meaning of `None` for that option.
|
||||
|
||||
### Option format
|
||||
|
||||
yt-dlp options in MeTube are expressed as JSON objects. The keys are yt-dlp API option names, which roughly correspond to command-line flags with dashes replaced by underscores. For example, the command-line flag `--embed-thumbnail` becomes `"embed_thumbnail": true` in JSON.
|
||||
yt-dlp options in MeTube are expressed as JSON objects. The keys are yt-dlp API option names, which roughly correspond to command-line flags with dashes replaced by underscores. For example, the command-line flag `--write-subs` becomes `"writesubtitles": true` in JSON.
|
||||
|
||||
> **Tip:** Some command-line flags don't have a direct single-key equivalent — for instance, `--recode-video` must be expressed via `"postprocessors"`. A full list of available API options can be found [in the yt-dlp source](https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/YoutubeDL.py#L224), and [this conversion script](https://github.com/yt-dlp/yt-dlp/blob/master/devscripts/cli_to_api.py) can help translate command-line flags to their API equivalents.
|
||||
> **Tip:** Some command-line flags don't have a direct single-key equivalent — for instance, `--embed-thumbnail` and `--recode-video` must be expressed via `"postprocessors"`. A full list of available API options can be found [in the yt-dlp source](https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/YoutubeDL.py#L224), and [this conversion script](https://github.com/yt-dlp/yt-dlp/blob/master/devscripts/cli_to_api.py) can help translate command-line flags to their API equivalents.
|
||||
|
||||
### Global options
|
||||
|
||||
@@ -118,7 +120,7 @@ Global options form the baseline for every download. There are two ways to defin
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- 'YTDL_OPTIONS={"writesubtitles": true, "subtitleslangs": ["en", "de"], "updatetime": false, "embed_thumbnail": true}'
|
||||
- 'YTDL_OPTIONS={"writesubtitles": true, "subtitleslangs": ["en", "de"], "updatetime": false, "writethumbnail": true}'
|
||||
```
|
||||
|
||||
**Via a JSON file** (`YTDL_OPTIONS_FILE`) — mount a file into the container and point to it:
|
||||
@@ -137,7 +139,7 @@ where `ytdl-options.json` contains:
|
||||
"writesubtitles": true,
|
||||
"subtitleslangs": ["en", "de"],
|
||||
"updatetime": false,
|
||||
"embed_thumbnail": true
|
||||
"writethumbnail": true
|
||||
}
|
||||
```
|
||||
|
||||
@@ -204,6 +206,8 @@ When a download starts, the final set of yt-dlp options is built in this order:
|
||||
2. Apply each selected **preset** in order (later presets overwrite earlier ones for conflicting keys).
|
||||
3. Apply any **per-download overrides** on top (overwrite everything else for conflicting keys).
|
||||
|
||||
MeTube always forces its own flat-extract behaviour during the initial metadata fetch (`extract_flat`, `noplaylist`, etc.); presets cannot override those keys for that phase.
|
||||
|
||||
**Example:** Suppose your global options set `"writesubtitles": false`, but you select a preset that sets `"writesubtitles": true`. Subtitles will be written for that download because the preset overrides the global setting. If you additionally enter `{"writesubtitles": false}` in the per-download overrides field, that value wins and subtitles will not be written.
|
||||
|
||||
### Configuration cookbooks
|
||||
@@ -226,7 +230,9 @@ In case you need to use your browser's cookies with MeTube, for example to downl
|
||||
|
||||
## 🔌 Browser extensions
|
||||
|
||||
Browser extensions allow right-clicking videos and sending them directly to MeTube. Please note that if you're on an HTTPS page, your MeTube instance must be behind an HTTPS reverse proxy (see below) for the extensions to work.
|
||||
Browser extensions allow right-clicking videos and sending them directly to MeTube. If you're on an HTTPS page, your MeTube instance must be behind an HTTPS reverse proxy (see below) for extensions to work.
|
||||
|
||||
Since browser extensions make requests from their own origin (`chrome-extension://...` or `moz-extension://...`), you must set `CORS_ALLOWED_ORIGINS=*` for them to work.
|
||||
|
||||
__Chrome:__ contributed by [Rpsl](https://github.com/rpsl). You can install it from [Google Chrome Webstore](https://chrome.google.com/webstore/detail/metube-downloader/fbmkmdnlhacefjljljlbhkodfmfkijdh) or use developer mode and install [from sources](https://github.com/Rpsl/metube-browser-extension).
|
||||
|
||||
@@ -254,23 +260,15 @@ javascript:!function(){xhr=new XMLHttpRequest();xhr.open("POST","https://metube.
|
||||
javascript:(function(){xhr=new XMLHttpRequest();xhr.open("POST","https://metube.domain.com/add");xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function(){if(xhr.status==200){alert("Sent to metube!")}else{alert("Send to metube failed. Check the javascript console for clues.")}}})();
|
||||
```
|
||||
|
||||
The above bookmarklets use `alert()` as a success/failure notification. The following will show a toast message instead:
|
||||
|
||||
Chrome:
|
||||
The above bookmarklets use `alert()` for notifications. This variant shows a toast instead (Chrome — for Firefox, replace the `!function(){...}()` wrapper with `(function(){...})()`):
|
||||
|
||||
```javascript
|
||||
javascript:!function(){function notify(msg) {var sc = document.scrollingElement.scrollTop; var text = document.createElement('span');text.innerHTML=msg;var ts = text.style;ts.all = 'revert';ts.color = '#000';ts.fontFamily = 'Verdana, sans-serif';ts.fontSize = '15px';ts.backgroundColor = 'white';ts.padding = '15px';ts.border = '1px solid gainsboro';ts.boxShadow = '3px 3px 10px';ts.zIndex = '100';document.body.appendChild(text);ts.position = 'absolute'; ts.top = 50 + sc + 'px'; ts.left = (window.innerWidth / 2)-(text.offsetWidth / 2) + 'px'; setTimeout(function () { text.style.visibility = "hidden"; }, 1500);}xhr=new XMLHttpRequest();xhr.open("POST","https://metube.domain.com/add");xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function() { if(xhr.status==200){notify("Sent to metube!")}else {notify("Send to metube failed. Check the javascript console for clues.")}}}();
|
||||
```
|
||||
|
||||
Firefox:
|
||||
|
||||
```javascript
|
||||
javascript:(function(){function notify(msg) {var sc = document.scrollingElement.scrollTop; var text = document.createElement('span');text.innerHTML=msg;var ts = text.style;ts.all = 'revert';ts.color = '#000';ts.fontFamily = 'Verdana, sans-serif';ts.fontSize = '15px';ts.backgroundColor = 'white';ts.padding = '15px';ts.border = '1px solid gainsboro';ts.boxShadow = '3px 3px 10px';ts.zIndex = '100';document.body.appendChild(text);ts.position = 'absolute'; ts.top = 50 + sc + 'px'; ts.left = (window.innerWidth / 2)-(text.offsetWidth / 2) + 'px'; setTimeout(function () { text.style.visibility = "hidden"; }, 1500);}xhr=new XMLHttpRequest();xhr.open("POST","https://metube.domain.com/add");xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function() { if(xhr.status==200){notify("Sent to metube!")}else {notify("Send to metube failed. Check the javascript console for clues.")}}})();
|
||||
```
|
||||
|
||||
## ⚡ Raycast extension
|
||||
|
||||
[dotvhs](https://github.com/dotvhs) has created an [extension for Raycast](https://www.raycast.com/dot/metube) that allows adding videos to MeTube directly from Raycast.
|
||||
[dotvhs](https://github.com/dotvhs) has created an [extension for Raycast](https://www.raycast.com/dot/metube) for adding videos to MeTube directly from Raycast.
|
||||
|
||||
## 🔒 HTTPS support, and running behind a reverse proxy
|
||||
|
||||
@@ -294,11 +292,9 @@ services:
|
||||
- KEYFILE=/ssl/key.pem
|
||||
```
|
||||
|
||||
It's also possible to run MeTube behind a reverse proxy, in order to support authentication. HTTPS support can also be added in this way.
|
||||
MeTube can also run behind a reverse proxy for HTTPS termination or authentication. When serving under a subdirectory, set `URL_PREFIX` accordingly.
|
||||
|
||||
When running behind a reverse proxy which remaps the URL (i.e. serves MeTube under a subdirectory and not under root), don't forget to set the URL_PREFIX environment variable to the correct value.
|
||||
|
||||
If you're using the [linuxserver/swag](https://docs.linuxserver.io/general/swag) image for your reverse proxying needs (which I can heartily recommend), it already includes ready snippets for proxying MeTube both in [subfolder](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subfolder.conf.sample) and [subdomain](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subdomain.conf.sample) modes under the `nginx/proxy-confs` directory in the configuration volume. It also includes Authelia which can be used for authentication.
|
||||
The [linuxserver/swag](https://docs.linuxserver.io/general/swag) image includes ready-made snippets for MeTube in [subfolder](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subfolder.conf.sample) and [subdomain](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subdomain.conf.sample) modes, plus Authelia for authentication.
|
||||
|
||||
### 🌐 NGINX
|
||||
|
||||
@@ -350,28 +346,20 @@ example.com {
|
||||
|
||||
## 🔄 Updating yt-dlp
|
||||
|
||||
The engine which powers the actual video downloads in MeTube is [yt-dlp](https://github.com/yt-dlp/yt-dlp). Since video sites regularly change their layouts, frequent updates of yt-dlp are required to keep up.
|
||||
|
||||
There's an automatic nightly build of MeTube which looks for a new version of yt-dlp, and if one exists, the build pulls it and publishes an updated docker image. Therefore, in order to keep up with the changes, it's recommended that you update your MeTube container regularly with the latest image.
|
||||
|
||||
I recommend installing and setting up [watchtower](https://github.com/nicholas-fedor/watchtower) for this purpose.
|
||||
MeTube is powered by [yt-dlp](https://github.com/yt-dlp/yt-dlp), which requires frequent updates as video sites change their layouts. A nightly build automatically publishes a new Docker image whenever a new yt-dlp version is available, so keep your container up to date — [watchtower](https://github.com/nicholas-fedor/watchtower) works well for this.
|
||||
|
||||
## 🔧 Troubleshooting and submitting issues
|
||||
|
||||
Before asking a question or submitting an issue for MeTube, please remember that MeTube is only a UI for [yt-dlp](https://github.com/yt-dlp/yt-dlp). Any issues you might be experiencing with authentication to video websites, postprocessing, permissions, other `YTDL_OPTIONS` configurations which seem not to work, or anything else that concerns the workings of the underlying yt-dlp library, need not be opened on the MeTube project. In order to debug and troubleshoot them, it's advised to try using the yt-dlp binary directly first, bypassing the UI, and once that is working, importing the options that worked for you into `YTDL_OPTIONS`.
|
||||
|
||||
In order to test with the yt-dlp command directly, you can either download it and run it locally, or for a better simulation of its actual conditions, you can run it within the MeTube container itself. Assuming your MeTube container is called `metube`, run the following on your Docker host to get a shell inside the container:
|
||||
MeTube is only a UI for [yt-dlp](https://github.com/yt-dlp/yt-dlp). Issues with authentication, postprocessing, permissions, or `YTDL_OPTIONS` should be debugged with yt-dlp directly first — once working, import those options into MeTube. To test inside the container:
|
||||
|
||||
```bash
|
||||
docker exec -ti metube sh
|
||||
cd /downloads
|
||||
```
|
||||
|
||||
Once there, you can use the yt-dlp command freely.
|
||||
|
||||
## 💡 Submitting feature requests
|
||||
|
||||
MeTube development relies on code contributions by the community. The program as it currently stands fits my own use cases, and is therefore feature-complete as far as I'm concerned. If your use cases are different and require additional features, please feel free to submit PRs that implement those features. It's advisable to create an issue first to discuss the planned implementation, because in an effort to reduce bloat, some PRs may not be accepted. However, note that opening a feature request when you don't intend to implement the feature will rarely result in the request being fulfilled.
|
||||
MeTube development relies on community contributions. If you need additional features, please submit a PR. Create an issue first to discuss the implementation — some PRs may not be accepted to reduce bloat. Feature requests without an accompanying PR are unlikely to be fulfilled.
|
||||
|
||||
## 🛠️ Building and running locally
|
||||
|
||||
|
||||
+6
-1
@@ -92,6 +92,11 @@ class Config:
|
||||
if not self.URL_PREFIX.endswith('/'):
|
||||
self.URL_PREFIX += '/'
|
||||
|
||||
for attr in ('PUBLIC_HOST_URL', 'PUBLIC_HOST_AUDIO_URL'):
|
||||
val = getattr(self, attr)
|
||||
if val and not val.endswith('/'):
|
||||
setattr(self, attr, val + '/')
|
||||
|
||||
# Convert relative addresses to absolute addresses to prevent the failure of file address comparison
|
||||
if self.YTDL_OPTIONS_FILE and self.YTDL_OPTIONS_FILE.startswith('.'):
|
||||
self.YTDL_OPTIONS_FILE = str(Path(self.YTDL_OPTIONS_FILE).resolve())
|
||||
@@ -915,7 +920,7 @@ app.router.add_route('OPTIONS', config.URL_PREFIX + 'delete-cookies', add_cors)
|
||||
|
||||
async def on_prepare(request, response):
|
||||
origin = request.headers.get('Origin')
|
||||
if origin and _cors_origins and origin in _cors_origins:
|
||||
if origin and _cors_origins and ('*' in _cors_origins or origin in _cors_origins):
|
||||
response.headers['Access-Control-Allow-Origin'] = origin
|
||||
response.headers['Access-Control-Allow-Headers'] = 'Content-Type'
|
||||
|
||||
|
||||
@@ -23,6 +23,47 @@ class ConfigTests(unittest.TestCase):
|
||||
c = Config()
|
||||
self.assertEqual(c.URL_PREFIX, "foo/")
|
||||
|
||||
def test_public_host_url_gets_trailing_slash(self):
|
||||
with patch.dict(
|
||||
os.environ,
|
||||
_base_env(PUBLIC_HOST_URL="https://ytdl.example.com"),
|
||||
clear=False,
|
||||
):
|
||||
c = Config()
|
||||
self.assertEqual(c.PUBLIC_HOST_URL, "https://ytdl.example.com/")
|
||||
|
||||
def test_public_host_audio_url_gets_trailing_slash(self):
|
||||
with patch.dict(
|
||||
os.environ,
|
||||
_base_env(PUBLIC_HOST_AUDIO_URL="https://audio.example.com"),
|
||||
clear=False,
|
||||
):
|
||||
c = Config()
|
||||
self.assertEqual(c.PUBLIC_HOST_AUDIO_URL, "https://audio.example.com/")
|
||||
|
||||
def test_public_host_url_empty_stays_empty(self):
|
||||
with patch.dict(
|
||||
os.environ,
|
||||
_base_env(PUBLIC_HOST_URL="", PUBLIC_HOST_AUDIO_URL=""),
|
||||
clear=False,
|
||||
):
|
||||
c = Config()
|
||||
self.assertEqual(c.PUBLIC_HOST_URL, "")
|
||||
self.assertEqual(c.PUBLIC_HOST_AUDIO_URL, "")
|
||||
|
||||
def test_public_host_url_already_slashed_unchanged(self):
|
||||
with patch.dict(
|
||||
os.environ,
|
||||
_base_env(
|
||||
PUBLIC_HOST_URL="https://ytdl.example.com/",
|
||||
PUBLIC_HOST_AUDIO_URL="https://audio.example.com/",
|
||||
),
|
||||
clear=False,
|
||||
):
|
||||
c = Config()
|
||||
self.assertEqual(c.PUBLIC_HOST_URL, "https://ytdl.example.com/")
|
||||
self.assertEqual(c.PUBLIC_HOST_AUDIO_URL, "https://audio.example.com/")
|
||||
|
||||
def test_ytdl_options_json_loaded(self):
|
||||
opts = {"quiet": True, "no_warnings": True}
|
||||
with patch.dict(
|
||||
|
||||
@@ -56,7 +56,7 @@ def test_get_returns_tuple_of_lists(dq_env):
|
||||
async def test_add_single_video_goes_to_pending_when_auto_start_false(dq_env):
|
||||
notifier = AsyncMock()
|
||||
|
||||
def fake_extract(self, url):
|
||||
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
|
||||
return {
|
||||
"_type": "video",
|
||||
"id": "vid1",
|
||||
@@ -86,7 +86,7 @@ async def test_add_single_video_goes_to_pending_when_auto_start_false(dq_env):
|
||||
async def test_cancel_removes_from_pending(dq_env):
|
||||
notifier = AsyncMock()
|
||||
|
||||
def fake_extract(self, url):
|
||||
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
|
||||
return {
|
||||
"_type": "video",
|
||||
"id": "vid1",
|
||||
@@ -118,7 +118,7 @@ async def test_cancel_removes_from_pending(dq_env):
|
||||
async def test_start_pending_moves_to_queue(dq_env):
|
||||
notifier = AsyncMock()
|
||||
|
||||
def fake_extract(self, url):
|
||||
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
|
||||
return {
|
||||
"_type": "video",
|
||||
"id": "vid1",
|
||||
@@ -187,7 +187,7 @@ async def test_add_merges_global_preset_and_override_options(dq_env):
|
||||
"Preset B": {"writesubtitles": False, "ratelimit": 1000},
|
||||
}
|
||||
|
||||
def fake_extract(self, url):
|
||||
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
|
||||
return {
|
||||
"_type": "video",
|
||||
"id": "vid2",
|
||||
@@ -219,3 +219,92 @@ async def test_add_merges_global_preset_and_override_options(dq_env):
|
||||
assert queued.ytdl_opts["ratelimit"] == 1000
|
||||
assert queued.ytdl_opts["proxy"] == "http://override"
|
||||
assert queued.ytdl_opts["embed_thumbnail"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_info_preset_null_download_archive_overrides_global(dq_env):
|
||||
"""Preset download_archive:null must apply during extract_info (global archive otherwise wins first)."""
|
||||
dq_env.YTDL_OPTIONS = {"download_archive": "/tmp/archive.txt"}
|
||||
dq_env.YTDL_OPTIONS_PRESETS = {"NoArchive": {"download_archive": None}}
|
||||
|
||||
captured_params: list = []
|
||||
|
||||
class FakeYoutubeDL:
|
||||
def __init__(self, params=None):
|
||||
captured_params.append(params)
|
||||
|
||||
def extract_info(self, url, download=False):
|
||||
return {
|
||||
"_type": "video",
|
||||
"id": "vid-archive",
|
||||
"title": "Archive Test",
|
||||
"url": url,
|
||||
"webpage_url": url,
|
||||
}
|
||||
|
||||
notifier = AsyncMock()
|
||||
dq = DownloadQueue(dq_env, notifier)
|
||||
with patch("ytdl.yt_dlp.YoutubeDL", FakeYoutubeDL):
|
||||
result = await dq.add(
|
||||
"https://example.com/archive-test",
|
||||
"video",
|
||||
"auto",
|
||||
"any",
|
||||
"best",
|
||||
"",
|
||||
"",
|
||||
0,
|
||||
auto_start=False,
|
||||
ytdl_options_presets=["NoArchive"],
|
||||
)
|
||||
|
||||
assert result["status"] == "ok"
|
||||
assert len(captured_params) == 1
|
||||
extract_params = captured_params[0]
|
||||
assert extract_params.get("download_archive") is None
|
||||
assert extract_params["extract_flat"] is True
|
||||
assert extract_params["noplaylist"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_info_metube_extract_keys_win_over_preset(dq_env):
|
||||
"""MeTube's flat-extract settings must not be overridden by presets."""
|
||||
dq_env.YTDL_OPTIONS = {}
|
||||
dq_env.YTDL_OPTIONS_PRESETS = {
|
||||
"TryOverride": {"extract_flat": False, "noplaylist": False},
|
||||
}
|
||||
|
||||
captured_params: list = []
|
||||
|
||||
class FakeYoutubeDL:
|
||||
def __init__(self, params=None):
|
||||
captured_params.append(params)
|
||||
|
||||
def extract_info(self, url, download=False):
|
||||
return {
|
||||
"_type": "video",
|
||||
"id": "vid-flat",
|
||||
"title": "Flat Test",
|
||||
"url": url,
|
||||
"webpage_url": url,
|
||||
}
|
||||
|
||||
notifier = AsyncMock()
|
||||
dq = DownloadQueue(dq_env, notifier)
|
||||
with patch("ytdl.yt_dlp.YoutubeDL", FakeYoutubeDL):
|
||||
result = await dq.add(
|
||||
"https://example.com/flat-test",
|
||||
"video",
|
||||
"auto",
|
||||
"any",
|
||||
"best",
|
||||
"",
|
||||
"",
|
||||
0,
|
||||
auto_start=False,
|
||||
ytdl_options_presets=["TryOverride"],
|
||||
)
|
||||
|
||||
assert result["status"] == "ok"
|
||||
assert captured_params[0]["extract_flat"] is True
|
||||
assert captured_params[0]["noplaylist"] is True
|
||||
|
||||
+26
-10
@@ -9,6 +9,7 @@ from collections import OrderedDict
|
||||
import time
|
||||
import asyncio
|
||||
import multiprocessing
|
||||
from functools import partial
|
||||
import logging
|
||||
import re
|
||||
import types
|
||||
@@ -796,9 +797,19 @@ class DownloadQueue:
|
||||
log.debug(f'Auto-clearing completed download: {url}')
|
||||
await self.clear([url])
|
||||
|
||||
def __extract_info(self, url):
|
||||
def _build_ytdl_options(self, ytdl_options_presets=None, ytdl_options_overrides=None):
|
||||
"""Merge global options, presets (in order), and per-download overrides."""
|
||||
opts = dict(self.config.YTDL_OPTIONS)
|
||||
for preset_name in ytdl_options_presets or []:
|
||||
opts.update(self.config.YTDL_OPTIONS_PRESETS.get(preset_name, {}))
|
||||
opts.update(ytdl_options_overrides or {})
|
||||
return opts
|
||||
|
||||
def __extract_info(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
|
||||
debug_logging = logging.getLogger().isEnabledFor(logging.DEBUG)
|
||||
return yt_dlp.YoutubeDL(params={
|
||||
user_opts = self._build_ytdl_options(ytdl_options_presets, ytdl_options_overrides)
|
||||
params = {
|
||||
**user_opts,
|
||||
'quiet': not debug_logging,
|
||||
'verbose': debug_logging,
|
||||
'no_color': True,
|
||||
@@ -806,9 +817,11 @@ class DownloadQueue:
|
||||
'ignore_no_formats_error': True,
|
||||
'noplaylist': True,
|
||||
'paths': {"home": self.config.DOWNLOAD_DIR, "temp": self.config.TEMP_DIR},
|
||||
**self.config.YTDL_OPTIONS,
|
||||
**({'impersonate': yt_dlp.networking.impersonate.ImpersonateTarget.from_str(self.config.YTDL_OPTIONS['impersonate'])} if 'impersonate' in self.config.YTDL_OPTIONS else {}),
|
||||
}).extract_info(url, download=False)
|
||||
}
|
||||
imp = user_opts.get('impersonate')
|
||||
if imp is not None:
|
||||
params['impersonate'] = yt_dlp.networking.impersonate.ImpersonateTarget.from_str(imp)
|
||||
return yt_dlp.YoutubeDL(params=params).extract_info(url, download=False)
|
||||
|
||||
def __calc_download_path(self, download_type, folder):
|
||||
base_directory = self.config.AUDIO_DOWNLOAD_DIR if download_type == 'audio' else self.config.DOWNLOAD_DIR
|
||||
@@ -844,10 +857,10 @@ class DownloadQueue:
|
||||
output = self.config.OUTPUT_TEMPLATE_CHANNEL
|
||||
sanitized = {k: _sanitize_path_component(v) for k, v in entry.items()}
|
||||
output = _resolve_outtmpl_fields(output, sanitized, ('channel',))
|
||||
ytdl_options = dict(self.config.YTDL_OPTIONS)
|
||||
for preset_name in getattr(dl, 'ytdl_options_presets', None) or []:
|
||||
ytdl_options.update(self.config.YTDL_OPTIONS_PRESETS.get(preset_name, {}))
|
||||
ytdl_options.update(getattr(dl, 'ytdl_options_overrides', {}) or {})
|
||||
ytdl_options = self._build_ytdl_options(
|
||||
getattr(dl, 'ytdl_options_presets', None),
|
||||
getattr(dl, 'ytdl_options_overrides', {}) or {},
|
||||
)
|
||||
playlist_item_limit = getattr(dl, 'playlist_item_limit', 0)
|
||||
if playlist_item_limit > 0:
|
||||
log.info(f'playlist limit is set. Processing only first {playlist_item_limit} entries')
|
||||
@@ -1037,7 +1050,10 @@ class DownloadQueue:
|
||||
else:
|
||||
already.add(url)
|
||||
try:
|
||||
entry = await asyncio.get_running_loop().run_in_executor(None, self.__extract_info, url)
|
||||
entry = await asyncio.get_running_loop().run_in_executor(
|
||||
None,
|
||||
partial(self.__extract_info, url, ytdl_options_presets, ytdl_options_overrides),
|
||||
)
|
||||
except yt_dlp.utils.YoutubeDLError as exc:
|
||||
return {'status': 'error', 'msg': str(exc)}
|
||||
return await self.__add_entry(
|
||||
|
||||
+10
-10
@@ -23,14 +23,14 @@
|
||||
},
|
||||
"private": true,
|
||||
"dependencies": {
|
||||
"@angular/animations": "^21.2.8",
|
||||
"@angular/common": "^21.2.8",
|
||||
"@angular/compiler": "^21.2.8",
|
||||
"@angular/core": "^21.2.8",
|
||||
"@angular/forms": "^21.2.8",
|
||||
"@angular/platform-browser": "^21.2.8",
|
||||
"@angular/platform-browser-dynamic": "^21.2.8",
|
||||
"@angular/service-worker": "^21.2.8",
|
||||
"@angular/animations": "^21.2.9",
|
||||
"@angular/common": "^21.2.9",
|
||||
"@angular/compiler": "^21.2.9",
|
||||
"@angular/core": "^21.2.9",
|
||||
"@angular/forms": "^21.2.9",
|
||||
"@angular/platform-browser": "^21.2.9",
|
||||
"@angular/platform-browser-dynamic": "^21.2.9",
|
||||
"@angular/service-worker": "^21.2.9",
|
||||
"@fortawesome/angular-fontawesome": "~4.0.0",
|
||||
"@fortawesome/fontawesome-svg-core": "^7.2.0",
|
||||
"@fortawesome/free-brands-svg-icons": "^7.2.0",
|
||||
@@ -50,8 +50,8 @@
|
||||
"@angular-eslint/builder": "21.1.0",
|
||||
"@angular/build": "^21.2.7",
|
||||
"@angular/cli": "^21.2.7",
|
||||
"@angular/compiler-cli": "^21.2.8",
|
||||
"@angular/localize": "^21.2.8",
|
||||
"@angular/compiler-cli": "^21.2.9",
|
||||
"@angular/localize": "^21.2.9",
|
||||
"@eslint/js": "^9.39.4",
|
||||
"angular-eslint": "21.1.0",
|
||||
"eslint": "^9.39.4",
|
||||
|
||||
Generated
+342
-342
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,6 @@
|
||||
allowBuilds:
|
||||
'@parcel/watcher': true
|
||||
core-js: true
|
||||
esbuild: true
|
||||
lmdb: true
|
||||
msgpackr-extract: true
|
||||
@@ -324,15 +324,15 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "deno"
|
||||
version = "2.7.11"
|
||||
version = "2.7.12"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d7/3f/a0477c72b847c0082ceb8885261eb14fb4addde22e8fb0146c011636979b/deno-2.7.11.tar.gz", hash = "sha256:342a656fca446fadc261ed22af35693b6c34e79129fa2bd387a1e5d39f496a99", size = 8167, upload-time = "2026-04-01T12:48:16.673Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/5c/ac/39b45b51e7aaf9cad1de36f91bd58677818a7d7a867fa0721faff43b67c4/deno-2.7.12.tar.gz", hash = "sha256:7ef413693b4a9d86837a6f4991738429908f6f42f6f3ed85254ab0e679438049", size = 8167, upload-time = "2026-04-09T20:39:50.505Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d1/64/c8cb5a9c50135ada59b412b7511852d551d2618e169f48a8e7b8e90a382a/deno-2.7.11-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:76aa633656b07f64cfda3aa94a1e16bc123034ba2fe676d23b179f8326728534", size = 47857669, upload-time = "2026-04-01T12:48:01.947Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/27/f3/250216e71e21cfc291ff6eb6503a479410ff927c9f97ebb644463c620692/deno-2.7.11-py3-none-macosx_11_0_arm64.whl", hash = "sha256:a1dd1ceb0c2109b54334a621503e5297d473f5c3e6702992dfa68d9b7cdf1177", size = 44607869, upload-time = "2026-04-01T12:48:05.23Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/01/51/b8fcb7d6882d659abd678f48fc5bc89aa0aa10f5a399fca3823680f277ca/deno-2.7.11-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:56578808c891d1daeadc279ac73445a24a83c2c721b37016fd92b1aa9224979b", size = 48394760, upload-time = "2026-04-01T12:48:08.049Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a0/18/91d1ceb2e15b4446fcecdf4be17ed59b02f5b35d8c98c94e7eaadf10591e/deno-2.7.11-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:58eea29d18fe8f80f2a4354d96225a31961ed3fa68336a7fcdd6add331df6acd", size = 50418090, upload-time = "2026-04-01T12:48:10.84Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b6/8d/7a801d85a0a8233d00ea1a381fd32e76c81be9db78ada162e211e7619221/deno-2.7.11-py3-none-win_amd64.whl", hash = "sha256:e2b69676e52543153b82e926f558983f9251db2251474f3225301cf7aca05a2b", size = 49417475, upload-time = "2026-04-01T12:48:14.158Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/77/f0/0be717182294e9b61e58cd89b88d1b711c7e9c134e53890474b8d1e95704/deno-2.7.12-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:74677d1be23426b1f16ce33c1a96868bc7554e1d41a12fb4d446017bb1955d1d", size = 47683364, upload-time = "2026-04-09T20:39:35.086Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8d/44/ca1de5368b193c9819c57d505ff8ceb86dc38e2e87856b23c23b58abd234/deno-2.7.12-py3-none-macosx_11_0_arm64.whl", hash = "sha256:acf94fa815ac84a948b196f53a7e088ed126ec2e390a25d92cdfdc8c5d8a851e", size = 44460697, upload-time = "2026-04-09T20:39:38.5Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ce/68/ce111604f5877220e53fb10c5180ffaa484f829f20fcbc7f7115a90afadd/deno-2.7.12-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:ab7c12ee0e10cd232dbb8f7cf5802bfdbcc3462c0b1453726b158d8a27d4185f", size = 48237362, upload-time = "2026-04-09T20:39:41.555Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d7/07/d8ba3284e098128076cd9f84e85d2e62d12fa1e5b8e5c47681983017af1a/deno-2.7.12-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:b396675fef296530c386387990d26d118398452bac47541e739b353b9ff8809f", size = 50247653, upload-time = "2026-04-09T20:39:44.956Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/07/b0/c6c215eca2aa54b700a95867901832245518b2f217edfc78e733401bf8b3/deno-2.7.12-py3-none-win_amd64.whl", hash = "sha256:ddeeb03427d7ce0979a395ae2d1aa5ebcddeb97ab0fd9e7dfa56a08e0e5b817c", size = 49200604, upload-time = "2026-04-09T20:39:48.218Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -593,11 +593,11 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "packaging"
|
||||
version = "26.0"
|
||||
version = "26.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/65/ee/299d360cdc32edc7d2cf530f3accf79c4fca01e96ffc950d8a52213bd8e4/packaging-26.0.tar.gz", hash = "sha256:00243ae351a257117b6a241061796684b084ed1c516a08c48a3f7e147a9d80b4", size = 143416, upload-time = "2026-01-21T20:50:39.064Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/df/de/0d2b39fb4af88a0258f3bac87dfcbb48e73fbdea4a2ed0e2213f9a4c2f9a/packaging-26.1.tar.gz", hash = "sha256:f042152b681c4bfac5cae2742a55e103d27ab2ec0f3d88037136b6bfe7c9c5de", size = 215519, upload-time = "2026-04-14T21:12:49.362Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b7/b9/c538f279a4e237a006a2c98387d081e9eb060d203d8ed34467cc0f0b9b53/packaging-26.0-py3-none-any.whl", hash = "sha256:b36f1fef9334a5588b4166f8bcd26a14e521f2b55e6b9de3aaa80d3ff7a37529", size = 74366, upload-time = "2026-01-21T20:50:37.788Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7a/c2/920ef838e2f0028c8262f16101ec09ebd5969864e5a64c4c05fad0617c56/packaging-26.1-py3-none-any.whl", hash = "sha256:5d9c0669c6285e491e0ced2eee587eaf67b670d94a19e94e3984a481aba6802f", size = 95831, upload-time = "2026-04-14T21:12:47.56Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||
Reference in New Issue
Block a user