2026-06-27 21:46:33 +01:00
|
|
|
"""Provider detection tests — build_chat_url / build_models_url routing (re: #768).
|
Provider detection: match by hostname instead of substring (re #768) (#815)
* Dedupe URL routing helpers and tighten adjacent hostname checks
* Match providers by hostname, not substring, in _detect_provider
_detect_provider used `"anthropic.com" in url`-style substring checks, so a URL
that merely contained a provider's domain in its path or query — or a look-alike
host like `anthropic.com.example` — was misclassified and picked the wrong
auth-header/payload shape. Switch it to the existing `_host_match` helper
(hostname exact/subdomain match), the same way the human-readable labels and
curated model lists already work, finishing that migration. Also harden
`_host_match` against trailing-dot FQDNs.
Not a credential-leak fix: _detect_provider only classifies a URL the admin
already configured next to its key, and the URL — not this function — decides
where the request goes. This is a correctness/consistency cleanup.
Adds tests that import the real helpers (test_endpoint_resolver.py tests local
copies, so it can't catch this) covering the substring false-positives.
Refs #768.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Import build_headers under its real name in model_routes
It was imported as `build_headers as _provider_headers`, which collides with
the unrelated llm_core._provider_headers(provider, headers) — same name,
different signature. Use the real name to remove the confusion.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Use hostname matching in URL builders, not raw suffix checks
PR review flagged that _detect_provider() was hardened to match on
hostname, but several helpers still used raw host.endswith("anthropic.com")
/ host.endswith("ollama.com"), which match adjacent hosts like
notanthropic.com / notollama.com.
Route the remaining checks through _host_match(): _is_ollama_native_url
and _ollama_api_root in llm_core, and _anthropic_api_root / _ollama_api_root
in endpoint_resolver. With _detect_provider already hostname-correct, the
trailing "or host.endswith(...)" clauses in build_chat_url / build_models_url
are redundant, so drop them rather than fix the substring match in place.
Add builder-level tests asserting look-alike and domain-in-path hosts route
to the OpenAI-compatible default. They import the real builders and fail on
the pre-fix code.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 19:11:17 -07:00
|
|
|
|
|
|
|
|
These import the *real* helpers from ``src.llm_core`` (not local copies) so a
|
|
|
|
|
regression in hostname matching is actually caught. The point of the change
|
|
|
|
|
under test is that provider detection keys off the URL's *hostname*, not a
|
|
|
|
|
substring of the whole URL — so a domain appearing in a path/query, or a
|
|
|
|
|
look-alike host, must not be misclassified.
|
|
|
|
|
"""
|
|
|
|
|
import pytest
|
|
|
|
|
|
|
|
|
|
from src import llm_core
|
|
|
|
|
from src import endpoint_resolver
|
|
|
|
|
from src.endpoint_resolver import build_chat_url, build_models_url
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class TestBuildersRejectLookalikeHosts:
|
|
|
|
|
"""build_chat_url / build_models_url must route look-alike and
|
|
|
|
|
domain-in-path hosts to the OpenAI-compatible default, not the
|
|
|
|
|
anthropic/ollama branches. Before #815's follow-up these builders still
|
|
|
|
|
fell back to ``host.endswith("anthropic.com")`` style checks, so
|
|
|
|
|
``notanthropic.com`` was misrouted to the Anthropic messages endpoint.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
|
|
|
def _stub_dns(self, monkeypatch):
|
|
|
|
|
# build_* call resolve_url(), which does real DNS + tailscale lookups.
|
|
|
|
|
# Provider routing is independent of name resolution, so stub it out to
|
|
|
|
|
# keep these deterministic and offline.
|
|
|
|
|
monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda u: u)
|
|
|
|
|
|
|
|
|
|
def test_real_anthropic_chat(self):
|
|
|
|
|
assert build_chat_url("https://api.anthropic.com") == "https://api.anthropic.com/v1/messages"
|
|
|
|
|
|
feat: add ChatGPT Subscription provider (#2876)
* feat: Add ChatGPT Subscription support and related features
- Introduced a new provider option for ChatGPT Subscription in the endpoint selection UI.
- Implemented OAuth flow for ChatGPT Subscription sign-in, including polling for authorization status.
- Updated admin interface to handle ChatGPT Subscription, including disabling API key input and providing user guidance.
- Enhanced cost tracking logic to differentiate between subscription and non-subscription endpoints.
- Added new slash commands for managing skills, including listing, searching, and invoking skills.
- Implemented caching for skill catalog to optimize performance.
- Updated tests to cover new ChatGPT Subscription functionality and ensure proper endpoint probing.
- Refactored existing code to accommodate new features and improve maintainability.
* refactor: share provider device-flow setup
- reuse one device-flow backend for Copilot and ChatGPT Subscription
- add one frontend device-flow helper for Settings and /setup
- put GitHub Copilot back into Add Models, now as a dropdown option
- make provider selection just select; clicking Add starts sign-in
- stop ChatGPT Subscription setup from opening auth tabs automatically
- make /setup copilot and /setup chatgpt-subscription work from chat
- show ChatGPT Subscription in the /setup suggestions
- show the real error message when setup fails
- add focused tests for the shared flow and setup UI
* feat(chatgpt-subscription): harden credential lifecycle and streamline auth UX
Backend:
- Resolve runtime bearer for provider-auth endpoints at probe time via a
shared _resolve_probe_key() that delegates to resolve_endpoint_runtime,
applied across all probe/refresh call sites.
- Skip live completion probes and health pings for discovery-only providers
(centralized behind _is_discovery_only_provider) — the Codex/Responses API
has no such endpoints, so status is derived from cached models.
- Never persist the short lived ChatGPT bearer to the plaintext sessions
table; proactively clear any stale bearer left by an earlier code path.
- Revoke orphaned ProviderAuthSession credentials when the last endpoint
backing them is deleted (_delete_orphaned_provider_auth), surfaced via
cleared_provider_auth in the delete response.
Frontend (admin.js):
- Auto-start the device-auth flow on provider selection so the authorization
panel (code + Authorize) shows immediately instead of behind a "Sign in" click.
- Remove the redundant top button for device auth providers, move retry
into the panel via an inline "Try again".
- Drop the self-evident hint text and add an execCommand clipboard fallback so
Copy works in non-secure (HTTP/LAN) contexts.
* fix: harden chatgpt subscription provider
* chore: remove PR media from branch
* Fix chatgpt subscription recovery and token handling
---------
Co-authored-by: 5p00kyy <admin@5p00ky.dev>
2026-06-08 18:19:18 +10:00
|
|
|
def test_chatgpt_subscription_chat_uses_responses(self):
|
|
|
|
|
assert build_chat_url("https://chatgpt.com/backend-api/codex") == "https://chatgpt.com/backend-api/codex/responses"
|
|
|
|
|
|
|
|
|
|
def test_chatgpt_subscription_models_uses_no_live_probe(self):
|
|
|
|
|
assert build_models_url("https://chatgpt.com/backend-api/codex") is None
|
|
|
|
|
|
Provider detection: match by hostname instead of substring (re #768) (#815)
* Dedupe URL routing helpers and tighten adjacent hostname checks
* Match providers by hostname, not substring, in _detect_provider
_detect_provider used `"anthropic.com" in url`-style substring checks, so a URL
that merely contained a provider's domain in its path or query — or a look-alike
host like `anthropic.com.example` — was misclassified and picked the wrong
auth-header/payload shape. Switch it to the existing `_host_match` helper
(hostname exact/subdomain match), the same way the human-readable labels and
curated model lists already work, finishing that migration. Also harden
`_host_match` against trailing-dot FQDNs.
Not a credential-leak fix: _detect_provider only classifies a URL the admin
already configured next to its key, and the URL — not this function — decides
where the request goes. This is a correctness/consistency cleanup.
Adds tests that import the real helpers (test_endpoint_resolver.py tests local
copies, so it can't catch this) covering the substring false-positives.
Refs #768.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Import build_headers under its real name in model_routes
It was imported as `build_headers as _provider_headers`, which collides with
the unrelated llm_core._provider_headers(provider, headers) — same name,
different signature. Use the real name to remove the confusion.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Use hostname matching in URL builders, not raw suffix checks
PR review flagged that _detect_provider() was hardened to match on
hostname, but several helpers still used raw host.endswith("anthropic.com")
/ host.endswith("ollama.com"), which match adjacent hosts like
notanthropic.com / notollama.com.
Route the remaining checks through _host_match(): _is_ollama_native_url
and _ollama_api_root in llm_core, and _anthropic_api_root / _ollama_api_root
in endpoint_resolver. With _detect_provider already hostname-correct, the
trailing "or host.endswith(...)" clauses in build_chat_url / build_models_url
are redundant, so drop them rather than fix the substring match in place.
Add builder-level tests asserting look-alike and domain-in-path hosts route
to the OpenAI-compatible default. They import the real builders and fail on
the pre-fix code.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 19:11:17 -07:00
|
|
|
def test_lookalike_anthropic_chat_is_openai(self):
|
|
|
|
|
assert build_chat_url("https://notanthropic.com") == "https://notanthropic.com/chat/completions"
|
|
|
|
|
|
|
|
|
|
def test_lookalike_anthropic_models_is_openai(self):
|
2026-06-15 18:13:33 +05:30
|
|
|
assert llm_core._detect_provider("https://anthropic.com.evil.com") == "openai"
|
2026-06-15 23:21:49 +09:00
|
|
|
assert build_models_url("https://anthropic.com.evil.com") == "https://anthropic.com.evil.com/models"
|
Provider detection: match by hostname instead of substring (re #768) (#815)
* Dedupe URL routing helpers and tighten adjacent hostname checks
* Match providers by hostname, not substring, in _detect_provider
_detect_provider used `"anthropic.com" in url`-style substring checks, so a URL
that merely contained a provider's domain in its path or query — or a look-alike
host like `anthropic.com.example` — was misclassified and picked the wrong
auth-header/payload shape. Switch it to the existing `_host_match` helper
(hostname exact/subdomain match), the same way the human-readable labels and
curated model lists already work, finishing that migration. Also harden
`_host_match` against trailing-dot FQDNs.
Not a credential-leak fix: _detect_provider only classifies a URL the admin
already configured next to its key, and the URL — not this function — decides
where the request goes. This is a correctness/consistency cleanup.
Adds tests that import the real helpers (test_endpoint_resolver.py tests local
copies, so it can't catch this) covering the substring false-positives.
Refs #768.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Import build_headers under its real name in model_routes
It was imported as `build_headers as _provider_headers`, which collides with
the unrelated llm_core._provider_headers(provider, headers) — same name,
different signature. Use the real name to remove the confusion.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Use hostname matching in URL builders, not raw suffix checks
PR review flagged that _detect_provider() was hardened to match on
hostname, but several helpers still used raw host.endswith("anthropic.com")
/ host.endswith("ollama.com"), which match adjacent hosts like
notanthropic.com / notollama.com.
Route the remaining checks through _host_match(): _is_ollama_native_url
and _ollama_api_root in llm_core, and _anthropic_api_root / _ollama_api_root
in endpoint_resolver. With _detect_provider already hostname-correct, the
trailing "or host.endswith(...)" clauses in build_chat_url / build_models_url
are redundant, so drop them rather than fix the substring match in place.
Add builder-level tests asserting look-alike and domain-in-path hosts route
to the OpenAI-compatible default. They import the real builders and fail on
the pre-fix code.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 19:11:17 -07:00
|
|
|
|
|
|
|
|
def test_anthropic_domain_in_path_is_openai(self):
|
|
|
|
|
assert build_chat_url("https://myproxy.internal/anthropic.com/v1") == "https://myproxy.internal/anthropic.com/v1/chat/completions"
|
|
|
|
|
|
|
|
|
|
def test_real_ollama_chat(self):
|
|
|
|
|
assert build_chat_url("https://ollama.com") == "https://ollama.com/api/chat"
|
|
|
|
|
|
|
|
|
|
def test_lookalike_ollama_chat_is_openai(self):
|
|
|
|
|
assert build_chat_url("https://notollama.com") == "https://notollama.com/chat/completions"
|
|
|
|
|
|
|
|
|
|
def test_lookalike_ollama_models_is_openai(self):
|
2026-06-15 18:13:33 +05:30
|
|
|
assert llm_core._detect_provider("https://notollama.com") == "openai"
|
2026-06-15 23:21:49 +09:00
|
|
|
assert build_models_url("https://notollama.com") == "https://notollama.com/models"
|
Provider detection: match by hostname instead of substring (re #768) (#815)
* Dedupe URL routing helpers and tighten adjacent hostname checks
* Match providers by hostname, not substring, in _detect_provider
_detect_provider used `"anthropic.com" in url`-style substring checks, so a URL
that merely contained a provider's domain in its path or query — or a look-alike
host like `anthropic.com.example` — was misclassified and picked the wrong
auth-header/payload shape. Switch it to the existing `_host_match` helper
(hostname exact/subdomain match), the same way the human-readable labels and
curated model lists already work, finishing that migration. Also harden
`_host_match` against trailing-dot FQDNs.
Not a credential-leak fix: _detect_provider only classifies a URL the admin
already configured next to its key, and the URL — not this function — decides
where the request goes. This is a correctness/consistency cleanup.
Adds tests that import the real helpers (test_endpoint_resolver.py tests local
copies, so it can't catch this) covering the substring false-positives.
Refs #768.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Import build_headers under its real name in model_routes
It was imported as `build_headers as _provider_headers`, which collides with
the unrelated llm_core._provider_headers(provider, headers) — same name,
different signature. Use the real name to remove the confusion.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Use hostname matching in URL builders, not raw suffix checks
PR review flagged that _detect_provider() was hardened to match on
hostname, but several helpers still used raw host.endswith("anthropic.com")
/ host.endswith("ollama.com"), which match adjacent hosts like
notanthropic.com / notollama.com.
Route the remaining checks through _host_match(): _is_ollama_native_url
and _ollama_api_root in llm_core, and _anthropic_api_root / _ollama_api_root
in endpoint_resolver. With _detect_provider already hostname-correct, the
trailing "or host.endswith(...)" clauses in build_chat_url / build_models_url
are redundant, so drop them rather than fix the substring match in place.
Add builder-level tests asserting look-alike and domain-in-path hosts route
to the OpenAI-compatible default. They import the real builders and fail on
the pre-fix code.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 19:11:17 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
class TestBuildersLocalAndDockerEndpoints:
|
|
|
|
|
"""Local and docker endpoints must keep working after the hostname change:
|
|
|
|
|
a local ``/v1`` base stays OpenAI-compatible, and a native Ollama ``/api``
|
|
|
|
|
path is still detected by path even on a non-ollama.com host such as
|
|
|
|
|
host.docker.internal.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
|
|
|
def _stub_dns(self, monkeypatch):
|
|
|
|
|
monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda u: u)
|
|
|
|
|
|
|
|
|
|
def test_local_v1_chat_is_openai_compatible(self):
|
|
|
|
|
assert build_chat_url("http://localhost:8000/v1") == "http://localhost:8000/v1/chat/completions"
|
|
|
|
|
|
|
|
|
|
def test_local_v1_models_is_openai_compatible(self):
|
|
|
|
|
assert build_models_url("http://127.0.0.1:1234/v1") == "http://127.0.0.1:1234/v1/models"
|
|
|
|
|
|
|
|
|
|
def test_docker_internal_ollama_api_path_is_native_chat(self):
|
|
|
|
|
assert build_chat_url("http://host.docker.internal:11434/api") == "http://host.docker.internal:11434/api/chat"
|
|
|
|
|
|
|
|
|
|
def test_docker_internal_ollama_api_path_is_native_models(self):
|
|
|
|
|
assert build_models_url("http://host.docker.internal:11434/api") == "http://host.docker.internal:11434/api/tags"
|