Vigil
← All news

v1.5.7 — wiki-scraped ability durations feeding M-BURST

- **Schema**: new nullable `abilities.duration_ms` column. Alembic migration `fbbd9a93c108_abilities_duration_ms`. 16 tables in dev DB. - **`ingest/wiki.py`**: small scraper for F…

Added — abilities.duration_ms + wiki scraper

  • Schema: new nullable abilities.duration_ms column. Alembic migration fbbd9a93c108_abilities_duration_ms. 16 tables in dev DB.
  • ingest/wiki.py: small scraper for FFXIV consolegameswiki.com.
    • extract_duration_ms(html) — tag-strips the HTML and grabs the first Duration: <N>s (handles Duration:</span>&nbsp;20s and &nbsp; / &#160; variants). The first occurrence is the primary buff duration; secondary trait-granted effects (e.g. Divination's "Divining" 30s extension) are correctly ignored.
    • fetch_duration_for_ability(client, name) — fetches one wiki page via httpx, returns duration in ms or None (on 404 / network failure / missing field).
    • scrape_durations_for_abilities(client, abilities, *, pacing_s=0.5) — batch fetch with polite pacing between requests.
    • No new dependencies — uses existing httpx + stdlib re + urllib.
  • scripts/scrape_ability_durations.py: one-shot runner. Pulls every ability labeled raid_buff / personal_buff / mit_party / mit_self / mit_boss_debuff, fetches each wiki page, writes duration_ms back. Flags: --force, --label X (repeatable), --limit N, --pacing-s N. Idempotent (--force to refresh).
  • M-BURST integration (analysis/burst.py): new duration_ms_for_abilities() helper. The window-construction step now reads each raid-buff cast's (ts, ability_id) and sizes the window from duration_ms when present, falling back to the 20s default. Per-cast window length, not per-fight — Mage's Ballad's 45s song window and a Divination 20s window can coexist correctly.

Live AC

  • Ran python -m scripts.scrape_ability_durations --label raid_buff --limit 10. All 10 extracted real values: Mage's Ballad 45s (bard song — the wiki captures this where XIVAPI doesn't), Divination 20s, Radiant Finale 20s, Starry Muse 20s (Pictomancer), the Balance / the Spear 15s (AST cards). Status-row equivalents picked up the same values via the existing name-matching pass.

Tests

  • 10 new tests in tests/test_wiki.py via httpx.MockTransport: primary-duration-wins on Divination-style multi-Duration page, span-split parsing on Brotherhood-style page, no-duration returns None, HTML entity handling, end-to-end mock fetch, 404 returns None, network-error swallowed, empty name returns None, URL encoding of spaces, batch scrape with no-pacing.
  • 2 new integration tests in tests/test_burst.py: per-ability duration_ms override produces a 15s window instead of 20s default; mixed durations (one ability with override, one without) produce disjoint windows of correct lengths.
  • 378 tests passing (366 → 378).