2026-09-11 22:17:13 +02:00
|
|
|
$ANSIBLE_VAULT;1.1;AES256
|
alerting: Signal via signal-cli-rest-api, and faster failure detection
Three changes: detection windows tightened, a Signal transport deployed, and
alerts attached to all 84 non-transport endpoints.
── Failing faster ──────────────────────────────────────────────────────────
The heartbeat is a TICKER, not a deadline: Gatus wakes every interval and asks
"did anything arrive in the last interval", so real detection is 1-2x the
window. And a window can only ever be as tight as the push frequency - which is
why two checks moved rather than just having their numbers changed.
liveness, cpu, ups, service-health, probes 16m -> 11m
disk, zfs daily/30h -> 6-hourly/7h
backup store + pull job daily/30h -> 6-hourly/7h
DNS records 24h -> 6h
backup dump 30h -> 26h
backup-dump stays slow because the dump genuinely is daily. The store-side check
catches the same fault within 6h by reading the source's dump timestamp out of
the artefact filename, so 26h is a backstop rather than the primary signal.
── Thresholds differ by check type, deliberately ───────────────────────────
`failure-threshold` counts consecutive failures, but "consecutive" is a
different amount of wall-clock time per check: a push endpoint produces one
failure per heartbeat window, a pulled one per interval. The default of 3 would
mean 33 minutes on an 11m heartbeat and over a day on a 7h one.
More importantly the heartbeat window ALREADY encodes the tolerance - an 11m
window on a 5-minute push is exactly "one missed push forgiven" - so stacking a
threshold of 3 triples a tolerance that was already chosen. Hence:
push/heartbeat endpoints failure-threshold 1
pulled, 5m (public) failure-threshold 3 (= 15 minutes)
pulled, 6h/24h (dns, domain) failure-threshold 1
── The Signal transport ────────────────────────────────────────────────────
roles/signal_api runs signal-cli-rest-api on the observability host, pinned by
digest, MODE=native.
It publishes NO PORTS. The API has no authentication of any kind - anything that
reaches it can send messages as you and read your Signal. Gatus talks to it over
a shared docker network by service name, which is also WHY the network exists:
Gatus runs in a container, so the host's loopback is unreachable from it and a
port published on 127.0.0.1 would not have worked.
MODE=native and not json-rpc because this VPS has 464MB of RAM and already runs
Gatus and Caddy. The json-rpc modes hold a resident JVM; native runs a binary
per request, and alerts are rare enough that startup cost per alert is the right
trade.
Monitored - Gatus polls /v1/health over the same network path the alerts take,
so it proves the delivery route rather than mere container liveness. Deliberately
NOT backed up: the data directory holds Signal private keys and the recovery
path is to link the device again from the phone.
Neither the signal-api endpoint nor Gatus's self-check carries a Signal alert.
If either is down, Signal is precisely what cannot deliver the alert.
── Four traps hit while linking, all now in the role README ────────────────
* /v1/qrcodelink is BROKEN in native mode - returns "no data to encode" while
the binary itself emits a perfectly good URI. Not worked around by switching
MODE, which would put a JVM in the path of every alert permanently.
* The data dir must be owned by uid 1000, not root. A root-owned 0700 dir
cannot be traversed by the container user, so linking silently never
completes and /v1/accounts returns "Failed to read local accounts list".
* `docker exec` runs as ROOT while the service runs as uid 1000, so without
--config the account is written to /root/... on the container's ephemeral
layer. It reports success and is destroyed on the next recreate.
* The phone reporting "network error" was IPv6: chat.signal.org resolves to
dualstack AAAA records first, the container has no IPv6 address, and this
host's IPv6 path is broken - the same edge that 404'd the Go tarball. Fixed
with a mounted gai.conf that prefers IPv4.
Verified: provider loads (configuredProviders=[signal]), a test message was
delivered and confirmed received, 84 endpoints carry alerts, 86 UP / 0 DOWN.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 21:40:37 +02:00
|
|
|
34383033613438623432656438313763613639313139626432303637383363643339613166626665
|
|
|
|
|
6537663336353661373437303636613364643736663031300a623461353135623537343763383333
|
|
|
|
|
37323731646262613436646231656263396532616132316237353762376363383062646132303435
|
|
|
|
|
3435316666316235660a323163613932616661303935336633656332353766393536393736356539
|
|
|
|
|
61396164386266303464373262373862363361363365363235383363613335663933313062663565
|
|
|
|
|
39613466353161356462656539613335613536363532333431393935303430373435373737363537
|
|
|
|
|
62383266323339373062313134316464303263313830646161333530383736336637333865373764
|
|
|
|
|
31313165396366653763343466643561383236643836616662383439666165343139326232326364
|
|
|
|
|
32633133373863616163343365656231353939366435346534383462396134653064663566343566
|
|
|
|
|
66343464373865383864313264646635316233336133346365396634626261613561373965643339
|
|
|
|
|
30353831393037316334313632376537393361336366326561343832373531303139323937613531
|
|
|
|
|
34386164303430316162363536386137316330336638303365393462386465633862663932376563
|
|
|
|
|
34323862636435306563333065383134343362333733623066656439613339376666353331373665
|
|
|
|
|
32353938663364313036373237323062393430306661633732356434333232376338313835663330
|
|
|
|
|
32306536393666323464663238306562613936376236613935616664653865373330626365626330
|
|
|
|
|
64613066653632646638306338393331376430383165316363323437373362613866313962663432
|
|
|
|
|
62356663636630356239613831353066303530363966396135376134633936313530643262353965
|
|
|
|
|
38383330613962313935366633353635363638303937316362303866343433353437363334396431
|
|
|
|
|
32393134626133353564643233616632376633303934653065613262353436333533376137303862
|
|
|
|
|
35303861383833326537376433336634633730373833313036643365373431666438653033653930
|
|
|
|
|
33346230396661636333613031353133353631306331666362386366633062666534303539316238
|
|
|
|
|
39366634656333646634643032393738333330636230663366623265666533356534623465316665
|
|
|
|
|
35336535326237343231653661613736626239316262386363666264613636326332366232333964
|
|
|
|
|
38353635616662663133376534306366643037613732343233373336653166316330643638353539
|
|
|
|
|
35353532643534613864333839393365313439343663333337373639633832383630316164306132
|
|
|
|
|
63636162373034363839656638353534613733663737626432636436373164303239316436313566
|
|
|
|
|
66623466396663353136323565643262383865323830616232373466373431373261306537646463
|
|
|
|
|
61353966623062356438626635663461396165343366393132393965383734633632666335313030
|
|
|
|
|
37333631373239303231396264393031373162373936303462363736313538343562343737356332
|
|
|
|
|
63663762633032356130303964386664366137373432353533386566613037613163383963623732
|
|
|
|
|
32343765376535343630656633333362393765666131653632613361363163383038323138333537
|
|
|
|
|
37373333316466316562613439356365316163336366623435666166653835616531653563343664
|
|
|
|
|
30303464663838646333373837363961356465613234636138633162366636633030383337643131
|
|
|
|
|
62636236663066316630396132333934353139383465363034363033656161383663656462313939
|
|
|
|
|
62353738396132343734343363636164323163616237613861643862303433633934656165343263
|
|
|
|
|
35313737373333386335663731643664393630653839396464376639306231653934386336353630
|
|
|
|
|
61393735336662663435613431386361373561363531643232303831363163636139653538393931
|
|
|
|
|
66313965646562616237383439343637623835303065333730613865666638383131616261346463
|
|
|
|
|
31396464306537613135653830653138323539393731626264326335336432666333333735646534
|
|
|
|
|
30616439393464326631633635363466303336613135346231626232313361303666323661616638
|
|
|
|
|
64363937343231363536353034363966326132333734386638613737636130646232363666643633
|
|
|
|
|
61353563636466356630346232613761636432303430336461663434333636623962643764336639
|
|
|
|
|
35303565613861323431393036303534303061666437366538306236373930313439353330313632
|
|
|
|
|
61363363303562316664303065613863613339626632643931386438326330373938613762303334
|
|
|
|
|
33316430363931393262373661623137633835656136313235613666613932313236343966306331
|
|
|
|
|
36343031303632323432366337633637336335343638393564313738386162386164613339343662
|
|
|
|
|
62383939663265373265613932633265626263623939303638383838646531343433393864363235
|
|
|
|
|
37313761396436623833376435393537376136373162393465343764326533666533393061643965
|
|
|
|
|
61306365346562333737313962633764313232323161623861336566343735353737633539656162
|
|
|
|
|
34323939306661373662663331653333346264323930663636633134666532623438323537383165
|
|
|
|
|
66343033363962633766303331623866306462623235373838616565653066646132633034363535
|
|
|
|
|
31613462356361613535383963643662373162363363653334393937313039666266366363653537
|
|
|
|
|
64356166396139393262313565303731346534646462333638316661383139346362333364613466
|
|
|
|
|
38303561316263636136303431393366343936653161336238613439366266346136636465303337
|
|
|
|
|
61373634376464333037303862616335623031656133636165636264386265643261373735663266
|
|
|
|
|
39396265366138363430343035356337303438346165366361316230656239326633653537626339
|
|
|
|
|
39613334316563376133343336343563363564653237633764393532336437303334363830396635
|
|
|
|
|
31333963666462313830333163646465356337643263636462356363613266323630393866626535
|
|
|
|
|
35336436663063333364353663616564626535396133386461326433323936323161613633353539
|
|
|
|
|
66323935626433353162666335613839346561343264323763303034613366663233613037343538
|
|
|
|
|
66376236373462383430313030373235373337653338333033306431616562353435656433356365
|
|
|
|
|
37653063363435366532306334373438306632306435653334396630343863623938666335656164
|
|
|
|
|
37386634366430383762316435396237313362313965353634613266616532633465646464313662
|
|
|
|
|
35303433363236393465396161346431306231666432666531626136623966663864376663373562
|
|
|
|
|
37653239633732346536323265643434616136326561666330633435383264333937353366653736
|
|
|
|
|
61353430633363323263323163646639663330663632343038623964396433626337353232373334
|
|
|
|
|
32343938356539303938666137323831646562393233323733336464316165623232326132316362
|
|
|
|
|
61313631626234336164656533616231393663363364623130313439623466396231383537656562
|
|
|
|
|
35393163323131383838356130643863633736613635306635306135666261623731613239303962
|
|
|
|
|
61373432336638353339646665653561653738353731373032366361313562306466373365653333
|
|
|
|
|
35306236336331333961613864346165613763383266613236343066623432303766333434333331
|
|
|
|
|
66376263363132616132366165393235626336323665373135653462653533376138346632393363
|
|
|
|
|
35396165383966326635333134663138366136336430393532373935373264616530613762386430
|
|
|
|
|
37643862316232613261343430616438333831623835663239656531356666613032653837373531
|
|
|
|
|
30386232643537306530323936386561646265616339376265353763663833623539653831656634
|
|
|
|
|
63626564633964343663656435636635656562666236386639353332356566316631643431353330
|
|
|
|
|
32643238623932643732353332633535643738663066383830376432346463393464333236386163
|
|
|
|
|
64376165333266323166613564313061643832656239333165393035343836376434626162643061
|
|
|
|
|
31373861393534636236383765313834356638613332626666653762343537363339383939666263
|
|
|
|
|
61366230623930643534353565316165656132396664393138326139613536343738396330653735
|
|
|
|
|
37623463343834626133313231393130646134363331393238663065643930383237326439343831
|
|
|
|
|
32393536613361616637373761393361303936363863333838666633343063386235316237323531
|
|
|
|
|
62633362316436623761616366383336613765373362393161303765613464643632666462343565
|
|
|
|
|
63616430396164346263666436336463623638313130656231643131336663336565616135373433
|
|
|
|
|
39326533643733333564373434333738343634383033353036646534646632383730613161393130
|
|
|
|
|
62653836376435663435353739373536366466646332653030653332353761626466353631663163
|
|
|
|
|
38613734383634663662376464653733306564313761336566336336656366653536643834653631
|
|
|
|
|
35333963306562393234623039613338643266333762333236366331333431343830393864623830
|
|
|
|
|
34323439386133613035363861623332663933313734313637623739613163663730316134303434
|
|
|
|
|
35363239363836666463366334356435666632396132306163303037663430383535393934323537
|
|
|
|
|
30353266373864346662333035393761646632313032383738346439343232663666343833333565
|
|
|
|
|
35363564623038376666363436663830356165323234653230636264303066666230643632356237
|
|
|
|
|
66633734346633366533623966393462313438333965623033343337323135353730613561363236
|
|
|
|
|
30636333613034626162393137323762306432663761393766303163393536656338366664643432
|
|
|
|
|
34636630323965396166356538623165663335333739323034333237306531393734306138343436
|
|
|
|
|
36613937373932633234303336373530663533646635396664343935326334333239646534343336
|
|
|
|
|
35396666353938326337646431353965653836323731313861613031356561376262363965623934
|
|
|
|
|
65336164623134366365303062393161353965363937646363396564313138323938656330333964
|
|
|
|
|
64663862363766613630333531623463616331363533343962653164376665303463336134346662
|
|
|
|
|
66376238663632663932656230323638323135653666363736643065336330636236323064386361
|
|
|
|
|
37646631303138616565383665663634333338613336346438366333636165366635313337373635
|
|
|
|
|
62656236653961393061373262326534393339363236353336613339656462336661663739396261
|
|
|
|
|
35363636653237316436656662633462306333313336313865333037636661383731336466666536
|
|
|
|
|
36646537653431666261353232653233653930346538383438633466346636656432383963333765
|
|
|
|
|
62346363633265306665366630383831303434336139343837616361666539656262633735363064
|
|
|
|
|
30376138386531336430613563366666333964663231363465326230316233323532366632663930
|
|
|
|
|
63653164373931353861653833326130643365373530663039363166366331303134366336613438
|
|
|
|
|
63633336363539636265396331653630633162316366316130326531363463616464376635376538
|
|
|
|
|
32376638353337343639633434613663666437313564333264383766623561393933333561663235
|
|
|
|
|
37666163626538353736383435373530663438363636643665626266316338336532623235353337
|
|
|
|
|
61653762383630616266353438386137653137616532633066623165383663646437343565366237
|
|
|
|
|
33333463613539346136613135386466656135616136663036313661323738363066633038656139
|
|
|
|
|
38353230313861666463383737303038613731316334333031643438343463666164633536313031
|
|
|
|
|
39613663376436643165333362636462303230656336333234363336633262336363643736336231
|
|
|
|
|
66316630613762383832613836333263303038306235663662323162633932653733626130356665
|
|
|
|
|
65316630346534356261373961353130393334396365376237353439326261336339393336303030
|
|
|
|
|
63323439666439313831626639323735653730356565643938376564316337613333616664643865
|
|
|
|
|
30633530383865663731383635366566613565393632336337306334663937396135303539363536
|
|
|
|
|
65656331386532306438383031313164626237393762303636663831666239336637666465633630
|
|
|
|
|
63383961336333663537666561376532376631343132363638633364616531306363623432396332
|
|
|
|
|
31343864626335613131653762363338393738343531626330663663376531336230656365363661
|
|
|
|
|
64363530333965386439376161313534333435393966656231613161633034303964653564363436
|
|
|
|
|
63396131623562303962383137666439373132633330363237353031306561306362666561376435
|
|
|
|
|
66383535333233663736363762323662363264393365643135323834366565653932313863353437
|
|
|
|
|
38363536323739336165356561613335653464363637636234313637383066633865353239393961
|
|
|
|
|
31613532373361383062393238303430353662383930373764373237383035633364323661623661
|
|
|
|
|
62653661623234366666616532663563316234663137616338393036333937633963323635366165
|
|
|
|
|
61613032363531666338353132346365323661626265626461396362646338373536306362303338
|
|
|
|
|
31646564343434633433663039366533656665643235336136333134366231376336343465336539
|
|
|
|
|
37363139346239666232313366613432646265626564313838346364306133306434313337626562
|
|
|
|
|
32346430316532623037633838303363346130623636386134313566383333613565316135373264
|
|
|
|
|
38393261313435323433326130303032393538333963646430646366653363613830613534353832
|
|
|
|
|
32643434613964613239363966376138376661636335373130356430336631333461623734383535
|
|
|
|
|
35356461326136636234663738343738303064386536336462303632303461303733333666383036
|
|
|
|
|
38383563393739343134363038623638393766316533373163336439336238613734656132623163
|
|
|
|
|
32626564666430663339316163393934323133306238353562323866633738363737643937376435
|
|
|
|
|
66306430623334643766623564363239346361393666663766306637313265313833396435626234
|
|
|
|
|
35366239616132623863343365663833363934316362636638303536656631633364646235366566
|
|
|
|
|
39386637613735373761643339396132323031323438316633363464336636316534643435323861
|
|
|
|
|
62663732366430646538626663313035616235643537643234356434323635373962343633336266
|
|
|
|
|
30323031306438656537626430356231393466313334633934623632663661306361313330633339
|
|
|
|
|
33396234386536336331616161363132343765306562303932313963623037366633383765663134
|
|
|
|
|
62636466636433666530363930633765626531613539363832313361666565323635326334326533
|
|
|
|
|
36336439363836663033643262336139613437623633616138656564393032393263393132303031
|
|
|
|
|
31626336663039633362366663306662333432643261613464303939326562653562346239383261
|
|
|
|
|
37373966383635626433363831653936333964626262363839383936356634633233336365623765
|
|
|
|
|
62313031353538636464303234383865613932323164386336316362323731303263346637346534
|
|
|
|
|
32353734343130343364646161643431353230336364366330363261326334613936633234646264
|
|
|
|
|
39623163346663613431323630303034383761663835373565663166366239303130386139396236
|
|
|
|
|
62393565353761636636366462353038366664663430616332326465373364396264323064666536
|
|
|
|
|
61653035633562316634336263636332363733663666626131383161656535383133613939623034
|
|
|
|
|
66363632613733306336323165336534346337386163656631343332323636353539353737316131
|
|
|
|
|
34616336343865656565646338653037333838663736333330376330663834373138363739633064
|
|
|
|
|
61646439636365323838376131663266636235333062616532613936616339633661303634323531
|
|
|
|
|
63356335386534353639323566376134373565353137333134363761666532366431633634333430
|
|
|
|
|
39326662633765626230326130386333393463323433366162363432613234336634663439313339
|
|
|
|
|
33653737333830336264343839396563316462613032376335356634383436353962613636333866
|
|
|
|
|
32333032623635323430356636623138366635386533646133626164316438393937626462313239
|
|
|
|
|
31636635363064363763396566306234363965346438653738333961623435303233396634643763
|
|
|
|
|
65343763386464366636383734336335646464306262623363303934393734333666636635356365
|
|
|
|
|
65353931303862343765343665373139376530656263356365366333323135383838383030613166
|
|
|
|
|
30303963383833643937636539656164336537383337346436303534303035313935323633663765
|
|
|
|
|
333464336235373230316337333632326138
|