Cookbook / Production

Calling other services safely

An outbound HTTP client with SSRF protection, timeouts, retries and a circuit breaker.

Calling a vendor API, or fetching a URL a user gave you, has failure modes a plain HTTP client ignores. axeon-client wraps reqwest with the protections you would otherwise write yourself: it refuses private and internal addresses (server-side request forgery), applies timeouts, retries only when it is safe, and stops hammering a service that is down.

Terminal

cargo add axeon-client

src/main.rs

use axeon::metrics::Metrics;
use axeon::{Json, Server, ServerError};
use axeon_client::{Client, Egress};
use axeon::json;
use serde::Deserialize;
use std::time::Duration;

// State is looked up by type, so give each client its own type.
struct Crawler(Client);
#[allow(dead_code)]
struct Vendor(Client);

#[derive(Deserialize)]
struct Fetch {
    url: String,
}

fn main() {
    tracing_subscriber::fmt().with_target(false).init(); // failures are logged, not sent to clients
    let metrics = Metrics::new();

    // URLs from users (a link preview, a crawler): any public host, nothing internal.
    let crawler = Client::builder("crawler")
        .egress(Egress::public())
        .timeout(Duration::from_secs(5))
        .metrics(&metrics)
        .build()
        .expect("client");

    // A vendor: only its hosts, with its credentials.
    let vendor = Client::builder("vendor")
        .egress(Egress::hosts(["api.vendor.example"]))
        .timeout(Duration::from_secs(5))
        .metrics(&metrics)
        .build()
        .expect("client");

    let mut app = Server::new();
    app.middleware(metrics.clone()).mount("/", metrics.router());
    app.state(Crawler(crawler));
    app.state(Vendor(vendor));

    // GET /fetch?url=https://example.com
    app.get("/fetch", |req| async move {
        let target: Fetch = req.query_as()?;
        let crawler = &req.state::<Crawler>().unwrap().0;
        // A blocked address or a failure becomes 502, 503 or 504 with `?`.
        let response = crawler.get(&target.url).context(&req).send().await?;
        let status = response.status().as_u16();
        let body = response.text().await.map_err(axeon_client::Error::from)?;
        Ok::<_, ServerError>(Json(json!({ "status": status, "bytes": body.len() })))
    });

    app.listen("127.0.0.1:3000").expect("server failed to start");
}

Terminal

curl "http://127.0.0.1:3000/fetch?url=https://example.com"
curl -i "http://127.0.0.1:3000/fetch?url=http://169.254.169.254/latest/meta-data/"   # refused
curl -i "http://127.0.0.1:3000/fetch?url=http://127.0.0.1:3000/metrics"              # refused

What you get#

Egress control. By default only public addresses are reachable. Private networks, loopback, link-local and cloud metadata addresses are refused, and the check runs on the addresses a name *resolves to*, on IP-literal URLs and on every redirect, so evil.example pointing at 10.0.0.1 is blocked too.
Timeouts. 10 seconds per attempt and 3 to connect, by default.
Retries, only when safe. Up to 3 attempts with backoff and jitter. Connection failures retry for any request, since nothing was sent. Timeouts and 429/502/503/504 retry only for idempotent requests (GET, PUT, DELETE, or anything with an Idempotency-Key). Retry-After is honoured.
Circuit breaking. After 5 failures in a row a service fails fast for 30 seconds, then one probe decides.
Request IDs. .context(&req) passes X-Request-Id and traceparent on, so one request can be followed across services.
Metrics per client name, so crawling arbitrary hosts cannot create unbounded series.

Notes#

Register each client under its own type, as above: shared state is looked up by type, so two bare Clients would replace each other.
Environment proxies are ignored on purpose: a proxy would resolve names itself, around the egress checks.
allow_private() exists for your own internal services. Never combine it with user-supplied URLs.