Skip to content
LavaverPublic

About

A simple yet powerful web request Swiss Army knife

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Uninet Client

A multi-protocol command-line client, compatible with the everyday usage of curl and PowerShell's Invoke-WebRequest, but not limited to HTTP. Uninet Client routes every resource — HTTP(S), WebSocket, UDP, DNS, FTP, SFTP, Gopher, file:// and data: — through one unified download pipeline with progress reporting, checksum verification, resume, retry, and file-lock handling.

Provenance and disclaimer. This project was generated by an AI assistant without human authorship. It is published for reference and experimentation only; it has not undergone an independent security audit. Review the code yourself before relying on it in any production or security-sensitive context.

Table of contents

Features

  • One pipeline, many protocols. Every source produces a byte stream that is written to stdout or a file with a deterministic progress bar, a suggested filename, and a lock-wait step that prevents two concurrent downloads from racing on the same file.
  • HTTP(S) with manual redirect following (each hop is logged), byte ranges, resume (-C), --retry with granular curl-style exit codes, and --fail semantics that suppress error bodies on HTTP errors.
  • Checksum verification against a user-supplied digest or an auto-discovered sidecar file.
  • Parallel segmented downloads (--segments N) that split a file into byte-range requests and reassemble it in order, with automatic fallback to a single connection when the server does not advertise ranges.
  • WebSocket (ws://, wss://) that sends an optional first message and streams every received message to the output.
  • UDP request/response and streaming receive, plus a built-in DNS resolver for A/AAAA lookups.
  • TLS via the Mozilla root store, with --cacert and --insecure overrides for HTTP(S).
  • Output control: -o, -O/--remote-name, --no-clobber, silent and verbose modes, --write-out formatting, and --lang en|zh bilingual output.
  • Windows Terminal integration: the tab progress ring reflects aggregate download progress.
  • Piped-script protection: by default, an HTTP(S) text body written to stdout has a zero-width space inserted between every character, so piping it straight into a shell no longer executes untrusted text while it still renders identically (see Piped-script protection).

Installation

The project requires a recent stable Rust toolchain (edition 2024).

git clone https://github.com/Lavaver/Uninet && cd webclient
cargo build --release

The resulting binary is target/release/webclient (webclient.exe on Windows).

Supported protocols

Scheme Description
http://, https:// HTTP(S) with redirects, ranges, resume, and --segments
ws://, wss:// WebSocket client
udp:// Generic UDP send/receive and streaming
dns:// DNS query (A, AAAA, CNAME, MX, TXT, NS, SOA, PTR, SRV, CAA) over UDP
file:// Local file
data: data: URI
gopher:// Gopher
ftp://, ftps:// FTP and explicit FTPS
sftp://, scp:// SSH file transfer

Usage

Uninet Client is invoked as webclient <URL> [options]. With no output flag, the response body is written to stdout; status and progress are written to stderr. Use --detail to surface transfer information, or -s/--silent to suppress everything but the body.

# Download a file with a progress bar.
webclient https://example.com/releases/app.zip -o=app.zip

# Verify a digest while downloading.
webclient https://example.com/app.zip -o=app.zip \
  --expected-hash sha256:7d865e959b2466918c9863afca942d0fb89d7c9ac0c99bafc3749504ded97730

# Auto-discover a sidecar checksum (<url>.sha256, .sha1, .md5, .sha512).
webclient https://example.com/app.zip -o=app.zip --verify

# Parallel segmented download (requires HTTP range support).
webclient https://example.com/app.iso -o=app.iso --segments 8

# Resume an interrupted download from the current file size.
webclient -C - https://example.com/app.iso -o=app.iso

# Retry transient failures up to 5 times.
webclient --retry 5 https://example.com/app.zip -o=app.zip

WebSocket

# Send a text message and stream the reply.
webclient ws://127.0.0.1:8080/chat --data-raw "hello"

# Send a binary message.
webclient ws://127.0.0.1:8080/echo --data-binary @payload.bin

UDP and DNS

# Send one datagram and print the reply.
webclient udp://127.0.0.1:9999 --data-raw "ping" --udp-timeout 3

# Receive datagrams continuously (for example, an RTP audio stream).
webclient udp://0.0.0.0:5004 --udp-listen -o=stream.bin

# Resolve a name, using the default resolver.
webclient dns://example.com

# Resolve a name through a specific resolver.
webclient dns://8.8.8.8/example.com

Piped-script protection

By default, when an HTTP(S) text body is written to stdout, Uninet Client inserts an invisible zero-width space (U+200B) between every character. The text renders identically in a terminal, but piping it straight into a shell —

webclient http://example.com/install.sh | sh     # no longer executes
webclient http://example.com/setup.ps1 | iex     # no longer executes

— no longer runs the fetched script. This guards against pipe-to-shell attacks where a compromised or untrusted URL serves a script.

The guard is applied only to text content, decided by the Content-Type header: text/*, JSON/XML/YAML, common script and markup types, and so on. Binary bodies (image/*, application/octet-stream, archives, video/audio, …) pass through unchanged, so webclient <url> | tar xz still works. A missing or unrecognised Content-Type is treated as text — the guard errs on the side of protecting.

The guard can be disabled in two ways:

  • Per invocation, with --no-control-characters, when you trust the source.
  • Per host, by listing the host as trusted (below), which disables the guard for that host whether or not the flag is passed.

Trusted hosts

Hosts you explicitly trust are exempt from the guard. List them in a trust-hosts.json file placed next to the executable, or in the UNIHOST_TRUST_HOSTS environment variable:

["example.com", "cdn.example.com:8443"]
# Comma, semicolon or whitespace separated.
export UNIHOST_TRUST_HOSTS="example.com, cdn.example.com:8443"

Matching is an exact, case-insensitive comparison of the hostname (or host:port); a bare hostname trusts that host on any port. Wildcards and regular expressions are deliberately not supported, so an entry names one fully-qualified host and nothing else. Uninet Client does not verify these sources or their scripts, and grants them no security warranty — mark a host trusted only when you are certain it cannot be used against you.

Note: file output (-o/-O), non-web schemes, and binary bodies are never affected — the guard only touches text written to stdout.

Command-line options

Option Description
-X, --request METHOD HTTP method (defaults to GET, or POST when a body is supplied).
-H, --header "Name: value" Add a request header (repeatable).
-d, --data DATA Request body; @file reads from a file.
--data-raw DATA Raw body without @file interpretation.
--data-binary DATA Binary body; @file reads from a file.
--json JSON JSON body; sets Content-Type: application/json.
-F, --form "name=value" Multipart form field (repeatable).
-o, --output=FILE Write the body to FILE. A bare -o derives the name.
-O, --remote-name Save using the remote filename.
--no-clobber Refuse to overwrite an existing file.
-L, --location Follow redirects (on by default; kept for curl compatibility).
-f, --fail Fail with exit code 22 on HTTP ≥400 and skip the error body.
-i, --include Include the status line and headers in the output.
-I, --head Fetch headers only (HEAD request).
-s, --silent Suppress all non-body output.
-v, --verbose Verbose request/response detail.
--detail Show status and progress alongside the body.
-w, --write-out FORMAT Print transfer variables such as %{http_code}.
-A, --user-agent UA Set the User-Agent header.
-u, --user USER:PASS HTTP basic authentication.
-e, --referer URL Set the Referer header.
-b, --cookie COOKIE Set the Cookie header.
-x, --proxy URL Route through an HTTP proxy.
-r, --range RANGE Fetch a byte range such as 0-1023.
-C, --continue-at OFFSET Resume a download (- means current file size).
--retry N Retry failed transfers on transient errors.
--retry-delay SECS Seconds between retries (default 1).
--segments N Parallel byte-range download (HTTP/HTTPS, file output).
--expected-hash ALGO:HEX Verify the file against a digest (repeatable).
--verify Auto-fetch a sidecar checksum and verify against it.
--udp-listen Keep receiving UDP datagrams until interrupted.
--udp-timeout SECS Seconds to wait for a UDP reply (default 5).
-m, --max-time SECS Maximum total transfer time.
--connect-timeout SECS Connection timeout.
-Z, --parallel N Maximum concurrent requests (default 1).
-k, --insecure Skip TLS certificate verification (HTTP(S)).
--cacert FILE Custom CA bundle for TLS (HTTP(S)).
--no-color Disable colored output.
--no-control-characters Disable the zero-width-space script-injection guard for stdout.
--lang en|zh Force the interface language.

Exit codes

Uninet Client returns granular, curl-style exit codes:

Code Meaning
0 Success
1 Generic error
2 Usage or configuration error
3 Invalid URL
5 Proxy failure
6 DNS resolution failure
7 Connection failure
22 HTTP status ≥400 (with --fail)
28 Timeout
90 Checksum mismatch
130 Interrupted (Ctrl+C)

Checksum verification

--expected-hash verifies a downloaded file against an explicit digest of the form ALGO:HEX, where ALGO is one of sha256, sha1, md5, or sha512. The option is repeatable to check multiple algorithms at once. On mismatch the client exits with code 90.

--verify fetches a sidecar checksum file by appending .sha256, .sha1, .md5, then .sha512 to the request URL, taking the first digest found. Both forms require an explicit -o=FILE.

Parallel segmented downloads

--segments N probes the server with a one-byte range request to learn the total size, splits the file into N byte ranges, fetches them concurrently into temporary part files, and reassembles them in byte order. When the server does not advertise byte ranges (or the output is not an explicit file), the client falls back to a single connection. Segmented downloads are disabled when an explicit --range or --continue-at is present.

Development

cargo build       # debug build
cargo test        # run the unit tests
cargo clippy      # lint

The unit tests cover the DNS wire encoding/decoding, checksum parsing and hashing, exit-code classification, redirect rules, filename derivation, and --write-out formatting.

License

Licensed under the Apache License, Version 2.0.

About

A simple yet powerful web request Swiss Army knife

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages