Commit Graph

1400 Commits (server-h2-handling-using-protocols)

Author SHA1 Message Date
Francis Lavoie 3d7d60f7cf
caddyhttp: Add `uuid` to access logs when used (#5859) 2023-12-13 15:40:15 -07:00
Mohammed Al Sahaf dc12bd9743
proxyprotocol: use github.com/pires/go-proxyproto (#5915)
* proxyprotocol: use github.com/pires/go-proxyproto

* Fix typo: r/generelly/generally

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* add config options for `Deny` CIDR and fallback policy

* use `netip` package & trust unix sockets

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-12-13 09:07:43 -07:00
Matt Holt 4a09cf0dc0
caddytls: Sync distributed storage cleaning (#5940)
* caddytls: Log out remote addr to detect abuse

* caddytls: Sync distributed storage cleaning

* Handle errors

* Update certmagic to fix tiny bug

* Split off port when logging remote IP

* Upgrade CertMagic
2023-12-07 11:00:02 -07:00
Andreas Kohn b24ae63ea6
caddytls: Context to DecisionFunc (#5923)
See https://github.com/caddyserver/certmagic/pull/255
2023-12-07 10:40:13 -07:00
Mohammed Al Sahaf 4173e2c77a
tls: accept placeholders in string values of certificate loaders (#5963)
* tls: loader: accept placeholders in string values

* appease the linter
2023-12-04 09:23:15 -07:00
Matt Holt 18f34290d2
templates: Offically make templates extensible (#5939)
* templates: Offically make templates extensible

This supercedes #4757 (and #4568) by making template extensions
configurable.

The previous implementation was never documented AFAIK and had only
1 consumer, which I'll notify as a courtesy.

* templates: Add 'maybe' function for optional components

* Try to fix lint error
2023-11-28 09:39:14 -07:00
WeidiDeng 22eecdb90c
http2 uses new round-robin scheduler (#5946) 2023-11-24 01:54:27 +00:00
WeidiDeng 4de2c1c65e
panic when reading from backend failed to propagate stream error (#5952) 2023-11-23 03:18:18 -05:00
WeidiDeng 96f638eaad
httpredirectlistener: Only set read limit for when request is HTTP (#5917) 2023-11-20 12:31:36 +00:00
Matthew Holt 7e52db8280
fileserver: Add .m4v for browse template icon 2023-11-14 13:39:57 -07:00
Mohammed Al Sahaf 3b3d678714
Revert "caddyhttp: Use sync.Pool to reduce lengthReader allocations (#5848)" (#5924) 2023-11-01 13:17:02 -04:00
Mariano Cano ac0ad4da84
Upgrade acmeserver to github.com/go-chi/chi/v5 (#5913)
This commit upgrades the router used in the acmeserver to
github.com/go-chi/chi/v5. In the latest release of step-ca, the router
used by certificates was upgraded to that version.

Fixes #5911

Signed-off-by: Mariano Cano <mariano.cano@gmail.com>
2023-10-23 21:02:11 -04:00
Francis Lavoie 4c10a05431
caddyhttp: Adjust `scheme` placeholder docs (#5910) 2023-10-22 17:47:16 -04:00
Francis Lavoie 4e8245df0b
templates: Delete headers on `httpError` to reset to clean slate (#5905) 2023-10-18 16:43:14 -06:00
Harish Shan c8559c4485
caddyhttp: Use sync.Pool to reduce lengthReader allocations (#5848)
* Use sync.Pool to reduce lengthReader allocations

Signed-off-by: Harish Shan <140232061+perhapsmaple@users.noreply.github.com>

* Add defer putLengthReader to prevent leak

Signed-off-by: Harish Shan <140232061+perhapsmaple@users.noreply.github.com>

* Cleanup in putLengthReader

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

---------

Signed-off-by: Harish Shan <140232061+perhapsmaple@users.noreply.github.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-10-16 14:42:01 -06:00
WeidiDeng 7c82e265da
core: quic listener will manage the underlying socket by itself (#5749)
* core: quic listener will manage the underlying socket by itself.

* format code

* rename sharedQUICTLSConfig to sharedQUICState, and it will now manage the number of active requests

* add comment

* strict unwrap type

* fix unwrap

* remove comment
2023-10-16 09:28:15 -06:00
Francis Lavoie 0900844c81
templates: Clarify `include` args docs, add `.ClientIP` (#5898) 2023-10-15 20:58:46 -04:00
Forza 130f6d1f83
fileserver: Set canonical URL on browse template (#5867)
* Browse.html: Add canonical URL and home-link

When contents are equal, but maybe just a sort order is different, it is good to add `<link rel="canonical" href="base-path/" />`. This helps search engines propeely index the page.

I also added a link to the home page with the name of `{{.Host}}` just above the bread crumbs to make the page clearer.

https://paste.tnonline.net/files/28Wun5CQZiqA_Screenshot_20231007_134435_Opera.png

* Update browse.html
2023-10-11 13:47:38 -06:00
Bas Westerbaan 289934f3d1
tls: Add X25519Kyber768Draft00 PQ "curve" behind build tag (#5852)
… when compiled with cfgo (https://github.com/cloudflare/go).
2023-10-11 13:45:37 -06:00
Matt Holt 3a3182fba3
reverseproxy: Add more debug logs (#5793)
* reverseproxy: Add more debug logs

This makes debug logging very noisy when reverse proxying, but I guess
that's the point.

This has shown to be useful in troubleshooting infrastructure issues.

* Update modules/caddyhttp/reverseproxy/streaming.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddyhttp/reverseproxy/streaming.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Add opt-in `trace_logs` option

* Rename to VerboseLogs

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-10-11 13:36:20 -06:00
Francis Lavoie e8b8d4a8cd
reverseproxy: Fix `least_conn` policy regression (#5862) 2023-10-11 16:04:28 +00:00
Francis Lavoie a8586b05aa
reverseproxy: Add logging for dynamic A upstreams (#5857) 2023-10-11 09:50:44 -06:00
Francis Lavoie 05dbe1c171
reverseproxy: Replace health header placeholders (#5861) 2023-10-11 09:50:28 -06:00
Fred Cox b245ecd325
reverseproxy: fix parsing Caddyfile fails for unlimited request/response buffers (#5828) 2023-10-11 04:42:40 -04:00
Francis Lavoie 2a6859a5e4
reverseproxy: Fix retries on "upstreams unavailable" error (#5841) 2023-10-10 22:07:20 +00:00
Christoph e0aaefab80
encode: Add `application/wasm*` to the default content types (#5869) 2023-10-10 21:18:37 +00:00
Kévin Dunglas fa5a579b60
fileserver: Add command shortcuts `-l` and `-a` (#5854) 2023-10-10 20:57:18 +00:00
Thanmay Nath 5653c36bc2
templates: Add dummy `RemoteAddr` to `httpInclude` request, proxy compatibility (#5845)
* Enhancement: Allow X-Forwarded-For Header in httpInclude Virtual Requests

The goal of this enhancement is to modify the funcHTTPInclude function in the Caddy codebase to include the X-Forwarded-For header in the virtual request. This change will enable reverse proxies to set the X-Forwarded-For header, ensuring that the client's IP address is correctly provided to the target endpoint. This modification is essential for applications that depend on the X-Forwarded-For header for various functionalities, such as authentication, logging, or content customization.

* Updated tplcontext.go - set `virtReq.RemoteAddr = "127.0.0.1"`

i have made the suggested changes

* Apply suggestions from code review

* Update modules/caddyhttp/templates/tplcontext.go

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-10-07 20:47:34 +00:00
Patrick Koenig 4feac4d83c
reverseproxy: Allow fallthrough for response handlers without routes (#5780) 2023-10-05 23:15:26 -04:00
Matthew Holt 58ab3a01a0
caddyhttp: Use LimitedReader for HTTPRedirectListener 2023-09-26 07:32:46 -06:00
glowinthedark a306c5f769
fileserver: browse template SVG icons and UI tweaks (#5812)
* fileserver browse.html UI tweaks: folder-symlink icon, search

fileserver browse.html UI tweaks: folder-symlink icon, search

- ui - add folder-symlink SVG icon
- search: use `<input type="search">` instead of `text`
- fix npe with `sizebar.style.width` = null in grid mode

* tabify whitespace

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-09-15 19:05:45 -06:00
Pascal Vorwerk 1e0dea59ef
reverseproxy: fix nil pointer dereference in AUpstreams.GetUpstreams (#5811)
fix a nil pointer dereference in AUpstreams.GetUpstreams when AUpstreams.Versions is not set (fixes caddyserver#5809)

Signed-off-by: Pascal Vorwerk <info@fossores.de>
2023-09-10 19:08:02 -04:00
Paul Jeannot 1b73e3862d
logging: query filter for array of strings (#5779)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-08-29 22:59:43 +00:00
Francis Lavoie c46ec3b500
logging: Clone array on log filters, prevent side-effects (#5786)
Fixes https://caddy.community/t/is-caddy-mutating-header-content-from-logging-settings/20947
2023-08-29 11:41:39 -06:00
Matthew Holt ed8bb13c5d
fileserver: Export BrowseTemplate
This allows programs embedding Caddy to customize the browse template.
2023-08-29 09:34:20 -06:00
Matt Holt b377208ede
chore: Appease gosec linter (#5777)
These happen to be harmless memory aliasing
but I guess the linter can't know that and we
can't really prove it in general.
2023-08-23 20:47:54 -06:00
Karun Agarwal 288216e1fb
httpcaddyfile: Stricter errors for site and upstream address schemes (#5757)
Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-08-19 07:28:25 -04:00
Mohammed Al Sahaf 0a6d3333b2
fileserver: docs: clarify the ability to produce JSON array with `browse` (#5751) 2023-08-18 19:04:08 +00:00
Matt Holt 936ee918ee
reverseproxy: Always return new upstreams (fix #5736) (#5752)
* reverseproxy: Always return new upstreams (fix #5736)

* Fix healthcheck logger race
2023-08-17 11:33:40 -06:00
Jacob Gadikian d6f86cccf5
ci: use gci linter (#5708)
* use gofmput to format code

* use gci to format imports

* reconfigure gci

* linter autofixes

* rearrange imports a little

* export GOOS=windows golangci-lint run ./... --fix
2023-08-14 09:41:15 -06:00
Matthew Holt 2d7d806fcf
fileserver: Slightly more fitting icons 2023-08-11 20:53:11 -06:00
Matthew Holt 080db93817
caddytls: Update docs for on-demand config 2023-08-09 11:15:01 -06:00
Francis Lavoie a8492c064d
fileserver: Don't repeat error for invalid method inside error context (#5705) 2023-08-09 17:12:09 +00:00
Matt Holt 6cdcc2a782
ci: Update to Go 1.21 (#5719)
* ci: Update to Go 1.21

* Bump quic-go to v0.37.4

* Check EnableFullDuplex err

* Linter bug suppression

See https://github.com/timakin/bodyclose/issues/52

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-08-09 12:34:28 -04:00
Jacob Gadikian b32f265eca
ci: Use gofumpt to format code (#5707) 2023-08-07 19:40:31 +00:00
Matthew Holt 431adc0980
templates: Fix httpInclude (fix #5698)
Allowable during feature freeze because this is a simple, non-invasive
bug fix only.
2023-08-07 12:53:21 -06:00
Mohammed Al Sahaf 65e33fc1ee
reverseproxy: do not parse upstream address too early if it contains replaceble parts (#5695)
* reverseproxy: do not parse upstream address too early if it contains replaceble parts

* remove unused method

* cleanup

* accommodate partially replaceable port
2023-08-05 23:30:02 +02:00
Matthew Holt c049bab458
fileserver: browse: Render SVG images in grid 2023-08-03 12:53:47 -06:00
WeidiDeng e2fc08bd34
reverseproxy: Fix hijack ordering which broke websockets (#5679) 2023-08-03 04:08:12 +00:00
Francis Lavoie 1913930783
encode: Fix infinite recursion (#5672) 2023-08-02 18:21:11 -06:00
Francis Lavoie cd486c25d1
caddyhttp: Make use of `http.ResponseController` (#5654)
* caddyhttp: Make use of http.ResponseController

Also syncs the reverseproxy implementation with stdlib's which now uses ResponseController as well 2449bbb5e6

* Enable full-duplex for HTTP/1.1

* Appease linter

* Add warning for builds with Go 1.20, so it's less surprising to users

* Improved godoc for EnableFullDuplex, copied text from stdlib

* Only wrap in encode if not already wrapped
2023-08-02 20:03:26 +00:00
mmm444 da23501457
reverseproxy: Connection termination cleanup (#5663) 2023-08-01 14:01:12 +00:00
Omar Ramadan d7d16360d4
reverseproxy: Export ipVersions type (#5648)
allows AUpstreams to be instantiated externally
2023-07-25 12:50:21 -06:00
Matthew Holt 18c309b5fa
caddyhttp: Preserve original error (fix #5652) 2023-07-25 09:41:56 -06:00
ydylla e041962b66
fileserver: add lazy image loading (#5646) 2023-07-22 15:50:36 +00:00
Marten Seemann f45a6de20d
go.mod: Update quic-go to v0.37.0, bump to Go 1.20 minimum (#5644)
* update quic-go to v0.37.0

* Bump to Go 1.20

* Bump golangci-lint version, yml syntax consistency

* Use skip-pkg-cache workaround

* Workaround needed for both?

* Seeding weakrand is no longer necessary

---------

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-07-21 22:00:48 -06:00
Matthew Holt 4e36b4c9d1
fileserver: Tweak grid view of browse template
All cells on row have same height.
Center-align vertically.
2023-07-17 11:18:40 -06:00
Mohammed Al Sahaf 27bc16abed
fileserver: add `export-template` sub-command to `file-server` (#5630) 2023-07-13 15:54:48 -06:00
Matt Holt 0e2c7e1d35
caddytls: Reuse certificate cache through reloads (#5623)
* caddytls: Don't purge cert cache on config reload

* Update CertMagic

This actually avoids reloading managed certs from storage
when already in the cache, d'oh.

* Fix bug; re-implement HasCertificateForSubject

* Update go.mod: CertMagic tag
2023-07-11 19:10:58 +00:00
Matthew Holt 5dec11f2a0
reverseproxy: Pointer receiver
This avoids copying the Upstream, which has an atomically-accessed value
in it.
2023-07-08 13:42:51 -06:00
Matthew Holt 66114cb155
caddyhttp: Trim dot/space only on Windows (fix #5613)
Follow-up to #2917. Path matcher needs to trim dots and spaces but only
on Windows.
2023-07-08 13:42:13 -06:00
Francis Lavoie 7a69ae7571
reverseproxy: Honor `tls_except_port` for active health checks (#5591) 2023-06-22 16:20:30 -06:00
Saber Haj Rabiee 361946eb0c
reverseproxy: weighted_round_robin load balancing policy (#5579)
* added weighted round robin algorithm to load balancer

* added an adapt integration test for wrr and fixed a typo

* changed args format to Caddyfile args convention

* added provisioner and validator for wrr

* simplified the code and improved doc
2023-06-20 11:42:58 -06:00
mmm444 424ae0f420
reverseproxy: Experimental streaming timeouts (#5567)
* reverseproxy: WIP streaming timeouts

* More verbose logging by using the child logger

* reverseproxy: Implement streaming timeouts

* reverseproxy: Refactor cleanup

* reverseproxy: Avoid **time.Timer

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-06-19 15:54:43 -06:00
guangwu 4548b7de8e
chore: remove refs of deprecated io/ioutil (#5576) 2023-06-16 21:27:57 -06:00
Francis Lavoie 3b19aa2b5a
headers: Allow `>` to defer shortcut for replacements (#5574) 2023-06-15 17:18:55 -06:00
Dominik Roos 6a41b62e70
caddyhttp: Support custom network for HTTP/3 (#5573)
Allow registering a custom network mapping for HTTP/3. This is useful
if the original network for HTTP/1.1 and HTTP/2 is not a standard `unix`,
`tcp4`, or `tcp6` network. To keep backwards compatibility, we fall back
to `udp` if the original network is not registered in the mapping.

Fixes #5555
2023-06-13 19:33:39 -06:00
Corin Langosch 2ddb717144
reverseproxy: Fix parsing of source IP in case it's an ipv6 address (#5569) 2023-06-12 09:35:22 -06:00
365cent 56af1ceb32
fileserver: browse: Better grid layout (#5564)
* feat: better implementation of grid layout

* fix: vertical alignment
2023-06-05 07:39:57 +00:00
Matthew Holt 4ba03c9d38
caddytls: Clarify some JSON config docs 2023-06-04 22:15:50 -06:00
kassienull 31d75acc9c
templates: Add `readFile` action that does not evaluate templates (#5553)
* Create an includeRaw template function to include a file without parsing it as a template.

Some formatting fixes

* Rename to readFile, various docs adjustments

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-05-26 10:16:28 -06:00
Matt Holt 5bd9c49042
fileserver: Don't set Etag if mtime is 0 or 1 (close #5548) (#5550) 2023-05-22 14:17:15 -06:00
pistasjis cdd3884b32
fileserver: browse: minor tweaks for grid view, dark mode (#5545)
* Make grid entries take up full width on mobile and fix breadcrumb color issue in dark mode

Signed-off-by: Pistasj <odyssey346@disroot.org>

* Do mholt's suggestions

Signed-off-by: Pistasj <odyssey346@disroot.org>

---------

Signed-off-by: Pistasj <odyssey346@disroot.org>
2023-05-20 17:23:17 -06:00
Charles Duffy 2615c9c524
fileserver: Only set Etag if not already set (fix #5546) (#5547) 2023-05-20 17:21:43 -06:00
pistasjis 5336bc0fb6
fileserver: Fix file browser breadcrumb font (#5543)
Signed-off-by: Pistasj <odyssey346@disroot.org>
2023-05-19 11:08:47 -06:00
WeidiDeng 29452647d8
caddyhttp: Fix h3 shutdown (#5541)
* swap h3server close and listener close, avoid quic-listener not closing

* fix typo
2023-05-19 10:00:00 -06:00
Matthew Holt bd34cb6b4e fileserver: More filetypes for browse icons 2023-05-19 09:59:44 -06:00
pistasjis 2d236ead3e
fileserver: Fix file browser footer in grid mode (#5536)
* Fix file browser footer in grid

Signed-off-by: Odyssey <odyssey346@disroot.org>

* Fix file browser footer while in grid mode

Signed-off-by: Pistasj <odyssey346@disroot.org>

* Do mholt's suggestions

Signed-off-by: Odyssey <odyssey346@disroot.org>

---------

Signed-off-by: Odyssey <odyssey346@disroot.org>
Signed-off-by: Pistasj <odyssey346@disroot.org>
2023-05-19 09:51:21 -06:00
Francis Lavoie cbf16f6d9e
caddyhttp: Implement named routes, `invoke` directive (#5107)
* caddyhttp: Implement named routes, `invoke` directive

* gofmt

* Add experimental marker

* Adjust route compile comments
2023-05-16 15:27:52 +00:00
Tran Phong 13a37688dc
rewrite: use escaped path, fix #5278 (#5504)
* use escaped path while rewriting

Signed-off-by: TP-O <letranphong2k1@gmail.com>

* restore line break

---------

Signed-off-by: TP-O <letranphong2k1@gmail.com>
2023-05-16 09:16:07 -06:00
Francis Lavoie e8352aef38
headers: Add > Caddyfile shortcut for enabling defer (#5535) 2023-05-16 01:18:13 -04:00
Francis Lavoie 75b690d248
reverseproxy: Expand port ranges to multiple upstreams in CLI + Caddyfile (#5494)
* reverseproxy: Expand port ranges to multiple upstreams in CLI + Caddyfile

* Add clarifying comment
2023-05-15 12:14:50 -06:00
Matt Holt 52d7335c2b
fileserver: Use EscapedPath for browse (#5534)
* fileserver: Use EscapedPath for browse

Fix #5143

* Fixes if filter element is not present

* Remove extraneous line
2023-05-15 10:48:05 -06:00
Matt Holt 96919acc9d
caddyhttp: Refactor cert Managers (fix #5415) (#5533) 2023-05-15 10:47:30 -06:00
Matt Holt a02ecb0f88
caddytls: Check for nil ALPN; close #5470 (#5473)
* Check for nil ALPN; close #5470

* Apply patch

* Actually I want to try this
2023-05-13 07:09:20 -06:00
jjiang-stripe cfc85ae8ca
caddyhttp: Add a getter for Server.name (#5531) 2023-05-11 10:34:05 -06:00
Matt Holt faf0399e80
caddytls: Configurable fallback SNI (#5527)
* Initial implementation of fallback_sni

* Apply upstream patch
2023-05-10 14:29:29 -06:00
Francis Lavoie b19946f6af
reverseproxy: Optimize base case for least_conn and random_choose policies (#5487)
When only a single request has the least amount of requests, there's no need to compute a random number, because the modulo of 1 will always be 0 anyways.
2023-05-05 20:53:48 -06:00
Francis Lavoie 335cd2e8a4
reverseproxy: Fix active health check header canonicalization, refactor (#5446) 2023-05-05 15:19:22 -06:00
Francis Lavoie 48598e1f2a
reverseproxy: Add `fallback` for some policies, instead of always random (#5488) 2023-05-05 15:08:10 -06:00
Matthew Holt cdce452edc
logging: Actually honor the SoftStart parameter 2023-05-04 16:30:34 -06:00
Matthew Holt f3e8b9d95f
logging: Soft start for net writer (close #5520)
If enabled and there is an error when opening the net writer, ignore the
error and report it along with subsequent logs to stderr.
2023-05-04 16:29:03 -06:00
eanavitarte c8032867b1
fastcgi: Fix `capture_stderr` (#5515) 2023-05-04 00:40:49 +00:00
Francis Lavoie 3f20a7c9f3
acmeserver: Configurable `resolvers`, fix smallstep deprecations (#5500)
* acmeserver: Configurable `resolvers`, fix smallstep deprecations

* Improve default net/port

* Update proxy resolvers parsing to use the new function

* Update listeners.go

Co-authored-by: itsxaos <33079230+itsxaos@users.noreply.github.com>

---------

Co-authored-by: itsxaos <33079230+itsxaos@users.noreply.github.com>
2023-05-03 17:07:22 +00:00
Dave Henderson f0e3981774
logging: Add traceID field to access logs when tracing is active (#5507)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-04-27 02:46:41 +00:00
Kévin Dunglas 1c9ea0113d
caddyhttp: Impl `ResponseWriter.Unwrap()`, prep for Go 1.20's `ResponseController` (#5509)
* feat: add support for ResponseWriter.Unwrap()

* cherry-pick Francis' code
2023-04-26 19:44:01 -04:00
Y.Horie 2b04e09fa7
reverseproxy: Fix reinitialize upstream healthy metrics (#5498)
Co-authored-by: Dávid Szabó <david.szabo97@gmail.com>
2023-04-25 09:59:26 -06:00
cui fliter 3443a8a056
fix some comments (#5508)
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-04-25 09:54:42 -06:00
Stéphane Mourey 2943c41884
templates: Add `fileStat` function (#5497)
* Add isDir template function

* Update modules/caddyhttp/templates/tplcontext.go

Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>

* Fix funcIsDir return value on error

* Fix funcIsDir return false when root file system not specified

* Add stat function, remove isDir function

* Remove isDir function (really)

* Rename stat to fileStat

---------

Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
2023-04-24 10:36:37 -06:00
Francis Lavoie 53b6fab125
caddyfile: Stricter parsing, error for brace on new line (#5505) 2023-04-20 18:43:51 +00:00
Francis Lavoie b301a3df70
celmatcher: Implement `pkix.Name` conversion to string (#5492) 2023-04-19 11:55:22 -04:00
Francis Lavoie 4636109ce1
reverseproxy: Remove deprecated `lookup_srv` (#5396) 2023-04-10 20:08:40 +00:00
WeidiDeng d8d87a378f
caddyhttp: Serve http2 when listener wrapper doesn't return *tls.Conn (#4929)
* Serve http2 when listener wrapper doesn't return *tls.Conn

* close conn when h2server serveConn returns

* merge from upstream

* rebase from latest

* run New and Closed ConnState hook for h2 conns

* go fmt

* fix lint

* Add comments

* reorder import
2023-04-10 17:05:02 +00:00
Francis Lavoie f8b59e77f8
reverseproxy: Add `query` and `client_ip_hash` lb policies (#5468) 2023-04-04 03:31:47 +00:00
Kid f9bd2d3e92
fileserver: Add color-scheme meta tag (#5475) 2023-04-02 22:44:21 -04:00
Corin Langosch b6fe5d4b41
proxyprotocol: Add PROXY protocol support to `reverse_proxy`, add HTTP listener wrapper (#5424)
Co-authored-by: WeidiDeng <weidi_deng@icloud.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-03-31 17:44:53 -04:00
Francis Lavoie 66e571e687
reverseproxy: Add mention of which half a copyBuffer err comes from (#5472)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2023-03-31 15:46:29 -04:00
Francis Lavoie 2b3046de36
caddyhttp: Log request body bytes read (#5461) 2023-03-27 22:40:15 +00:00
Mohammed Al Sahaf 1aef807c71
log: Make sink logs encodable (#5441)
* log: make `sink` encodable

* deduplicate logger fields

* extract common fields into `BaseLog` and embed it into `SinkLog`

* amend godoc on `BaseLog` and `SinkLog`

* minor style change

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-03-27 21:41:24 +00:00
Francis Lavoie e16a886814
caddytls: Eval replacer on automation policy subjects (#5459)
Also renamed the field to SubjectsRaw, which can be considered a breaking change but I don't expect this to affect much.
2023-03-27 21:16:22 +00:00
黑墨水鱼 dd86171d67
headers: Support deleting all headers as first op (#5464)
* Delete all existing fields when fieldName is `*`

* Rearrange deletion before addition in headers

* Revert "Rearrange deletion before addition in headers"

This reverts commit 1b50eeeccc92ccd660c7896d8283c7d9e5d1fcb0.

* Treat deleting all headers as a special case

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

---------

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2023-03-27 21:05:18 +00:00
Francis Lavoie 10b265d252
reverseproxy: Header up/down support for CLI command (#5460) 2023-03-27 20:35:31 +00:00
Francis Lavoie 05e9974570
caddyhttp: Determine real client IP if trusted proxies configured (#5104)
* caddyhttp: Determine real client IP if trusted proxies configured

* Support customizing client IP header

* Implement client_ip matcher, deprecate remote_ip's forwarded option
2023-03-27 20:22:59 +00:00
Matt Holt 0cc49c053f
caddytls: Zero out throttle window first (#5443)
* caddytls: Zero out throttle window first

* Don't error for on-demand 

Fixes b97c76fb47

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-03-20 12:06:00 -06:00
Trea Hauet 2182270a2c
reverseproxy: Reset Content-Length to prevent FastCGI from hanging (#5435)
Fixes: https://github.com/caddyserver/caddy/issues/5420
2023-03-16 11:42:16 -06:00
Matthew Holt a7af7c486e
caddytls: Allow on-demand w/o ask for internal-only 2023-03-14 10:29:27 -06:00
Matthew Holt b97c76fb47
caddytls: Require 'ask' endpoint for on-demand TLS 2023-03-14 10:02:44 -06:00
Matt Holt 6cc3cbbc69
fileserver: New file browse template (#5427)
* fileserver: New file browse template

* Redo extension/icon logic; minor color tweaks

* Fine-tune image display
2023-03-10 18:19:31 +00:00
Chris Reeves b420561737
tracing: Support autoprop from OTEL_PROPAGATORS (#5147)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2023-03-09 09:02:35 -07:00
WeidiDeng b3f0cea2c3
encode: flush status code when hijacked. (#5419) 2023-03-06 09:13:48 -07:00
esell 94d41a9d86
fileserver: Remove trailing slash on fs filenames (#5417) 2023-03-03 14:45:17 -07:00
Matt Holt 99d47050e9
core: Eliminate unnecessary shutdown delay on Unix (#5413)
* core: Eliminate unnecessary shutdown delay on Unix

Fix #5393, alternate to #5405

* Comments, cleanup, adjust logs

* Fix build constraint
2023-03-03 04:00:18 +00:00
Francis Lavoie 85375861f6
caddyhttp: Fix `vars_regexp` matcher with placeholders (#5408)
Changed to match the `vars` matcher's logic for handling placeholders
2023-03-02 09:01:54 -07:00
Francis Lavoie f6bab8ba85
context: Rename func to `AppIfConfigured` (#5397) 2023-02-27 18:58:27 +00:00
Emily Lange 941eae5f61
reverseproxy: allow specifying ip version for dynamic `a` upstream (#5401)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2023-02-27 17:23:09 +00:00
Francis Lavoie 960150bb03
caddyfile: Implement heredoc support (#5385) 2023-02-26 00:34:27 +00:00
Francis Lavoie 9e6919550b
cmd: Expand cobra support, add short flags (#5379)
* cmd: Expand cobra support

* Convert commands to cobra, add short flags

* Fix version command typo

Co-authored-by: Emily Lange <git@indeednotjames.com>

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

---------

Co-authored-by: Emily Lange <git@indeednotjames.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2023-02-24 16:09:12 -07:00
Mohammed Al Sahaf e3909cc385
reverseproxy: refactor HTTP transport layer (#5369)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Weidi Deng <weidi_deng@icloud.com>
2023-02-24 19:54:04 +00:00
Francis Lavoie be53e432fc
caddytls: Relax the warning for on-demand (#5384) 2023-02-22 11:41:01 -07:00
Francis Lavoie 5ded580444
cmd: Adjust documentation for commands (#5377) 2023-02-16 09:14:07 -07:00
Matt Holt 4b119a475f
reverseproxy: Don't buffer chunked requests (fix #5366) (#5367)
* reverseproxy: Don't buffer chunked requests (fix #5366)

Mostly reverts 845bc4d50b (#5289)

Adds warning for unsafe config.

Deprecates unsafe properties in favor of simpler, safer designed ones.

* Update modules/caddyhttp/reverseproxy/caddyfile.go

Co-authored-by: Y.Horie <u5.horie@gmail.com>

* Update modules/caddyhttp/reverseproxy/reverseproxy.go

Co-authored-by: Y.Horie <u5.horie@gmail.com>

* Update modules/caddyhttp/reverseproxy/reverseproxy.go

Co-authored-by: Y.Horie <u5.horie@gmail.com>

* Remove unused code

---------

Co-authored-by: Y.Horie <u5.horie@gmail.com>
2023-02-11 17:25:29 -07:00
Francis Lavoie 90798f3eea
go.mod: Upgrade various dependencies (#5362)
* chore: Upgrade various dependencies

* Support CEL file matcher with no args

* Document `http.request.orig_uri.path.*`, reorder placeholders in docs

---------

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2023-02-08 17:49:17 +00:00
Steffen Brüheim 536c28d4dc
core: Support Windows absolute paths for UDS proxy upstreams (#5114)
* added some tests for parseUpstreamDialAddress

Test 4 fails because it produces "[[::1]]:80" instead of "[::1]:80"

* support absolute windows path in unix reverse proxy address

* make IsUnixNetwork public, support +h2c and reuse it
* add new tests
2023-02-08 10:05:09 -07:00
WeidiDeng c77a6bea66
reverseproxy: Log status code and byte count for websockets (#5140)
* log response size for websocket request

* record size when using hijack bufio.Writer
2023-02-06 16:14:59 -07:00
Francis Lavoie 12bcbe2c49
caddyhttp: Pluggable trusted proxy IP range sources (#5328)
* caddyhttp: Pluggable trusted proxy IP range sources

* Add request to the IPRangeSource interface
2023-02-06 12:44:11 -07:00
Y.Horie 8d3a1b8bcb
caddyauth: Use singleflight for basic auth (#5344)
* caddyauth: Add singleflight for basic auth

* Fixes #5338
* it occurred the thunder herd problem like this https://medium.com/@mhrlife/avoid-duplicate-requests-while-filling-cache-98c687879f59

* Update modules/caddyhttp/caddyauth/basicauth.go

Fix comment

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

---------

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2023-02-06 18:29:03 +00:00
Francis Lavoie e62b5fb586
chore: Build with Go 1.20, keep minimum at 1.18 for now (#5353) 2023-02-06 11:29:20 -05:00
Francis Lavoie 201b9b41f9
chore: Fix warning "range variable captured by func literal" (#5348) 2023-01-31 03:07:57 -05:00
Matthew Holt 0a3efd1641
caddytls: Debug log for ask endpoint 2023-01-30 09:30:53 -07:00
Y.Horie e9d95ab29f
reverseproxy: Add flag to short command to disable redirects (#5330)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Fixes undefined
2023-01-25 09:40:08 -05:00
David Frickert 962310204f
tracing: Support placeholders in span name (#5329)
Fixes https://github.com/caddyserver/caddy/issues/5171
2023-01-25 02:26:44 -05:00
Francis Lavoie 223cbe3d0b
caddyhttp: Add server-level `trusted_proxies` config (#5103) 2023-01-10 00:08:23 -05:00
Y.Horie 845bc4d50b
reverseproxy: Fix hanging for Transfer-Encoding: chunked (#5289)
* Fixes #5236
* enable request body buffering in reverse proxy
  when the request header has Transfer-Encoding: chunked
2023-01-09 00:13:34 -07:00
Emily Lange e450a7377b
reverseproxy: Don't enable auto-https when `--from` flag is http (#5269) 2023-01-06 15:42:07 -05:00
Matt Holt d74f6fd967
reverseproxy: Set origreq in active health check (#5284)
* reverseproxy: Set origreq in active health check

Fix #5281

* Oops; dereference Request
2023-01-06 15:06:38 -05:00
Yannick Ihmels 55035d327a
caddytls: Add `dns_ttl` config, improve Caddyfile `tls` options (#5287) 2023-01-06 14:44:00 -05:00
Matthew Holt 4e9ad50f65
fileserver: Add a couple test cases
With placeholders
2023-01-04 11:07:27 -07:00
darkweak dac7cacd4d
encode: Respect Cache-Control no-transform (#5257)
* encode: respect Cache-Control HTTP header no-transform

* encode: switch to strings.Contains
2022-12-20 13:26:53 -07:00
Matthew Holt fac35db9dc
go.mod: Update quic-go to v0.31.0
And fix a comment typo
2022-12-08 08:55:04 -07:00
Kyle McCullough bfaf2a8201
acme_server: Configurable default lifetime for issued certificates (#5232)
* acme_server: add certificate lifetime configuration option

Signed-off-by: Kyle McCullough <kylemcc@gmail.com>

* pki: allow intermediate cert lifetime to be configured

Signed-off-by: Kyle McCullough <kylemcc@gmail.com>

Signed-off-by: Kyle McCullough <kylemcc@gmail.com>
2022-12-06 00:12:26 -07:00
Alban Lecocq d4a7d89f56
reverseproxy: Improve hostByHashing distribution (#5229)
* If upstreams are all using same host but with different ports
ie:
foobar:4001
foobar:4002
foobar:4003
...
Because fnv-1a has not a good enough avalanche effect
Then the hostByHashing result is not well balanced over
all upstreams

As last byte FNV input tend to affect few bits, the idea is to change
the concatenation order between the key and the upstream strings
So the upstream last byte have more impact on hash diffusion
2022-12-05 11:28:12 -07:00
Matthew Holt ae77a56ac8
Clarify some docs 2022-11-30 16:03:31 -07:00
Mariano Cano 6f8fe01da1
caddypki: Use go.step.sm/crypto to generate the PKI (#5217)
This commit replaces the use of github.com/smallstep/cli to generate the
root and intermediate certificates and uses go.step.sm/crypto instead.

It also upgrades the version of github.com/smallstep/certificates to the
latest version.
2022-11-23 20:47:42 -07:00
Francis Lavoie ee7c92ec9b
reverseproxy: Mask the WS close message when we're the client (#5199)
* reverseproxy: Mask the WS close message when we're the client

* weakrand

* Bump golangci-lint version so path ignores work on Windows

* gofmt

* ugh, gofmt everything, I guess
2022-11-14 09:38:02 -07:00
Jonathan Garcia 33fdea8f26
caddypki: Prefer user-configured root instead of generating new one (#5189)
instead of generating a new root certificate at the default location
load the certificate from the configuration.
fixes: #5181
2022-11-08 12:13:46 -07:00
Alexander Graf 087f126cf4
caddyhttp: Canonicalize header field names (#5176) 2022-10-29 16:35:44 -04:00
XYenon ed503118dd
caddyhttp: add placeholder {http.request.orig_uri.path.*} (#5161) 2022-10-24 11:57:50 -06:00
Matt Holt a3ae146cbd
fileserver: Reject non-GET/HEAD requests (close #5166) (#5167)
* fileserver: Reject non-GET/HEAD requests (close #5166)

* Set Allow header according to RFC 9110 10.2.1
2022-10-24 10:23:57 -06:00
Matt Holt 4bf6cb4199
fileserver: Reject ADS and short name paths; trim trailing dots and spaces on Windows (#5148)
* fileserver: Reject ADS and short name paths

* caddyhttp: Trim trailing space and dot on Windows

Windows ignores trailing dots and spaces in filenames.

* Fix test

* Adjust path filters

* Revert Windows test

* Actually revert the test

* Just check for colons
2022-10-18 21:55:25 -06:00
Scott Mebberson 72e7edda1f
map: Clarified how destination values should be formatted (#5156) 2022-10-18 18:14:53 -06:00
Francis Lavoie 1cd594963e
docs: Fix templates documentation, stray newline breaks godoc (#5149) 2022-10-16 12:25:44 -04:00
Matt Holt 6bad878a22
httpcaddyfile: Improve detection of indistinguishable TLS automation policies (#5120)
* httpcaddyfile: Skip some logic if auto_https off

* Try removing this check altogether...

* Refine test timeouts slightly, sigh

* caddyhttp: Assume udp for unrecognized network type

Seems like the reasonable thing to do if a plugin registers its own
network type.

* Add comment to document my lack of knowledge

* Clean up and prepare to merge

Add comments to try to explain what happened
2022-10-13 11:30:57 -06:00
Matt Holt 3e1fd2a8d4
httpcaddyfile: Wrap site block in subroute if host matcher used (#5130)
* httpcaddyfile: Wrap site block in subroute if host matcher used (fix #5124)

* Correct boolean logic (oops)
2022-10-12 09:27:08 -06:00
Abdussamet Koçak 33f60da9f2
fileserver: stop listing dir when request context is cancelled (#5131)
Prevents caddy from performing disk IO needlessly when the request is cancelled before the listing is finished.

Closes #5129
2022-10-08 12:56:35 -06:00
Francis Lavoie 99ffe93388
logging: Fix `skip_hosts` with wildcards (#5102)
Fix #4859
2022-10-05 12:14:13 -06:00
Matthew Holt e07a267276
caddytest: Revise sleep durations
Attempt to reduce flakiness a bit more

Test suite needs to be rewritten.
2022-10-05 11:40:41 -06:00
Matt Holt 2153a81ec8
forwardauth: Canonicalize header fields (fix #5038) (#5097) 2022-10-05 01:37:01 -04:00
Francis Lavoie ea58d51907
logging: Perform filtering on arrays of strings (where possible) (#5101)
* logging: Perform filtering on arrays of strings (where possible)

* Add test for ip_mask filter

* Oops, need to continue when it's not an IP

* Test for invalid IPs
2022-10-04 23:21:23 -06:00
Francis Lavoie 9e1d964bd6
logging: Add `time_local` option to use local time instead of UTC (#5108) 2022-10-05 00:23:14 -04:00
xufanglu 2be56c526c
fileserver: Treat invalid file path as NotFound (#5099)
treat invalid file path as notFound so that PassThru can work
2022-10-04 21:32:40 -06:00
Francis Lavoie 01e192edc9
logging: Better `console` encoder defaults (#5109)
This is something that has bothered me for a while, so I figured I'd do something about it now since I'm playing in the logging code lately.

The `console` encoder doesn't actually match the defaults that zap's default logger uses. This makes it match better with the rest of the logs when using the `console` encoder alongside somekind of filter, which requires you to configure an encoder to wrap.
2022-10-04 21:18:48 -06:00
iliana etaoin b1d04f5b39
fileserver: better dark mode visited link contrast (#5105)
PR #4066 added a dark color scheme to the file_server browse template.
PR #4356 later set the links for the `:visited` pseudo-class, but did
not set anything for the dark mode, resulting in poor contrast. I
selected some new colors by feel.

This commit also adds an `a:visited:hover` for both, to go along with
the normal blue hover colors.
2022-10-01 18:14:27 -06:00
Matthew Holt 9873ff9918
caddyhttp: Remote IP prefix placeholders
See https://github.com/mholt/caddy-ratelimit/issues/12
2022-09-30 13:29:33 -06:00
Matt Holt 5e52bbb136
map: Remove infinite recursion check (#5094)
It was not accurate. Placeholders could be used in outputs that are
defined in the same mapping as long as that placeholder does not do the
same.

A more general solution would be to detect it at run-time in the
replacer directly, but that's a bit tedious
and will require allocations I think.

A better implementation of this check could still be done, but I don't
know if it would always be accurate. Could be a "best-effort" thing?
But I've also never heard of an actual case where someone configured
infinite recursion...
2022-09-29 12:46:38 -06:00
Matthew Holt fcdbc69fab
Fix comment
I apparently read the diff backwards in
2a8c458ffe
2022-09-29 12:38:36 -06:00
Matthew Holt 2a8c458ffe
reverseproxy: Parse humanized byte size (fix #5095) 2022-09-29 12:37:06 -06:00
Matt Holt e2991eb019
reverseproxy: On 103 don't delete own headers (#5091)
See #5074
2022-09-29 08:19:56 -06:00
Matt Holt 897a38958c
Merge pull request #5076 from caddyserver/fastcgi-redir
fastcgi: Redirect using original URI path (fix #5073) and rewrite: Only trim prefix if matched
2022-09-28 15:22:45 -06:00
Will Norris 61822f129b
caddyhttp: replace placeholders in map defaults (#5081)
This updates the map directive to replace placeholders in default values
in the same way as matched values.
2022-09-28 13:38:20 -06:00
Matt Holt e3e8aabbcf
core: Refactor and improve listener logic (#5089)
* core: Refactor, improve listener logic

Deprecate:
- caddy.Listen
- caddy.ListenTimeout
- caddy.ListenPacket

Prefer caddy.NetworkAddress.Listen() instead.

Change:
- caddy.ListenQUIC (hopefully to remove later)
- caddy.ListenerFunc signature (add context and ListenConfig)

- Don't emit Alt-Svc header advertising h3 over HTTP/3

- Use quic.ListenEarly instead of quic.ListenEarlyAddr; this gives us
more flexibility (e.g. possibility of HTTP/3 over UDS) but also
introduces a new issue:
https://github.com/lucas-clemente/quic-go/issues/3560#issuecomment-1258959608

- Unlink unix socket before and after use

* Appease the linter

* Keep ListenAll
2022-09-28 13:35:51 -06:00
Matthew Holt 013b510352
rewrite: Only trim prefix if matched
See #5073
2022-09-28 00:13:12 -06:00
lemmi d0556929a4
reverseproxy: fix upstream scheme handling in command (#5088)
e338648fed introduced multiple upstream
addresses. A comment notes that mixing schemes isn't supported and
therefore the first valid scheme is supposed to be used.

Fixes setting the first scheme.

fixes #5087
2022-09-27 13:03:30 -06:00
Matthew Holt 7041970059
headers: Support repeated WriteHeader if 1xx (fix #5074) 2022-09-23 17:11:53 -06:00
Matthew Holt f7c1a51efb
fastcgi: Redirect using original URI path (fix #5073) 2022-09-23 14:36:38 -06:00
Matthew Holt 9206e8a738 Tweak some comments 2022-09-21 12:59:44 -06:00
Matt Holt 1426c97da5
core: Reuse unix sockets (UDS) and don't try to serve HTTP/3 over UDS (#5063)
* core: Reuse unix sockets

* Don't serve HTTP/3 over unix sockets

This requires upstream support, if even useful

* Don't use unix build tag... yet

* Fix build tag

* Allow ErrNotExist when unlinking socket
2022-09-21 12:55:23 -06:00
WeidiDeng 44ad0cedaf
encode: don't WriteHeader unless called (#5060) 2022-09-21 08:30:42 -06:00
Matthew Holt beb7dcbf2a
fileserver: Reinstate --debug flag
I think it got lost during a rebase or something
2022-09-20 16:56:02 -06:00
Francis Lavoie e3d04ff86b
caddyhttp: Skip inserting HTTP->HTTPS redir if catch-all for both exist (#5051) 2022-09-19 22:11:19 -06:00
Matt Holt da8b7fe58f
caddyhttp: Honor grace period in background (#5043)
* caddyhttp: Honor grace period in background

This avoids blocking during config reloads.

* Don't quit process until servers shut down

* Make tests more likely to pass on fast CI (#5045)

* caddyhttp: Even faster shutdowns

Simultaneously shut down all HTTP servers, rather than one at a time.

In practice there usually won't be more than 1 that lingers. But this
code ensures that they all Shutdown() in their own goroutine
and then we wait for them at the end (if exiting).

We also wait for them to start up so we can be fairly confident the
shutdowns have begun; i.e. old servers no longer
accepting new connections.

* Fix comment typo

* Pull functions out of loop, for readability
2022-09-19 21:54:47 -06:00
Matthew Holt 0950ba4f0b
events: Make event data exported
This could lead to bugs if handlers are not careful, but it is surely
useful. We'll see how it goes, what the feedback is like, etc.
2022-09-19 16:20:58 -06:00
WeidiDeng c7a6bc5934
caddyhttp: responseRecorder save status in all cases (#5049) 2022-09-17 18:47:53 -06:00
Matthew Holt 00beec2e34
caddyhttp: Fix write header on responseRecorder 2022-09-17 11:28:13 -06:00
Matthew Holt e43b6d8178 core: Variadic Context.Logger(); soft deprecation
Ideally I'd just remove the parameter to caddy.Context.Logger(), but
this would break most Caddy plugins.

Instead, I'm making it variadic and marking it as partially deprecated.
In the future, I might completely remove the parameter once most
plugins have updated.
2022-09-16 16:55:36 -06:00
WeidiDeng bffc258732
caddyhttp: Support configuring Server from handler provisioning (#4933)
* configuring http.Server from handlers.

* Minor tweaks

* Run gofmt

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-09-16 14:48:55 -06:00
David Manouchehri 616418281b
caddyhttp: Support TLS key logging for debugging (#4808)
* Add SSL key logging.

* Resolve merge conflict with master

* Add Caddyfile support; various fixes

* Also commit go.mod and go.sum, oops

* Appease linter

* Minor tweaks

* Add doc comment

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-16 14:05:37 -06:00
Matt Holt 74547f5bed
caddyhttp: Make metrics opt-in (#5042)
* caddyhttp: Make metrics opt-in

Related to #4644

* Make configurable in Caddyfile
2022-09-16 13:32:49 -06:00
Matthew Holt 258071d857
caddytls: Debug log on implicit tailscale error (#5041) 2022-09-16 09:42:05 -06:00
Matthew Holt b6cec37893
caddyhttp: Add --debug flag to commands
file-server and reverse-proxy

This might be useful!
2022-09-15 23:10:16 -06:00
WeidiDeng 48d723c07c
encode: Fix Accept-Ranges header; HEAD requests (#5039)
* fix encode handler header manipulation
also avoid implementing ReadFrom because it breaks when io.Copied to directly

* strconv.Itoa should be tried as a last resort
WriteHeader during Close
2022-09-15 16:05:08 -06:00
Isaac Parker e338648fed
reverseproxy: Support repeated --to flags in command (#4693)
* feat: Multiple 'to' upstreams in reverse-proxy cmd

* Repeat --to for multiple upstreams, rather than comma-separating in a single flag

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-15 12:35:38 -06:00
Francis Lavoie 9ad0ebc956
caddyhttp: Add 'skip_log' var to omit request from logs (#4691)
* caddyhttp: Implement `skip_log` handler

* Refactor to use vars middleware

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-15 10:05:36 -06:00
Matthew Holt 754fe4f7b4
httpcaddyfile: Fix sorting of repeated directives
Fixes #5037
2022-09-13 13:43:21 -06:00
Matthew Holt 20d487be57
caddyhttp: Very minor optimization to path matcher
If * is in the matcher it will always match so we can just put it first.
2022-09-13 11:26:10 -06:00
Francis Lavoie 61c75f74de
caddyhttp: Explicitly disallow multiple regexp matchers (#5030)
* caddyhttp: Explicitly disallow multiple regexp matchers

Fix #5028

Since the matchers would overwrite eachother, we should error out to tell the user their config doesn't make sense.

* Update modules/caddyhttp/matchers.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-13 11:18:37 -06:00
Matthew Holt d35f618b10
caddytls: Error if placeholder is empty in 'ask'
Fixes #5036
2022-09-13 08:59:03 -06:00
Matthew Holt 076a8b8095
Very minor tweaks 2022-09-08 13:10:40 -06:00
Matthew Holt c19f207237
fileserver: Ignore EOF when browsing empty dir
Thanks to @WeidiDeng for reporting this
2022-09-07 21:14:11 -06:00
fleandro dd9813c65b
caddyhttp: ensure ResponseWriterWrapper and ResponseRecorder use ReadFrom if the underlying response writer implements it. (#5022)
Doing so allows for splice/sendfile optimizations when available.
Fixes #4731

Co-authored-by: flga <flga@users.noreply.github.com>
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-09-07 21:13:35 +01:00
Dave Henderson fded2644f8
Drop requirement for filesystems to implement fs.StatFS
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2022-09-05 19:25:34 -04:00
Matt Holt d5ea43fb4b
fileserver: Support glob expansion in file matcher (#4993)
* fileserver: Support glob expansion in file matcher

* Fix tests

* Fix bugs and tests

* Attempt Windows fix, sigh

* debug Windows, WIP

* Continue debugging Windows

* Another attempt at Windows

* Plz Windows

* Cmon...

* Clean up, hope I didn't break anything
2022-09-05 13:53:41 -06:00
Matt Holt ca4fae64d9
caddyhttp: Support `respond` with HTTP 103 Early Hints (#5006)
* caddyhttp: Support sending HTTP 103 Early Hints

This adds support for early hints in the static_response handler.

* caddyhttp: Don't record 1xx responses
2022-09-05 13:50:44 -06:00
Matthew Holt ad69503aef
Remove unnecessary error check 2022-09-05 13:42:59 -06:00
Francis Lavoie 6e3063b15a
caddyauth: Speed up basicauth provision, deprecate scrypt (#4720)
* caddyauth: Speed up basicauth provisioning, precalculate fake password

* Deprecate scrypt, allow using decoded bcrypt hashes

* Add TODO note

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-05 13:32:58 -06:00
Matt Holt 66476d8c8f
reverseproxy: Close hijacked conns on reload/quit (#4895)
* reverseproxy: Close hijacked conns on reload/quit

We also send a Close control message to both ends of
WebSocket connections. I have tested this many times in
my dev environment with consistent success, although
the variety of scenarios was limited.

* Oops... actually call Close() this time

* CloseMessage --> closeMessage

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Use httpguts, duh

* Use map instead of sync.Map

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-09-02 17:01:55 -06:00
Matt Holt d3c3fa10bd
core: Refactor listeners; use SO_REUSEPORT on Unix (#4705)
* core: Refactor listeners; use SO_REUSEPORT on Unix

Just an experiment for now

* Fix lint by logging error

* TCP Keepalive configuration (#4865)

* initial attempt at TCP Keepalive configuration

* core: implement tcp-keepalive for linux

* move canSetKeepAlive interface

* Godoc for keepalive server parameter

* handle return values

* log keepalive errors

* Clean up after bad merge

* Merge in pluggable network types

From 1edc1a45e3

* Slight refactor, fix from recent merge conflict

Co-authored-by: Karmanyaah Malhotra <karmanyaah.gh@malhotra.cc>
2022-09-02 16:59:11 -06:00
WeidiDeng 83b26975bd
fastcgi: Optimize FastCGI transport (#4978)
* break up code and use lazy reading and pool bufio.Writer

* close underlying connection when operation failed

* allocate bufWriter and streamWriter only once

* refactor record writing

* rebase from master

* handle err

* Fix type assertion

Also reduce some duplication

* Refactor client and clientCloser for logging

Should reduce allocations

* Minor cosmetic adjustments; apply Apache license

* Appease the linter

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-09-02 16:57:55 -06:00
Matthew Holt 005c5a6382
Minor style adjustments for HTTP redir logging 2022-09-02 13:04:31 -06:00
Avdhut 66959d9f18
templates: Document `httpError` function (#4972)
* added the httpError function into the document

* Update templates.go

* Update templates.go

* Fix gofmt

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-01 22:07:52 -06:00
fleandro f2a7e7c966
fastcgi: allow users to log stderr output (#4967) (#5004)
Co-authored-by: flga <flga@users.noreply.github.com>
2022-09-02 00:02:48 -04:00
Francis Lavoie d062fb4020
caddyhttp: Copy logger config to HTTP server during AutoHTTPS (#4990) 2022-09-01 23:31:54 -04:00
Matthew Holt 73d4a8ba02 map: Coerce val to string, fix #4987
Also prevent infinite recursion, and enforce placeholder syntax.
2022-09-01 21:15:44 -06:00
Matthew Holt 7c35bfa57c
caddyhttp: Accept placeholders in vars matcher key
Until now, the vars matcher has unintentionally lacked parity with the
map directive: the destination placeholders of the map directive would
be expressed as placeholders, i.e. {foo}. But the vars matcher would
not use { }: vars foo value

This looked weird, and was confusing, since it implied that the key
could be dynamic, which doesn't seem helpful here.

I think this is a proper bug fix, since we're not used to accessing
placeholders literally without { } in the Caddyfile.
2022-09-01 16:49:18 -06:00
Matt Holt 1edc1a45e3
core: Plugins can register listener networks (#5002)
* core: Plugins can register listener networks

This can be useful for custom listeners.

This feature/API is experimental and may change!

* caddyhttp: Expose server listeners
2022-09-01 16:30:03 -06:00
Matthew Holt cb849bd664
caddyhttp: Disable draft versions of QUIC
See comment in #4996
2022-08-31 18:49:34 -06:00
Matthew Holt 3cd7437b3d
events: Tune logging and context cancellation 2022-08-31 18:48:46 -06:00
Francis Lavoie d4d8bbcfc6
events: Implement event system (#4912)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-08-31 15:01:30 -06:00
Matt Holt 2d5a30b908
caddyhttp: Set Content-Type for static response (#4999) 2022-08-31 09:43:46 -06:00
Matthew Holt 8cb3cf540c Minor cleanup, resolve a couple lint warnings 2022-08-29 12:31:56 -06:00
Abirdcfly e1801fdb19
Remove duplicate words in comments (#4986) 2022-08-27 14:39:26 -06:00
Dávid Szabó 0c57facc67
reverseproxy: Add upstreams healthy metrics (#4935) 2022-08-27 12:30:23 -06:00
Matthew Holt 5fb5b81439
reverseproxy: Multiple dynamic upstreams
This allows users to, for example, get upstreams from multiple SRV
endpoints in order (such as primary and secondary clusters).

Also, gofmt went to town on the comments, sigh
2022-08-25 21:42:48 -06:00
Matthew Holt 2cc5d38229
Fix comment indentation 2022-08-25 13:28:58 -06:00
Simon Legner 66596f2d74
zstd: fix typo in comment (#4985) 2022-08-25 12:00:05 +03:00
Matthew Holt 3aabbc49a2 caddytls: Log error if ask request fails
Errors returned from the DecisionFunc (whether to get a cert on-demand)
are used as a signal whether to allow a cert or not; *any* error
will forbid cert issuance.

We bubble up the error all the way to the caller, but that caller is the
Go standard library which might gobble it up.
Now we explicitly log connection errors so sysadmins can
ensure their ask endpoints are working.

Thanks to our sponsor AppCove for reporting this!
2022-08-23 22:28:15 -06:00
jedy e289ba6187
templates: cap of slice should not be smaller than length (#4975) 2022-08-23 08:26:02 -06:00
Francis Lavoie a22c08a638
caddyhttp: Fix for nil `handlerErr.Err` (#4977) 2022-08-23 08:17:46 -06:00
Francis Lavoie 72541f1cb8
caddyhttp: Set `http.error.message` to the HandlerError message (#4971) 2022-08-22 23:31:07 -06:00
WilczyńskiT c7772588bd
core: Change net.IP to netip.Addr; use netip.Prefix (#4966)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-08-17 16:10:57 -06:00
Matthew Holt a944de4ab7
Clean up metrics test code
No need to use != for booleans
2022-08-16 10:03:19 -06:00
Matt Holt a479943acd
caddyhttp: Smarter path matching and rewriting (#4948)
Co-authored-by: RussellLuo <luopeng.he@gmail.com>
2022-08-16 08:48:57 -06:00
Abdussamet Koçak dc62d468e9
fileserver: reset buffer before using it (#4962) (#4963) 2022-08-15 22:31:45 -06:00
Matt Holt c79c08627d
caddyhttp: Enable HTTP/3 by default (#4707) 2022-08-15 12:01:58 -06:00
Francis Lavoie e2a5e2293a
reverseproxy: Add `unix+h2c` Caddyfile network shortcut (#4953) 2022-08-12 17:09:18 -04:00
Matt Holt f5dce84a70
reverseproxy: Ignore context cancel in stream mode (#4952) 2022-08-12 13:15:41 -06:00
Francis Lavoie 922d9f5c25
reverseproxy: Fix H2C dialer using new stdlib `DialTLSContext` (#4951) 2022-08-12 13:11:13 -06:00
Kévin Dunglas 085df25c7e
reverseproxy: Support 1xx status codes (HTTP early hints) (#4882) 2022-08-09 10:53:24 -06:00
Francis Lavoie fe61209df2
logging: Fix `cookie` filter (#4943) 2022-08-08 19:11:02 -06:00
Matthew Holt 7ab61f46f0
fileserver: Better fix for Etag of compressed files 2022-08-08 13:09:57 -06:00
Matthew Holt 8c72f34357
fileserver: Generate Etag from sidecar file
Don't use the primary/uncompressed file for Etag when serving sidecars.

This was just overlooked initially.
2022-08-08 12:50:06 -06:00
Matthew Holt b9618b8b98
Improve docs for ZeroSSL issuer 2022-08-08 12:50:06 -06:00
Chirag Maheshwari d26559316f
Replace strings.Index with strings.Cut (#4932) 2022-08-06 22:03:37 -06:00
WilczyńskiT 2642bd72b7
Replace strings.Index usages with strings.Cut (#4930) 2022-08-04 11:17:35 -06:00
Matt Holt 17ae5acaba
cmd: Use newly-available version information (#4931) 2022-08-04 11:16:59 -06:00
Matt Holt 1960a0dc11
httpserver: Configurable shutdown delay (#4906) 2022-08-03 11:04:51 -06:00
Francis Lavoie 141872ed80
chore: Bump up to Go 1.19, minimum 1.18 (#4925) 2022-08-02 16:39:09 -04:00
Matt Holt f783290f40
caddyhttp: Implement `caddy respond` command (#4870) 2022-08-01 13:36:22 -06:00
Matthew Holt ebd6abcbd5
fileserver: Support virtual file system in Caddyfile 2022-07-31 21:41:26 -06:00
Matt Holt 6668271661
fileserver: Support virtual file systems (#4909)
* fileserver: Support virtual file systems (close #3720)

This change replaces the hard-coded use of os.Open() and os.Stat() with
the use of the new (Go 1.16) io/fs APIs, enabling virtual file systems.
It introduces a new module namespace, caddy.fs, for such file systems.

Also improve documentation for the file server. I realized it was one of
the first modules written for Caddy 2, and the docs hadn't really been
updated since!

* Virtualize FS for file matcher; minor tweaks

* Fix tests and rename dirFS -> osFS

(Since we do not use a root directory, it is dynamic.)
2022-07-30 13:07:44 -06:00
WingLim 1e0cdc54f8
core: Windows service integration (#4790)
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-07-29 14:06:54 -06:00
Matthew Holt 35a81d7c5b
Ignore linter warnings
Use of non-cryptographic random numbers in the load balancing
is intentional.
2022-07-28 15:40:23 -06:00
Francis Lavoie ff2ba6de8a
caddyhttp: Clear out matcher error immediately after grabbing it (#4916)
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-07-28 15:19:48 -06:00
Matthew Holt 4fced0b6e1
Finish fixing lint errors from ea8df6ff
Follows up #4915
2022-07-28 15:16:36 -06:00
Matthew Holt 1bdd451913
caddytls: Remove PreferServerCipherSuites
It has been deprecated by Go
2022-07-28 14:50:51 -06:00
Matthew Holt ea8df6ff11
caddyhttp: Use new CEL APIs (fix #4915)
Hahaha this is the ultimate "I have no idea what I'm doing" commit but it
compiles and the tests pass and I declare victory!

... probably broke something, should be tested more.

It is nice that the protobuf dependency becomes indirect now.
2022-07-28 14:50:28 -06:00
Matthew Holt 0bebea0d4c
caddyhttp: Log shutdown errors, don't return (fix #4908) 2022-07-25 10:39:59 -06:00
Matt Holt a379fa4c6c
reverseproxy: Implement read & write timeouts for HTTP transport (#4905) 2022-07-23 22:38:41 -06:00
Francis Lavoie abad9bc256
cmd: Fix reload with stdin (#4900) 2022-07-20 18:14:33 -06:00
Matthew Holt 8bdee04651
caddyhttp: Enhance comment 2022-07-16 23:33:49 -06:00
Francis Lavoie 7d1f7771c9
reverseproxy: Implement retry count, alternative to try_duration (#4756)
* reverseproxy: Implement retry count, alternative to try_duration

* Add Caddyfile support for `retry_match`

* Refactor to deduplicate matcher parsing logic

* Fix lint
2022-07-13 14:15:00 -06:00
Matthew Holt 04a14ee37a
caddyhttp: Make query matcher more efficient
Only parse query string once
2022-07-13 12:20:00 -06:00
Matthew Holt c2bbe42fc3
reverseproxy: Export SetScheme() again
Turns out the NTLM transport uses it. Oops.
2022-07-13 08:52:30 -06:00
Francis Lavoie 53c4d788d4
headers: Only replace known placeholders (#4880) 2022-07-12 12:16:03 -06:00
Matthew Holt d6bc9e0b5c
reverseproxy: Err 503 if all upstreams unavailable 2022-07-08 13:01:32 -06:00
Francis Lavoie 54d1923ccb
reverseproxy: Adjust new TLS Caddyfile directive names (#4872) 2022-07-08 13:04:22 -04:00
Matthew Holt c0f76e9ed4
fileserver: Use safe redirects in file browser 2022-07-07 14:10:19 -06:00
Francis Lavoie 8bac134f26
go.mod: Bump up quic-go to v0.28.0, fixes for BC breaks (#4867) 2022-07-06 12:14:32 -06:00
Matt Holt 412dcc07d3
caddytls: Reuse issuer between PreCheck and Issue (#4866)
This enables EAB reuse for ZeroSSLIssuer (which is now supported by ZeroSSL).
2022-07-05 18:12:25 -06:00
Francis Lavoie 58e05cab15
forwardauth: Fix case when `copy_headers` is omitted (#4856)
See https://caddy.community/t/using-forward-auth-and-writing-my-own-authenticator-in-php/16410, apparently it didn't work when `copy_headers` wasn't used. This is because we were skipping adding a handler to the routes in the "good response handler", but this causes the logic in `reverseproxy.go` to ignore the response handler since it's empty. Instead, we can just always put in the `header` handler, even with an empty `Set` operation, it's just a no-op, but it fixes that condition in the proxy code.
2022-06-28 19:23:30 -06:00
Tristan Swadell 10f85558ea
Expose several Caddy HTTP Matchers to the CEL Matcher (#4715)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-06-22 18:53:46 -04:00
Francis Lavoie 98468af8b6
reverseproxy: Fix double headers in response handlers (#4847) 2022-06-22 15:10:14 -04:00
Francis Lavoie 25f10511e7
reverseproxy: Fix panic when TLS is not configured (#4848)
* reverseproxy: Fix panic when TLS is not configured

* Refactor and simplify setScheme

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-06-22 15:01:57 -04:00
Kiss Károly Pál b6e96fa3c5
reverseproxy: Skip TLS for certain configured ports (#4843)
* Make reverse proxy TLS server name replaceable for SNI upstreams.

* Reverted previous TLS server name replacement, and implemented thread safe version.

* Move TLS servername replacement into it's own function

* Moved SNI servername replacement into httptransport.

* Solve issue when dynamic upstreams use wrong protocol upstream.

* Revert previous commit.

Old commit was: Solve issue when dynamic upstreams use wrong protocol upstream.
Id: 3c9806ccb6

* Added SkipTLSPorts option to http transport.

* Fix typo in test config file.

* Rename config option as suggested by Matt

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update code to match renamed config option.

* Fix typo in config option name.

* Fix another typo that I missed.

* Tests not completing because of apparent wrong ordering of options.

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-20 11:51:42 -06:00
Francis Lavoie 0b6f764356
forwardauth: Support renaming copied headers, block support (#4783) 2022-06-16 14:28:11 -06:00
Matt Holt 0bcd02d5f6
headers: Support wildcards for delete ops (close #4830) (#4831) 2022-06-15 09:57:43 -06:00
Kiss Károly Pál c82fe91104
reverseproxy: Dynamic ServerName for TLS upstreams (#4836)
* Make reverse proxy TLS server name replaceable for SNI upstreams.

* Reverted previous TLS server name replacement, and implemented thread safe version.

* Move TLS servername replacement into it's own function

* Moved SNI servername replacement into httptransport.

* Solve issue when dynamic upstreams use wrong protocol upstream.

* Revert previous commit.

Old commit was: Solve issue when dynamic upstreams use wrong protocol upstream.
Id: 3c9806ccb6

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-14 21:53:05 -06:00
Matthew Holt f9b42c3772
reverseproxy: Make TLS renegotiation optional 2022-06-14 09:05:25 -06:00
Yaacov Akiba Slama aaf6794b31
reverseproxy: Add renegotiation param in TLS client (#4784)
* Add renegotiation option in reverseproxy tls client

* Update modules/caddyhttp/reverseproxy/httptransport.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-10 09:33:35 -06:00
Matthew Holt 1498132ea3
caddyhttp: Log error from CEL evaluation (fix #4832) 2022-06-08 16:42:24 -06:00
Francis Lavoie 7f9b1f43c9
reverseproxy: Correct the `tls_server_name` docs (#4827)
* reverseproxy: Correct the `tls_server_name` docs

* Update modules/caddyhttp/reverseproxy/httptransport.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-06 12:37:09 -06:00
Matt Holt 5e729c1e85
reverseproxy: HTTP 504 for upstream timeouts (#4824)
Closes #4823
2022-06-03 14:13:47 -06:00
Gr33nbl00d 0a14f97e49
caddytls: Make peer certificate verification pluggable (#4389)
* caddytls: Adding ClientCertValidator for custom client cert validations

* caddytls: Cleanups for ClientCertValidator changes

caddytls: Cleanups for ClientCertValidator changes

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Unexported field Validators, corrected renaming of LeafVerificationValidator to LeafCertClientAuth

* admin: Write proper status on invalid requests (#4569) (fix #4561)

* Apply suggestions from code review

* Register module; fix compilation

* Add log for deprecation notice

Co-authored-by: Roettges Florian <roettges.florian@scheidt-bachmann.de>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Alok Naushad <alokme123@gmail.com>
2022-06-02 14:25:07 -06:00
Matthew Holt 9864b138fb
reverseproxy: api: Remove misleading 'healthy' value
In v2.5.0, upstream health was fixed such that whether an upstream is
considered healthy or not is mostly up to each individual handler's
config. Since "healthy" is an opinion, it is not a global value.

I unintentionally left in the "healthy" field in the API endpoint for
checking upstreams, and it is now misleading (see #4792).

However, num_requests and fails remains, so health can be determined by
the API client, rather than having it be opaquely (and unhelpfully)
determined for the client.

If we do restore this value later on, it'd need to be replicated once
per reverse_proxy handler according to their individual configs.
2022-06-02 12:32:23 -06:00
Matthew Holt 3d18bc56b9
go.mod: Update go-yaml to v3 2022-06-01 15:15:20 -06:00
Matthew Holt 886ba84baa
Fix #4822 and fix #4779
The fix for 4822 is the change at the top of the file, and
4779's fix is toward the bottom of the file.
2022-06-01 15:12:57 -06:00
Alexander M a9267791c4
reverseproxy: Add --internal-certs CLI flag #3589 (#4817)
added flag --internal-certs
when set, for non-local domains the internal CA will be used for cert generation
2022-05-29 14:33:01 -06:00
Aleks 6891f7f421
templates: Add `humanize` function (#4767)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-05-24 19:47:08 -04:00
David Larlet 9e760e2e0c
templates: Documentation consistency (#4796) 2022-05-17 18:56:40 -04:00
Matt Holt 57d27c1b58
reverseproxy: Support http1.1>h2c (close #4777) (#4778) 2022-05-10 17:25:58 -04:00
Matthew Holt 693e9b5283
rewrite: Handle fragment before query (fix #4775) 2022-05-09 11:09:42 -06:00
Francis Lavoie f7be0ee101
map: Prevent output destinations overlap with Caddyfile shorthands (#4657) 2022-05-06 10:25:31 -06:00
Francis Lavoie f6900fcf53
reverseproxy: Support performing pre-check requests (#4739) 2022-05-06 10:50:26 -04:00
Francis Lavoie ec86a2f7a3
caddyfile: Shortcut for `remote_ip` for private IP ranges (#4753) 2022-05-04 12:42:37 -06:00
Francis Lavoie e7fbee8c82
reverseproxy: Permit resolver addresses to not specify a port (#4760)
Context: https://caddy.community/t/caddy-2-5-dynamic-upstreams-and-consul-srv-dns/15839

I realized it probably makes sense to allow `:53` to be omitted, since it's the default port for DNS.
2022-05-04 12:40:39 -06:00
Tyler Kropp e84e19a04e
templates: Add custom template function registration (#4757)
* Add custom template function registration

* Rename TemplateFunctions to CustomFunctions

* Add documentation

* Document CustomFunctions interface

* Preallocate custom functions map list

* Fix interface name in error message
2022-05-02 14:55:34 -06:00
Francis Lavoie 4a223f5203
reverseproxy: Fix Caddyfile support for `replace_status` (#4754) 2022-05-02 11:44:28 -06:00
Francis Lavoie 0be3d99543
logging: Implement rename filter, changes field key names (#4745) 2022-04-28 11:38:44 -04:00
Francis Lavoie dcc98da4d2
caddyhttp: Improve listen addr error message for IPv6 (#4740) 2022-04-28 08:18:45 -06:00
Marco Kaufmann 3ab648382d
templates: Add missing backticks in docs (#4737) 2022-04-27 11:41:37 -06:00
Matt Holt 40b193fb79
reverseproxy: Improve hashing LB policies with HRW (#4724)
* reverseproxy: Improve hashing LB policies with HRW

Previously, if a list of upstreams changed, hash-based LB policies
would be greatly affected because the hash relied on the position of
upstreams in the pool. Highest Random Weight or "rendezvous" hashing
is apparently robust to pool changes. It runs in O(n) instead of
O(log n), but n is very small usually.

* Fix bug and update tests
2022-04-27 10:39:22 -06:00
Francis Lavoie d543ad1ffd
caddypki: Fix `caddy trust` command to use the correct API endpoint (#4730) 2022-04-25 22:00:39 -06:00
Francis Lavoie 3a1e0dbf47
httpcaddyfile: Deprecate paths in site addresses; use zap logs (#4728) 2022-04-25 10:12:10 -06:00
Francis Lavoie 77a77c0219
caddytls: Add `propagation_delay`, support `propagation_timeout -1` (#4723) 2022-04-22 16:09:11 -06:00
Francis Lavoie bc15b4b0e7
caddypki: Load intermediate for signing on-the-fly (#4669)
* caddypki: Load intermediate for signing on-the-fly

Fixes #4517

Big thanks to @maraino for adding an API in `smallstep/certificates` so that we can fix this

* Debug log

* Trying a hunch, does it need to be a pointer receiver?

* Clarify pointer receiver

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-04-13 10:20:42 -06:00
Francis Lavoie 00234c8ac2
templates: Switch to `BurntSushi/toml` (#4700) 2022-04-12 13:48:42 -06:00
Francis Lavoie 3e3bb00265
reverseproxy: Add `_ms` placeholders for proxy durations (#4666)
* reverseproxy: Add `_ms` placeholders for proxy durations

* Add http.request.duration_ms

Also add comments, and change duration_sec to duration_ms

* Add response.duration_ms for consistency

* Add missing godoc comment

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-04-11 13:04:05 -06:00
Francis Lavoie e4ce40f8ff
reverseproxy: Sync up `handleUpgradeResponse` with stdlib (#4664)
* reverseproxy: Sync up `handleUpgradeResponse` with stdlib

I had left this as a TODO for when we bump to minimum 1.17, but I should've realized it was under `internal` so it couldn't be used directly.

Copied the functions we needed for parity. Hopefully this is ok!

* Add tests and fix godoc comments

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-04-11 12:49:56 -06:00
Y.Horie afca242111
staticfiles: Expand placeholder for index files (#4679) 2022-04-07 15:01:09 -06:00
Francis Lavoie 7d229665ed
logging: Caddyfile support for `duration_format` (#4684)
Somehow, this was missed. Oops!
2022-04-07 11:23:28 -06:00
Matthew Holt d06d0e79f8
go.mod: Upgrade CertMagic to v0.16.0
Includes several breaking changes; code base updated accordingly.

- Added lots of context arguments
- Use fs.ErrNotExist
- Rename ACMEManager -> ACMEIssuer; CertificateManager -> Manager
2022-03-25 11:28:54 -06:00
Matthew Holt b8dbecb841
reverseproxy: Include port in A upstreams cache
Should fix #4659
2022-03-24 10:44:36 -06:00
Artem Mikheev c9b5e7f77b
Fix http3 servers dying after reload (#4654) 2022-03-22 19:47:57 -04:00
Matthew Holt 79cbe7bfd0
httpcaddyfile: Add 'vars' directive
See discussion in #4650
2022-03-22 10:47:21 -06:00
Matthew Holt 55b4c12e04
map: Evaluate placeholders in output vals (#4650) 2022-03-21 17:05:38 -06:00
Matthew Holt 2196c92c0e
reverseproxy: Don't clear name in SRV upstreams
Fix for dc4d147388
2022-03-21 08:33:24 -06:00
Francis Lavoie c5fffb4ac2
caddyfile: Support for raw token values; improve `map`, `expression` (#4643)
* caddyfile: Support for raw token values, improve `map`, `expression`

* Applied code review comments

* Rename RawVal to ValRaw

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-18 15:08:23 -06:00
Matthew Holt dc4d147388
reverseproxy: Expand SRV/A addrs for cache key
Hopefully fix #4645
2022-03-18 13:42:29 -06:00
Matthew Holt 93c99f6734
map: Support numeric and bool types with Caddyfile
Based on caddyserver/website#221
2022-03-17 17:53:32 -06:00
Francis Lavoie a9c7e94a38
chore: Comment fixes (#4634) 2022-03-13 01:38:11 -05:00
Matthew Holt 3d616e8c6d
requestbody: Return HTTP 413 (fix #4558) 2022-03-11 12:34:55 -07:00
Mohammed Al Sahaf b82e22b459
caddyhttp: retain all values of vars matcher when specified multiple times (#4629) 2022-03-11 10:55:37 -05:00
Francis Lavoie c7d6c4cbb9
reverseproxy: copy_response and copy_response_headers for handle_response routes (#4391)
* reverseproxy: New `copy_response` handler for `handle_response` routes

Followup to #4298 and #4388.

This adds a new `copy_response` handler which may only be used in `reverse_proxy`'s `handle_response` routes, which can be used to actually copy the proxy response downstream. 

Previously, if `handle_response` was used (with routes, not the status code mode), it was impossible to use the upstream's response body at all, because we would always close the body, expecting the routes to write a new body from scratch.

To implement this, I had to refactor `h.reverseProxy()` to move all the code that came after the `HandleResponse` loop into a new function. This new function `h.finalizeResponse()` takes care of preparing the response by removing extra headers, dealing with trailers, then copying the headers and body downstream.

Since basically what we want `copy_response` to do is invoke `h.finalizeResponse()` at a configurable point in time, we need to pass down the proxy handler, the response, and some other state via a new `req.WithContext(ctx)`. Wrapping a new context is pretty much the only way we have to jump a few layers in the HTTP middleware chain and let a handler pick up this information. Feels a bit dirty, but it works.

Also fixed a bug with the `http.reverse_proxy.upstream.duration` placeholder, it always had the same duration as `http.reverse_proxy.upstream.latency`, but the former was meant to be the time taken for the roundtrip _plus_ copying/writing the response.

* Delete the "Content-Length" header if we aren't copying

Fixes a bug where the Content-Length will mismatch the actual bytes written if we skipped copying the response, so we get a message like this when using curl:

```
curl: (18) transfer closed with 18 bytes remaining to read
```

To replicate:

```
{
	admin off
	debug
}

:8881 {
	reverse_proxy 127.0.0.1:8882 {
		@200 status 200
		handle_response @200 {
			header Foo bar
		}
	}
}

:8882 {
	header Content-Type application/json
	respond `{"hello": "world"}` 200
}
```

* Implement `copy_response_headers`, with include/exclude list support

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-03-09 11:00:51 -07:00
Andrii Kushch d0b608af31
tracing: New OpenTelemetry module (#4361)
* opentelemetry: create a new module

* fix imports

* fix test

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/tracer.go

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* rename error ErrUnsupportedTracesProtocol

* replace spaces with tabs in the test data

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* replace spaces with tabs in the README.md

* use default values for a propagation and exporter protocol

* set http attributes with helper

* simplify code

* Cleanup modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update link in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update documentation in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update link to naming spec in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename module from opentelemetry to tracing

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename span_name to span

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename span_name to span

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Simplify otel resource creation

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* handle extra attributes

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.opentelemetry.io/otel/semconv to 1.7.0

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.opentelemetry.io/otel version

* remove environment variable handling

* always use tracecontext,baggage as propagators

* extract tracer name into variable

* rename OpenTelemetry to Tracing

* simplify resource creation

* update go.mod

* rename package from opentelemetry to tracing

* cleanup tests

* update Caddyfile example in README.md

* update README.md

* fix test

* fix module name in README.md

* fix module name in README.md

* change names in README.md and tests

* order imports

* remove redundant tests

* Update documentation README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Fix grammar

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update comments

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update comments

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.sum

* update go.sum

* Add otelhttp instrumentation, update OpenTelemetry libraries.

* Use otelhttp instrumentation for instrumenting HTTP requests.

This change uses context.WithValue to inject the next handler into the
request context via a "nextCall" carrier struct, and pass it on to a
standard Go HTTP handler returned by otelhttp.NewHandler. The
underlying handler will extract the next handler from the context,
call it and pass the returned error to the carrier struct.

* use zap.Error() for the error log

* remove README.md

* update dependencies

* clean up the code

* change comment

* move serveHTTP method from separate file

* add syntax to the UnmarshalCaddyfile comment

* go import the file

* admin: Write proper status on invalid requests (#4569) (fix #4561)

* update dependencies

Co-authored-by: Dave Henderson <dhenderson@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Vibhav Pant <vibhavp@gmail.com>
Co-authored-by: Alok Naushad <alokme123@gmail.com>
Co-authored-by: Cedric Ziel <cedric@cedric-ziel.com>
2022-03-08 12:18:32 -07:00
Ran Chen d9b1d46325
caddytls: dns_challenge_override_domain for challenge delegation (#4596)
* Add a override_domain option to allow DNS chanllenge delegation

CNAME can be used to delegate answering the chanllenge to another DNS
zone. One usage is to reduce the exposure of the DNS credential [1].
Based on the discussion in caddy/certmagic#160, we are adding an option
to allow the user explicitly specify the domain to delegate, instead of
following the CNAME chain.

This needs caddy/certmagic#160.

* rename override_domain to dns_challenge_override_domain

* Update CertMagic; fix spelling

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-08 12:03:43 -07:00
Francis Lavoie c8f2834b51
fastcgi: Protect against requests with null bytes in the path (#4614) 2022-03-07 10:06:33 -07:00
Matt Holt ab0455922a
reverseproxy: Dynamic upstreams (with SRV and A/AAAA support) (#4470)
* reverseproxy: Begin refactor to enable dynamic upstreams

Streamed here: https://www.youtube.com/watch?v=hj7yzXb11jU

* Implement SRV and A/AAA upstream sources

Also get upstreams at every retry loop iteration instead of just once
before the loop. See #4442.

* Minor tweaks from review

* Limit size of upstreams caches

* Add doc notes deprecating LookupSRV

* Provision dynamic upstreams

Still WIP, preparing to preserve health checker functionality

* Rejigger health checks

Move active health check results into handler-specific Upstreams.

Improve documentation regarding health checks and upstreams.

* Deprecation notice

* Add Caddyfile support, use `caddy.Duration`

* Interface guards

* Implement custom resolvers, add resolvers to http transport Caddyfile

* SRV: fix Caddyfile `name` inline arg, remove proto condition

* Use pointer receiver

* Add debug logs

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-03-06 17:43:39 -07:00
Francis Lavoie c50094fc9d
reverseproxy: Implement trusted proxies for `X-Forwarded-*` headers (#4507) 2022-03-06 18:51:55 -05:00
Francis Lavoie d058dee11d
reverseproxy: Refactor dial address parsing, augment command parsing (#4616) 2022-03-05 16:34:19 -07:00
Francis Lavoie 09ba9e994e
fileserver: Add `pass_thru` Caddyfile option (#4613) 2022-03-04 20:50:05 -07:00
Matt Holt 2bb8550a4c
caddyhttp: Honor wildcard hosts in log SkipHosts (#4606) 2022-03-04 13:44:59 -07:00
Matthew Holt a6199cf814
templates: Fix docs for .Args 2022-03-03 11:12:37 -07:00
Francis Lavoie f5e104944e
reverseproxy: Make shallow-ish clone of the request (#4551)
* reverseproxy: Make shallow-ish clone of the request

* Refactor request cloning into separate function

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-03 09:54:45 -07:00
Matthew Holt 6b385a36f9
caddyhttp: Don't attempt to manage Tailscale certs
If .ts.net domains are explicitly added to config,
don't try to manage a cert for them (it will fail, and our
implicit Tailscale module will
get those certs at run-time).
2022-03-02 13:42:38 -07:00
Matthew Holt 9b7cdfa2f2
caddypki: Try to fix lint warnings 2022-03-02 13:38:05 -07:00
Matthew Holt 78e381b29f
caddypki: Refactor /pki/ admin endpoints
Remove /pki/certificates/<ca> endpoint and split into two endpoints:

- GET /pki/ca/<id> to get CA info and certs in JSON format
- GET /pki/ca/<id>/certificates to get cert in PEM chain
2022-03-02 13:00:37 -07:00
ttys3 de490c7cad
fastcgi: Set SERVER_PORT to 80 or 443 depending on scheme (#4572) 2022-03-02 11:24:16 -07:00
Francis Lavoie bbad6931e3
pki: Implement API endpoints for certs and `caddy trust` (#4443)
* admin: Implement /pki/certificates/<id> API

* pki: Lower "skip_install_trust" log level to INFO

See https://github.com/caddyserver/caddy/issues/4058#issuecomment-976132935

It's not necessary to warn about this, because this was an option explicitly configured by the user. Still useful to log, but we don't need to be so loud about it.

* cmd: Export functions needed for PKI app, return API response to caller

* pki: Rewrite `caddy trust` command to use new admin endpoint instead

* pki: Rewrite `caddy untrust` command to support using admin endpoint

* Refactor cmd and pki packages for determining admin API endpoint
2022-03-02 11:08:36 -07:00
BitWuehler ac14b64e08
caddyhttp: Support zone identifiers in remote_ip matcher (#4597)
* Update matchers.go

* Update matchers.go

* implementation of zone_id handling

* last changes in zone handling

* give return true values instead of bool

* Apply suggestions from code review

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* changes as suggested

* Apply suggestions from code review

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update matchers.go

* shortened the Match function

* changed mazcher handling

* Update matchers.go

* delete space

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-03-01 15:50:12 -07:00
Francis Lavoie 15c95e9d5b
fileserver: Canonical redir when whole path is stripped (#4549) 2022-03-01 15:32:39 -07:00
Francis Lavoie 87a1f228b4
reverseproxy: Move status replacement intercept to `replace_status` (#4300) 2022-03-01 14:12:43 -07:00