* caddyhttp: Make use of http.ResponseController
Also syncs the reverseproxy implementation with stdlib's which now uses ResponseController as well 2449bbb5e6
* Enable full-duplex for HTTP/1.1
* Appease linter
* Add warning for builds with Go 1.20, so it's less surprising to users
* Improved godoc for EnableFullDuplex, copied text from stdlib
* Only wrap in encode if not already wrapped
* update quic-go to v0.37.0
* Bump to Go 1.20
* Bump golangci-lint version, yml syntax consistency
* Use skip-pkg-cache workaround
* Workaround needed for both?
* Seeding weakrand is no longer necessary
---------
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* added weighted round robin algorithm to load balancer
* added an adapt integration test for wrr and fixed a typo
* changed args format to Caddyfile args convention
* added provisioner and validator for wrr
* simplified the code and improved doc
Allow registering a custom network mapping for HTTP/3. This is useful
if the original network for HTTP/1.1 and HTTP/2 is not a standard `unix`,
`tcp4`, or `tcp6` network. To keep backwards compatibility, we fall back
to `udp` if the original network is not registered in the mapping.
Fixes#5555
* Create an includeRaw template function to include a file without parsing it as a template.
Some formatting fixes
* Rename to readFile, various docs adjustments
---------
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Make grid entries take up full width on mobile and fix breadcrumb color issue in dark mode
Signed-off-by: Pistasj <odyssey346@disroot.org>
* Do mholt's suggestions
Signed-off-by: Pistasj <odyssey346@disroot.org>
---------
Signed-off-by: Pistasj <odyssey346@disroot.org>
When only a single request has the least amount of requests, there's no need to compute a random number, because the modulo of 1 will always be 0 anyways.
* Add isDir template function
* Update modules/caddyhttp/templates/tplcontext.go
Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* Fix funcIsDir return value on error
* Fix funcIsDir return false when root file system not specified
* Add stat function, remove isDir function
* Remove isDir function (really)
* Rename stat to fileStat
---------
Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* Serve http2 when listener wrapper doesn't return *tls.Conn
* close conn when h2server serveConn returns
* merge from upstream
* rebase from latest
* run New and Closed ConnState hook for h2 conns
* go fmt
* fix lint
* Add comments
* reorder import
* log: make `sink` encodable
* deduplicate logger fields
* extract common fields into `BaseLog` and embed it into `SinkLog`
* amend godoc on `BaseLog` and `SinkLog`
* minor style change
---------
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Delete all existing fields when fieldName is `*`
* Rearrange deletion before addition in headers
* Revert "Rearrange deletion before addition in headers"
This reverts commit 1b50eeeccc92ccd660c7896d8283c7d9e5d1fcb0.
* Treat deleting all headers as a special case
* Apply suggestions from code review
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
---------
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* caddyhttp: Determine real client IP if trusted proxies configured
* Support customizing client IP header
* Implement client_ip matcher, deprecate remote_ip's forwarded option
* cmd: Expand cobra support
* Convert commands to cobra, add short flags
* Fix version command typo
Co-authored-by: Emily Lange <git@indeednotjames.com>
* Apply suggestions from code review
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
---------
Co-authored-by: Emily Lange <git@indeednotjames.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* chore: Upgrade various dependencies
* Support CEL file matcher with no args
* Document `http.request.orig_uri.path.*`, reorder placeholders in docs
---------
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* added some tests for parseUpstreamDialAddress
Test 4 fails because it produces "[[::1]]:80" instead of "[::1]:80"
* support absolute windows path in unix reverse proxy address
* make IsUnixNetwork public, support +h2c and reuse it
* add new tests
* caddyauth: Add singleflight for basic auth
* Fixes#5338
* it occurred the thunder herd problem like this https://medium.com/@mhrlife/avoid-duplicate-requests-while-filling-cache-98c687879f59
* Update modules/caddyhttp/caddyauth/basicauth.go
Fix comment
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
---------
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* If upstreams are all using same host but with different ports
ie:
foobar:4001
foobar:4002
foobar:4003
...
Because fnv-1a has not a good enough avalanche effect
Then the hostByHashing result is not well balanced over
all upstreams
As last byte FNV input tend to affect few bits, the idea is to change
the concatenation order between the key and the upstream strings
So the upstream last byte have more impact on hash diffusion
This commit replaces the use of github.com/smallstep/cli to generate the
root and intermediate certificates and uses go.step.sm/crypto instead.
It also upgrades the version of github.com/smallstep/certificates to the
latest version.
* reverseproxy: Mask the WS close message when we're the client
* weakrand
* Bump golangci-lint version so path ignores work on Windows
* gofmt
* ugh, gofmt everything, I guess
* fileserver: Reject ADS and short name paths
* caddyhttp: Trim trailing space and dot on Windows
Windows ignores trailing dots and spaces in filenames.
* Fix test
* Adjust path filters
* Revert Windows test
* Actually revert the test
* Just check for colons
* httpcaddyfile: Skip some logic if auto_https off
* Try removing this check altogether...
* Refine test timeouts slightly, sigh
* caddyhttp: Assume udp for unrecognized network type
Seems like the reasonable thing to do if a plugin registers its own
network type.
* Add comment to document my lack of knowledge
* Clean up and prepare to merge
Add comments to try to explain what happened
* logging: Perform filtering on arrays of strings (where possible)
* Add test for ip_mask filter
* Oops, need to continue when it's not an IP
* Test for invalid IPs
This is something that has bothered me for a while, so I figured I'd do something about it now since I'm playing in the logging code lately.
The `console` encoder doesn't actually match the defaults that zap's default logger uses. This makes it match better with the rest of the logs when using the `console` encoder alongside somekind of filter, which requires you to configure an encoder to wrap.
PR #4066 added a dark color scheme to the file_server browse template.
PR #4356 later set the links for the `:visited` pseudo-class, but did
not set anything for the dark mode, resulting in poor contrast. I
selected some new colors by feel.
This commit also adds an `a:visited:hover` for both, to go along with
the normal blue hover colors.
It was not accurate. Placeholders could be used in outputs that are
defined in the same mapping as long as that placeholder does not do the
same.
A more general solution would be to detect it at run-time in the
replacer directly, but that's a bit tedious
and will require allocations I think.
A better implementation of this check could still be done, but I don't
know if it would always be accurate. Could be a "best-effort" thing?
But I've also never heard of an actual case where someone configured
infinite recursion...
* core: Refactor, improve listener logic
Deprecate:
- caddy.Listen
- caddy.ListenTimeout
- caddy.ListenPacket
Prefer caddy.NetworkAddress.Listen() instead.
Change:
- caddy.ListenQUIC (hopefully to remove later)
- caddy.ListenerFunc signature (add context and ListenConfig)
- Don't emit Alt-Svc header advertising h3 over HTTP/3
- Use quic.ListenEarly instead of quic.ListenEarlyAddr; this gives us
more flexibility (e.g. possibility of HTTP/3 over UDS) but also
introduces a new issue:
https://github.com/lucas-clemente/quic-go/issues/3560#issuecomment-1258959608
- Unlink unix socket before and after use
* Appease the linter
* Keep ListenAll
e338648fed introduced multiple upstream
addresses. A comment notes that mixing schemes isn't supported and
therefore the first valid scheme is supposed to be used.
Fixes setting the first scheme.
fixes#5087
* caddyhttp: Honor grace period in background
This avoids blocking during config reloads.
* Don't quit process until servers shut down
* Make tests more likely to pass on fast CI (#5045)
* caddyhttp: Even faster shutdowns
Simultaneously shut down all HTTP servers, rather than one at a time.
In practice there usually won't be more than 1 that lingers. But this
code ensures that they all Shutdown() in their own goroutine
and then we wait for them at the end (if exiting).
We also wait for them to start up so we can be fairly confident the
shutdowns have begun; i.e. old servers no longer
accepting new connections.
* Fix comment typo
* Pull functions out of loop, for readability
Ideally I'd just remove the parameter to caddy.Context.Logger(), but
this would break most Caddy plugins.
Instead, I'm making it variadic and marking it as partially deprecated.
In the future, I might completely remove the parameter once most
plugins have updated.
* fix encode handler header manipulation
also avoid implementing ReadFrom because it breaks when io.Copied to directly
* strconv.Itoa should be tried as a last resort
WriteHeader during Close
* feat: Multiple 'to' upstreams in reverse-proxy cmd
* Repeat --to for multiple upstreams, rather than comma-separating in a single flag
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* caddyhttp: Explicitly disallow multiple regexp matchers
Fix#5028
Since the matchers would overwrite eachother, we should error out to tell the user their config doesn't make sense.
* Update modules/caddyhttp/matchers.go
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Doing so allows for splice/sendfile optimizations when available.
Fixes#4731
Co-authored-by: flga <flga@users.noreply.github.com>
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
* fileserver: Support glob expansion in file matcher
* Fix tests
* Fix bugs and tests
* Attempt Windows fix, sigh
* debug Windows, WIP
* Continue debugging Windows
* Another attempt at Windows
* Plz Windows
* Cmon...
* Clean up, hope I didn't break anything
* caddyhttp: Support sending HTTP 103 Early Hints
This adds support for early hints in the static_response handler.
* caddyhttp: Don't record 1xx responses
* reverseproxy: Close hijacked conns on reload/quit
We also send a Close control message to both ends of
WebSocket connections. I have tested this many times in
my dev environment with consistent success, although
the variety of scenarios was limited.
* Oops... actually call Close() this time
* CloseMessage --> closeMessage
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Use httpguts, duh
* Use map instead of sync.Map
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* core: Refactor listeners; use SO_REUSEPORT on Unix
Just an experiment for now
* Fix lint by logging error
* TCP Keepalive configuration (#4865)
* initial attempt at TCP Keepalive configuration
* core: implement tcp-keepalive for linux
* move canSetKeepAlive interface
* Godoc for keepalive server parameter
* handle return values
* log keepalive errors
* Clean up after bad merge
* Merge in pluggable network types
From 1edc1a45e3
* Slight refactor, fix from recent merge conflict
Co-authored-by: Karmanyaah Malhotra <karmanyaah.gh@malhotra.cc>
* break up code and use lazy reading and pool bufio.Writer
* close underlying connection when operation failed
* allocate bufWriter and streamWriter only once
* refactor record writing
* rebase from master
* handle err
* Fix type assertion
Also reduce some duplication
* Refactor client and clientCloser for logging
Should reduce allocations
* Minor cosmetic adjustments; apply Apache license
* Appease the linter
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
* added the httpError function into the document
* Update templates.go
* Update templates.go
* Fix gofmt
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Until now, the vars matcher has unintentionally lacked parity with the
map directive: the destination placeholders of the map directive would
be expressed as placeholders, i.e. {foo}. But the vars matcher would
not use { }: vars foo value
This looked weird, and was confusing, since it implied that the key
could be dynamic, which doesn't seem helpful here.
I think this is a proper bug fix, since we're not used to accessing
placeholders literally without { } in the Caddyfile.
* core: Plugins can register listener networks
This can be useful for custom listeners.
This feature/API is experimental and may change!
* caddyhttp: Expose server listeners
This allows users to, for example, get upstreams from multiple SRV
endpoints in order (such as primary and secondary clusters).
Also, gofmt went to town on the comments, sigh
Errors returned from the DecisionFunc (whether to get a cert on-demand)
are used as a signal whether to allow a cert or not; *any* error
will forbid cert issuance.
We bubble up the error all the way to the caller, but that caller is the
Go standard library which might gobble it up.
Now we explicitly log connection errors so sysadmins can
ensure their ask endpoints are working.
Thanks to our sponsor AppCove for reporting this!
* fileserver: Support virtual file systems (close#3720)
This change replaces the hard-coded use of os.Open() and os.Stat() with
the use of the new (Go 1.16) io/fs APIs, enabling virtual file systems.
It introduces a new module namespace, caddy.fs, for such file systems.
Also improve documentation for the file server. I realized it was one of
the first modules written for Caddy 2, and the docs hadn't really been
updated since!
* Virtualize FS for file matcher; minor tweaks
* Fix tests and rename dirFS -> osFS
(Since we do not use a root directory, it is dynamic.)
Hahaha this is the ultimate "I have no idea what I'm doing" commit but it
compiles and the tests pass and I declare victory!
... probably broke something, should be tested more.
It is nice that the protobuf dependency becomes indirect now.
* reverseproxy: Implement retry count, alternative to try_duration
* Add Caddyfile support for `retry_match`
* Refactor to deduplicate matcher parsing logic
* Fix lint
See https://caddy.community/t/using-forward-auth-and-writing-my-own-authenticator-in-php/16410, apparently it didn't work when `copy_headers` wasn't used. This is because we were skipping adding a handler to the routes in the "good response handler", but this causes the logic in `reverseproxy.go` to ignore the response handler since it's empty. Instead, we can just always put in the `header` handler, even with an empty `Set` operation, it's just a no-op, but it fixes that condition in the proxy code.
* Make reverse proxy TLS server name replaceable for SNI upstreams.
* Reverted previous TLS server name replacement, and implemented thread safe version.
* Move TLS servername replacement into it's own function
* Moved SNI servername replacement into httptransport.
* Solve issue when dynamic upstreams use wrong protocol upstream.
* Revert previous commit.
Old commit was: Solve issue when dynamic upstreams use wrong protocol upstream.
Id: 3c9806ccb6
* Added SkipTLSPorts option to http transport.
* Fix typo in test config file.
* Rename config option as suggested by Matt
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* Update code to match renamed config option.
* Fix typo in config option name.
* Fix another typo that I missed.
* Tests not completing because of apparent wrong ordering of options.
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* Make reverse proxy TLS server name replaceable for SNI upstreams.
* Reverted previous TLS server name replacement, and implemented thread safe version.
* Move TLS servername replacement into it's own function
* Moved SNI servername replacement into httptransport.
* Solve issue when dynamic upstreams use wrong protocol upstream.
* Revert previous commit.
Old commit was: Solve issue when dynamic upstreams use wrong protocol upstream.
Id: 3c9806ccb6
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
In v2.5.0, upstream health was fixed such that whether an upstream is
considered healthy or not is mostly up to each individual handler's
config. Since "healthy" is an opinion, it is not a global value.
I unintentionally left in the "healthy" field in the API endpoint for
checking upstreams, and it is now misleading (see #4792).
However, num_requests and fails remains, so health can be determined by
the API client, rather than having it be opaquely (and unhelpfully)
determined for the client.
If we do restore this value later on, it'd need to be replicated once
per reverse_proxy handler according to their individual configs.
* reverseproxy: Improve hashing LB policies with HRW
Previously, if a list of upstreams changed, hash-based LB policies
would be greatly affected because the hash relied on the position of
upstreams in the pool. Highest Random Weight or "rendezvous" hashing
is apparently robust to pool changes. It runs in O(n) instead of
O(log n), but n is very small usually.
* Fix bug and update tests
* caddypki: Load intermediate for signing on-the-fly
Fixes#4517
Big thanks to @maraino for adding an API in `smallstep/certificates` so that we can fix this
* Debug log
* Trying a hunch, does it need to be a pointer receiver?
* Clarify pointer receiver
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* reverseproxy: Sync up `handleUpgradeResponse` with stdlib
I had left this as a TODO for when we bump to minimum 1.17, but I should've realized it was under `internal` so it couldn't be used directly.
Copied the functions we needed for parity. Hopefully this is ok!
* Add tests and fix godoc comments
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
Includes several breaking changes; code base updated accordingly.
- Added lots of context arguments
- Use fs.ErrNotExist
- Rename ACMEManager -> ACMEIssuer; CertificateManager -> Manager
* caddyfile: Support for raw token values, improve `map`, `expression`
* Applied code review comments
* Rename RawVal to ValRaw
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
* reverseproxy: New `copy_response` handler for `handle_response` routes
Followup to #4298 and #4388.
This adds a new `copy_response` handler which may only be used in `reverse_proxy`'s `handle_response` routes, which can be used to actually copy the proxy response downstream.
Previously, if `handle_response` was used (with routes, not the status code mode), it was impossible to use the upstream's response body at all, because we would always close the body, expecting the routes to write a new body from scratch.
To implement this, I had to refactor `h.reverseProxy()` to move all the code that came after the `HandleResponse` loop into a new function. This new function `h.finalizeResponse()` takes care of preparing the response by removing extra headers, dealing with trailers, then copying the headers and body downstream.
Since basically what we want `copy_response` to do is invoke `h.finalizeResponse()` at a configurable point in time, we need to pass down the proxy handler, the response, and some other state via a new `req.WithContext(ctx)`. Wrapping a new context is pretty much the only way we have to jump a few layers in the HTTP middleware chain and let a handler pick up this information. Feels a bit dirty, but it works.
Also fixed a bug with the `http.reverse_proxy.upstream.duration` placeholder, it always had the same duration as `http.reverse_proxy.upstream.latency`, but the former was meant to be the time taken for the roundtrip _plus_ copying/writing the response.
* Delete the "Content-Length" header if we aren't copying
Fixes a bug where the Content-Length will mismatch the actual bytes written if we skipped copying the response, so we get a message like this when using curl:
```
curl: (18) transfer closed with 18 bytes remaining to read
```
To replicate:
```
{
admin off
debug
}
:8881 {
reverse_proxy 127.0.0.1:8882 {
@200 status 200
handle_response @200 {
header Foo bar
}
}
}
:8882 {
header Content-Type application/json
respond `{"hello": "world"}` 200
}
```
* Implement `copy_response_headers`, with include/exclude list support
* Apply suggestions from code review
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* opentelemetry: create a new module
* fix imports
* fix test
* Update modules/caddyhttp/opentelemetry/README.md
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Update modules/caddyhttp/opentelemetry/README.md
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Update modules/caddyhttp/opentelemetry/README.md
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Update modules/caddyhttp/opentelemetry/tracer.go
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* rename error ErrUnsupportedTracesProtocol
* replace spaces with tabs in the test data
* Update modules/caddyhttp/opentelemetry/README.md
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Update modules/caddyhttp/opentelemetry/README.md
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* replace spaces with tabs in the README.md
* use default values for a propagation and exporter protocol
* set http attributes with helper
* simplify code
* Cleanup modules/caddyhttp/opentelemetry/README.md
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Update link in README.md
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Update documentation in README.md
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Update link to naming spec in README.md
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Rename module from opentelemetry to tracing
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Rename span_name to span
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Rename span_name to span
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Simplify otel resource creation
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* handle extra attributes
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* update go.opentelemetry.io/otel/semconv to 1.7.0
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* update go.opentelemetry.io/otel version
* remove environment variable handling
* always use tracecontext,baggage as propagators
* extract tracer name into variable
* rename OpenTelemetry to Tracing
* simplify resource creation
* update go.mod
* rename package from opentelemetry to tracing
* cleanup tests
* update Caddyfile example in README.md
* update README.md
* fix test
* fix module name in README.md
* fix module name in README.md
* change names in README.md and tests
* order imports
* remove redundant tests
* Update documentation README.md
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Fix grammar
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Update comments
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* Update comments
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
* update go.sum
* update go.sum
* Add otelhttp instrumentation, update OpenTelemetry libraries.
* Use otelhttp instrumentation for instrumenting HTTP requests.
This change uses context.WithValue to inject the next handler into the
request context via a "nextCall" carrier struct, and pass it on to a
standard Go HTTP handler returned by otelhttp.NewHandler. The
underlying handler will extract the next handler from the context,
call it and pass the returned error to the carrier struct.
* use zap.Error() for the error log
* remove README.md
* update dependencies
* clean up the code
* change comment
* move serveHTTP method from separate file
* add syntax to the UnmarshalCaddyfile comment
* go import the file
* admin: Write proper status on invalid requests (#4569) (fix#4561)
* update dependencies
Co-authored-by: Dave Henderson <dhenderson@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Vibhav Pant <vibhavp@gmail.com>
Co-authored-by: Alok Naushad <alokme123@gmail.com>
Co-authored-by: Cedric Ziel <cedric@cedric-ziel.com>
* Add a override_domain option to allow DNS chanllenge delegation
CNAME can be used to delegate answering the chanllenge to another DNS
zone. One usage is to reduce the exposure of the DNS credential [1].
Based on the discussion in caddy/certmagic#160, we are adding an option
to allow the user explicitly specify the domain to delegate, instead of
following the CNAME chain.
This needs caddy/certmagic#160.
* rename override_domain to dns_challenge_override_domain
* Update CertMagic; fix spelling
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
* reverseproxy: Begin refactor to enable dynamic upstreams
Streamed here: https://www.youtube.com/watch?v=hj7yzXb11jU
* Implement SRV and A/AAA upstream sources
Also get upstreams at every retry loop iteration instead of just once
before the loop. See #4442.
* Minor tweaks from review
* Limit size of upstreams caches
* Add doc notes deprecating LookupSRV
* Provision dynamic upstreams
Still WIP, preparing to preserve health checker functionality
* Rejigger health checks
Move active health check results into handler-specific Upstreams.
Improve documentation regarding health checks and upstreams.
* Deprecation notice
* Add Caddyfile support, use `caddy.Duration`
* Interface guards
* Implement custom resolvers, add resolvers to http transport Caddyfile
* SRV: fix Caddyfile `name` inline arg, remove proto condition
* Use pointer receiver
* Add debug logs
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* reverseproxy: Make shallow-ish clone of the request
* Refactor request cloning into separate function
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
If .ts.net domains are explicitly added to config,
don't try to manage a cert for them (it will fail, and our
implicit Tailscale module will
get those certs at run-time).
Remove /pki/certificates/<ca> endpoint and split into two endpoints:
- GET /pki/ca/<id> to get CA info and certs in JSON format
- GET /pki/ca/<id>/certificates to get cert in PEM chain
* admin: Implement /pki/certificates/<id> API
* pki: Lower "skip_install_trust" log level to INFO
See https://github.com/caddyserver/caddy/issues/4058#issuecomment-976132935
It's not necessary to warn about this, because this was an option explicitly configured by the user. Still useful to log, but we don't need to be so loud about it.
* cmd: Export functions needed for PKI app, return API response to caller
* pki: Rewrite `caddy trust` command to use new admin endpoint instead
* pki: Rewrite `caddy untrust` command to support using admin endpoint
* Refactor cmd and pki packages for determining admin API endpoint
* Update matchers.go
* Update matchers.go
* implementation of zone_id handling
* last changes in zone handling
* give return true values instead of bool
* Apply suggestions from code review
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* changes as suggested
* Apply suggestions from code review
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Update matchers.go
* shortened the Match function
* changed mazcher handling
* Update matchers.go
* delete space
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Huge thank-you to Tailscale (https://tailscale.com) for making this change possible!
This is a great feature for Caddy and Tailscale is a great fit for a standard implementation.
* caddytls: GetCertificate modules; Tailscale
* Caddyfile support for get_certificate
Also fix AP provisioning in case of empty subject list (persist loaded
module on struct, much like Issuers, to surive reprovisioning).
And implement start of HTTP cert getter, still WIP.
* Update modules/caddytls/automation.go
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Use tsclient package, check status for name
* Implement HTTP cert getter
And use reuse CertMagic's PEM functions for private keys.
* Remove cache option from Tailscale getter
Tailscale does its own caching and we don't need the added complexity...
for now, at least.
* Several updates
- Option to disable cert automation in auto HTTPS
- Support multiple cert managers
- Remove cache feature from cert manager modules
- Minor improvements to auto HTTPS logging
* Run go mod tidy
* Try to get certificates from Tailscale implicitly
Only for domains ending in .ts.net.
I think this is really cool!
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
The TestFileListing test in tplcontext_test has one test that verifies
if directory traversal is not happening. The context root is set to
'/tmp' and then it tries to open '../../../../../etc', which gets
normalized to '/tmp/etc'.
The test then expects an error to be returned, assuming that '/tmp/etc'
does not exist on the system. When it does exist, it results in a test
failure:
```
--- FAIL: TestFileListing (0.00s)
tplcontext_test.go:422: Test 4: Expected error but had none
FAIL
FAIL
github.com/caddyserver/caddy/v2/modules/caddyhttp/templates 0.042s
```
Instead of using '/tmp' as root, use a dedicated directory created with
`os.MkdirTemp()` instead. That way, we know that the directory is empty.
* caddyhttp: Redirect HTTP requests on the HTTPS port to https://
* Apply suggestions from code review
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
This is a followup to #4407, in response to a report on the forums: https://caddy.community/t/php-fastcgi-phishing-redirection/14542
Turns out that doing `TrimRight` to remove trailing dots, _before_ cleaning the path, will cause double-dots at the end of the path to not be cleaned away as they should. We should instead remove the dots _after_ cleaning.
* caddyhttp: Enhance vars matcher
Enable "or" logic for multiple values.
Fall back to checking placeholders if not a var name.
* Fix tests (thanks @mohammed90 !)
* fastcgi: Fix a TODO, prevent zap using reflection for logging env
* Update modules/caddyhttp/reverseproxy/fastcgi/fastcgi.go
Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* reverseproxy: Adjust defaults, document defaults
Related to some of the issues in https://github.com/caddyserver/caddy/issues/4245, a complaint about the proxy transport defaults not being properly documented in https://caddy.community/t/default-values-for-directives/14254/6.
- Dug into the stdlib to find the actual defaults for some of the timeouts and buffer limits, documenting them in godoc so the JSON docs get them next release.
- Moved the keep-alive and dial-timeout defaults from `reverseproxy.go` to `httptransport.go`. It doesn't make sense to set defaults in the proxy, because then any time the transport is configured with non-defaults, the keep-alive and dial-timeout defaults are lost!
- Sped up the dial timeout from 10s to 3s, in practice it rarely makes sense to wait a whole 10s for dialing. A shorter timeout helps a lot with the load balancer retries, so using something lower helps with user experience.
* reverseproxy: Make keepalive interval configurable via Caddyfile
* fastcgi: DialTimeout default for fastcgi transport too
Fixes#4428
It's best to still log handler errors at debug level so that they're hidden by default, but still accessible if additional details are necessary.
This makes it easier for users to find the default browse template if they
want to create a custom template based on that. It also makes it easier to
view the template with proper syntax highlighting.
* caddyhttp: Sanitize scheme and host on incoming requests
* reverseproxy: Sanitize the URL scheme and host before proxying
* Apply suggestions from code review
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* Update tplcontext.go
Add {{ render "/path/to/file.ext" $data }} via funcRender
* Update tplcontext.go
* Refactor funcInclude, add funcImport to enable {{block}} and {{template}}
* Fix funcImport return of nil showing up in html
* Update godocs for and
* Add tests for funcInclude
* Add tests for funcImport
* os.RemoveAll -> os.Remove for TestFuncInclude and TestFuncImport
Related to (closed) Issue #2094 on template inheritance. This PR adds a new function called "import" which works like "include", except it only takes one argument and passes it to the referenced file to be used as "." in that file.
* Update tplcontext.go
Add {{ render "/path/to/file.ext" $data }} via funcRender
* Update tplcontext.go
* Refactor funcInclude, add funcImport to enable {{block}} and {{template}}
* Fix funcImport return of nil showing up in html
* Update godocs for and
* caddyhttp: Add support for triggering errors from `try_files`
* caddyhttp: Use vars instead of placeholders/replacer for matcher errors
* caddyhttp: Add comment for matcher error var key
* encode: ignore flushing until after first write (fix#4314)
The first write will determine if encoding has to be done and will add an Content-Encoding. Until then Flushing has to be delayed so the Content-Encoding header can be added before headers and status code is written. (A passthrough flush would write header and status code)
* Update modules/caddyhttp/encode/encode.go
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
From reading through the code, I think this code path is now obsoleted by the changes made in https://github.com/caddyserver/caddy/pull/4266.
Basically, `h.flushInterval()` will set the flush interval to `-1` if we're in a bi-directional stream, and the recent PR ensured that `h.copyResponse()` properly flushes headers immediately when the flush interval is non-zero. So now there should be no need to call Flush before calling `h.copyResponse()`.
This commit fixes the `sortByNameDirFirst` variable inside fileserver to
match what browse's default template has.
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Tweak compression settings
zstd: Limit window sizes to 128K to keep memory in control both server and client size.
zstd: Write 0 length frames. This may be needed for compatibility.
zstd: Create fewer encoders. Small memory improvement.
gzip: Allow -2 (Huffman only) and -3 (stateless) compression modes.
* Update modules/caddyhttp/encode/zstd/zstd.go
Update docs.
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
This is the more correct implementation of 23dadc0d86 (#4179)... I think. This commit effectively undoes the revert in 8848df9c5d, but with corrections to the logic.
We *do* need to use the original request path (the path the browser knows) for redirects, since they are external, and rewrites are only internal.
However, if the path was rewritten to a non-canonical path, we should not redirect to canonicalize that, since rewrites are intentional by the site owner. Canonicalizing the path involves modifying only the suffix (base element, or filename) of the path. Thus, if a rewrite involves only the prefix (like how handle_path strips a path prefix), then we can (hopefully!) safely redirect using the original URI since the filename was not rewritten.
So basically, if rewrites modify the filename, we should not canonicalize those requests. If rewrites only modify another part of the path (commonly a prefix), we should be OK to redirect.
Templates are parsed at request-time (like they are in the templates middleware) to allow live changes to the template while the server is running. Fixes race condition.
Also refactored use of a buffer so a buffer put back in the pool will not continue to be used (written to client) in the meantime.
A couple of benchmarks removed due to refactor, which is fine, since we know pooling helps here.
Also split the Caddyfile subdirective keepalive_idle_conns into two properties so the conns and conns_per_host can be set separately.
This is technically a breaking change, but probably anyone who this breaks already had a broken config anyway, and silently fixing it won't help them fix their configs.
Francis Lavoie
mmm444
Omar Ramadan
Matthew Holt
ydylla
Marten Seemann
Mohammed Al Sahaf
Saber Haj Rabiee
guangwu
Dominik Roos
Corin Langosch
365cent
kassienull
pistasjis
Charles Duffy
WeidiDeng
Tran Phong
jjiang-stripe
eanavitarte
Dave Henderson
Kévin Dunglas
Y.Horie
cui fliter
Stéphane Mourey
Kid
黑墨水鱼
Trea Hauet
Chris Reeves
esell
Emily Lange
Steffen Brüheim
David Frickert
Yannick Ihmels
darkweak
Kyle McCullough
Alban Lecocq
Mariano Cano
Jonathan Garcia
Alexander Graf
XYenon
Scott Mebberson
Abdussamet Koçak
xufanglu
iliana etaoin
Will Norris
lemmi
David Manouchehri
Isaac Parker
fleandro
Avdhut
Abirdcfly
Dávid Szabó
Simon Legner
jedy
WilczyńskiT
Abdussamet Koçak
Chirag Maheshwari
WingLim
Tristan Swadell
Kiss Károly Pál
Kiss Károly Pál
Yaacov Akiba Slama
Gr33nbl00d
Alexander M
Aleks
David Larlet
Tyler Kropp
Marco Kaufmann
Artem Mikheev
Andrii Kushch
Ran Chen
ttys3
BitWuehler
Kevin Daudt
Vojtech Vitek
GallopingKylin
rayjlinden
Денис Телюх
Rainer Borene
Adam Burgess
Kévin Dunglas
Tim Culverhouse
Jeremy Lin
Klaus Helenius
Simão Gomes Viana
Oleg
KallyDev
Slavik
HayatoShiba
Scott Mebberson
Leo Di Donato
diamondburned
elnoro
hmol233
mritd
Klaus Post
Klooven
Peter Magnusson
diamondburned