StripPathSuffix is documented to behave like StripPathPrefix: the suffix
is matched in normalized (unescaped) space except where the pattern uses
an escape sequence. But suffix stripping was implemented as
reverse(trimPathPrefix(reverse(escapedPath), reverse(suffix)))
Reversing the strings moves the '%' to the *end* of each "%xx" escape,
which defeats trimPathPrefix's escape detection (it expects '%' to
precede the two hex digits). As a result the escape-aware, normalized
comparison never happened for suffixes: a decoded pattern failed to
match a percent-encoded path.
Concretely, StripPathPrefix "/a/b/c" strips "/a%2Fb/c/d" to "/d", but the
mirror StripPathSuffix "/b/c" left "/a/b%2Fc" untouched instead of
producing "/a"; likewise StripPathSuffix "bc" did not strip "/a%62c".
This has been the behavior since #4948, which introduced both the
escape-aware trimPathPrefix and the reverse-based suffix trimming.
Replace the reverse trick with a dedicated trimPathSuffix that iterates
from the ends of both strings and applies the same escape-aware,
case-insensitive comparison as trimPathPrefix. An escape in the pattern
itself is still compared literally, so "%2fsuffix" continues to require
the path to contain that exact escape.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* rewrite: don't drop a trailing '=' from the query string
buildQueryString scanned for '=' unconditionally when looking for the
end of a component, but '=' only delimits a key from its value once;
any further '=' bytes are literal data. When the query ended with '=',
that byte was consumed as a delimiter with nothing following it to
re-emit, so it was silently lost:
?x=1&sig=YWJjZA== => ?x=1&sig=YWJjZA=
This corrupts base64 padding in the last query parameter, which is the
shape of an S3 presigned URL (X-Amz-Signature), turning a valid
signature into a 403. Only the final '=' of the query was affected;
?sig=YWJjZA==&x=1 came through intact.
Disable the '=' search while consuming a value so that only '&' ends it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* rewrite: honor a query string injected by a replacement value
Which URI components get written back was decided from the literal
config string, before placeholders were expanded, but an injected query
is only detected after expansion. The two were never reconciled: for
`rewrite * {rp.header.X-Accel-Redirect}` there is no literal '?', so
qsStart stayed -1, and the correctly-built query string was computed and
then discarded by the `if qsStart >= 0` guard.
Only half the split was applied. The path was still truncated at the
injected '?', so the query was not preserved either -- it was dropped,
and any query already on the request survived in its place:
GET /orig?keep=me, X-Accel-Redirect: /hello?some=param
=> /hello?keep=me
Track whether a query was actually injected and include that in the
write-back condition. Appending a literal '?' to the rewrite value was
the known workaround precisely because it set qsStart; that keeps
working and is now unnecessary.
The flag is only set where the injected query is adopted, so an
explicitly configured query still wins, and a value with no '?' still
leaves the query untouched -- which is what the implicit rewrites of
try_files and php_fastcgi rely on. Those stay safe regardless, since
escapePathPlaceholders already escapes the two placeholders they use, so
a client-supplied %3F cannot split the URI.
Fixes#5208
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* rewrite: drop a fragment injected by a replacement value
The scan that separates path, query and fragment runs on the literal
config string, so a '#' arriving later via a replacement value was never
treated as a delimiter. It leaked into whichever component it landed in:
X-Accel-Redirect: /hello?p=x#frag => RawQuery = "p=x#frag"
Everything after '#' is fragment (RFC 3986 section 4.2) and a fragment is
never sent to the server, so drop it before the path is split, mirroring
how the scan already handles a literal '#'. An escaped %23 is unaffected,
so a real '#' in a path or query is still expressible, and a configured
fragment still wins over an injected one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
changePath cleared RawPath whenever it equalled url.Path, which
discarded a valid non-canonical percent-encoding produced by a
replacement. This compare `RawPath` against the default escaping of Path
instead.
* rewrite: scope keyed query replace to its named key
* rewrite: cover keyed search_regexp query replace in test
* rewrite: provision query replace test via Provision path
When the rewrite URI template ends with a literal '?' and contains a placeholder that expands to client-controlled bytes (e.g. {http.request.header.X-Fwd}), those bytes flow into buildQueryString which runs a second Replacer pass. If the bytes contain placeholder syntax such as {env.SECRET}, that placeholder is evaluated, allowing disclosure of environment variables, files (via {file./path}), or internal request vars through the rewritten request URI.
Escape '{' and '}' in the injected query before assigning it to the query variable, so the second pass cannot find any placeholder syntax to evaluate. Operator-written placeholders in the rewrite template are already expanded by the first pass on the path component, so the only '{' or '}' surviving into the injected query must have come from replacement values.
Fixes GHSA-j8px-rmrx-76h9.
Includes three regression tests mirroring the 'is not re-expanded' tests in modules/caddyhttp/vars_test.go.
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* chore: upgrade .golangci.yml and workflow to v2
run `golangci-lint fmt`
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* run `golangci-lint run --fix`
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* more lint fixes
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* bring back comments to .golangci.yml
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* appease the linter some more
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* oops
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* use embedded structs
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* use embedded structs where they were used before
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* disable rule `-QF1006`
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* missed a spot
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
---------
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* Implemented basic uri query operations
* Added support for query operations block
* Applied Replacer on all query keys and values
* Implemented rename query key opration
* Rewrite struct: Changed QueryOperations field to Query and comments cleanup
* Cleaned up comments, changed the order of operations and added more tests
* Changed order of fields in queryOps struct to match the operations order
* use gofmput to format code
* use gci to format imports
* reconfigure gci
* linter autofixes
* rearrange imports a little
* export GOOS=windows golangci-lint run ./... --fix
Ideally I'd just remove the parameter to caddy.Context.Logger(), but
this would break most Caddy plugins.
Instead, I'm making it variadic and marking it as partially deprecated.
In the future, I might completely remove the parameter once most
plugins have updated.
If a placeholder in the path component injects a query string such as
the {http.request.uri} placeholder is wont to do, we need to separate it
out from the path.
The fix that was initially put forth in #2971 was good, but only for
up to one layer of nesting. The real problem was that we forgot to
increment nesting when already inside a block if we saw another open
curly brace that opens another block (dispenser.go L157-158).
The new 'handle' directive allows HTTP Caddyfiles to be designed more
like nginx location blocks if the user prefers. Inside a handle block,
directives are still ordered just like they are outside of them, but
handler blocks at a given level of nesting are mutually exclusive.
This work benefitted from some refactoring and cleanup.
Before, modifying the path might have affected how a new query string
was built if the query string relied on the path. Now, we build each
component in isolation and only change the URI on the request later.
Also, prevent trailing & in query string.
Our new parser also preserves original parameter order, rather than
re-encoding using the std lib (which sorts).
The renamed parameters are a breaking change but they're new enough
that I don't think anyone is using them.
Previously, all matchers in a route would be evaluated before any
handlers were executed, and a composite route of the matching routes
would be created. This made rewrites especially tricky, since the only
way to defer later matchers' evaluation was to wrap them in a subroute,
or to invoke a "rehandle" which often caused bugs.
Instead, this new sequential design evaluates each route's matchers then
its handlers in lock-step; matcher-handlers-matcher-handlers...
If the first matching route consists of a rewrite, then the second route
will be evaluated against the rewritten request, rather than the original
one, and so on.
This should do away with any need for rehandling.
I've also taken this opportunity to avoid adding new values to the
request context in the handler chain, as this creates a copy of the
Request struct, which may possibly lead to bugs like it has in the past
(see PR #1542, PR #1481, and maybe issue #2463). We now add all the
expected context values in the top-level handler at the server, then
any new values can be added to the variable table via the VarsCtxKey
context key, or just the GetVar/SetVar functions. In particular, we are
using this facility to convey dial information in the reverse proxy.
Had to be careful in one place as the middleware compilation logic has
changed, and moved a bit. We no longer compile a middleware chain per-
request; instead, we can compile it at provision-time, and defer only the
evaluation of matchers to request-time, which should slightly improve
performance. Doing this, however, we take advantage of multiple function
closures, and we also changed the use of HandlerFunc (function pointer)
to Handler (interface)... this led to a situation where, if we aren't
careful, allows one request routed a certain way to permanently change
the "next" handler for all/most other requests! We avoid this by making
a copy of the interface value (which is a lightweight pointer copy) and
using exclusively that within our wrapped handlers. This way, the
original stack frame is preserved in a "read-only" fashion. The comments
in the code describe this phenomenon.
This may very well be a breaking change for some configurations, however
I do not expect it to impact many people. I will make it clear in the
release notes that this change has occurred.
This commit goes a long way toward making automated documentation of
Caddy config and Caddy modules possible. It's a broad, sweeping change,
but mostly internal. It allows us to automatically generate docs for all
Caddy modules (including future third-party ones) and make them viewable
on a web page; it also doubles as godoc comments.
As such, this commit makes significant progress in migrating the docs
from our temporary wiki page toward our new website which is still under
construction.
With this change, all host modules will use ctx.LoadModule() and pass in
both the struct pointer and the field name as a string. This allows the
reflect package to read the struct tag from that field so that it can
get the necessary information like the module namespace and the inline
key.
This has the nice side-effect of unifying the code and documentation. It
also simplifies module loading, and handles several variations on field
types for raw module fields (i.e. variations on json.RawMessage, such as
arrays and maps).
I also renamed ModuleInfo.Name -> ModuleInfo.ID, to make it clear that
the ID is the "full name" which includes both the module namespace and
the name. This clarity is helpful when describing module hierarchy.
As of this change, Caddy modules are no longer an experimental design.
I think the architecture is good enough to go forward.
* logging: Initial implementation
* logging: More encoder formats, better defaults
* logging: Fix repetition bug with FilterEncoder; add more presets
* logging: DiscardWriter; delete or no-op logs that discard their output
* logging: Add http.handlers.log module; enhance Replacer methods
The Replacer interface has new methods to customize how to handle empty
or unrecognized placeholders. Closes#2815.
* logging: Overhaul HTTP logging, fix bugs, improve filtering, etc.
* logging: General cleanup, begin transitioning to using new loggers
* Fixes after merge conflict
Use piles from which to draw config values.
Module values can return their name, so now we can do two-way mapping
from value to name and name to value; whereas before we could only map
name to value. This was problematic with the Caddyfile adapter since
it receives values and needs to know the name to put in the config.
Differentiating middleware and responders has one benefit, namely that
it's clear which module provides the response, but even then it's not
a great advantage. Linear handler config makes a little more sense,
giving greater flexibility and simplifying the core a bit, even though
it's slightly awkward that handlers which are responders may not use
the 'next' handler that is passed in at all.
TowyTowy
Francis Lavoie
Saleh
alhuda
Lohit
Rayan Salhab
Tao
Matthew Holt
Mohammed Al Sahaf
Kévin Dunglas
Alexander Stecher
Aziz Rmadi
Jacob Gadikian
Tran Phong
Chirag Maheshwari
go-d