Skip to content

Instantly share code, notes, and snippets.

@sqlbot

sqlbot/error.xsl

Last active January 8, 2022 17:22
Show Gist options
  • Save sqlbot/ce5388fd452eeb302dfd to your computer and use it in GitHub Desktop.
Save sqlbot/ce5388fd452eeb302dfd to your computer and use it in GitHub Desktop.
Download ZIP
Rewrite XML error responses from S3 REST endpoints using HAProxy 1.6 and Lua
Raw
error.xsl
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<!-- This stylesheet will be loaded by the browser, when the Lua script, below -->
<!-- inserts a reference to it in the XML error response from S3. -->
<!-- Place this file in the bucket as /error.xsl -->
<!-- set the Content-Type to text/xsl and make the file public -->
<!-- also you should probably remove this comment block :) -->
<html>
<title>We&#8217;ve encountered a problem</title>
<meta charset="utf-8" />
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<style type="text/css">
body { background-color: #f0f0f2; margin: 0; padding: 0;
font-family: "Open Sans", "Helvetica Neue", Calibri, Helvetica, Arial, sans-serif; }
p { font-size: 90%; }
li { font-size: 90%; }
div { width: 700px; margin: 1em auto; padding: 40px; padding-top: 25px; background-color: #fff; border-radius: 1em; }
a:link, a:visited { color: #38488f; text-decoration: none; }
@media (max-width: 800px) { body { background-color: #fff; } div { width: auto; margin: 0 auto; border-radius: 0; padding: 1em; } }
</style>
<body>
<div>
<h2>This resource cannot be accessed at the moment.</h2>
<p>While attempting to process your request, we encountered the following problem:</p>
<ul>
<li><b>
<xsl:choose>
<!-- none of the messages returned by S3 are considered to expose exploitable information... -->
<!-- however, when the user is not an S3 customer, some of the stock responses are particularly -->
<!-- unhelpful or are prone to cause confusion; this is not AWS's fault, since they are, after all, -->
<!-- targeted to devops, not end users. In some cases, we want to replace the "Message" with -->
<!-- something more useful to the end user who will see it. -->
<xsl:when test="//Code = 'SignatureDoesNotMatch'">
The authorization credentials received by the underlying system were not valid.
</xsl:when>
<xsl:when test="//Code = 'InvalidAccessKeyId'">
The authorization credentials received by the underlying system were not valid.
</xsl:when>
<xsl:when test="//Code = 'NoSuchKey'">
This request was authorized, but the requested object does not actually exist.
</xsl:when>
<xsl:when test="//Message = 'Query-string authentication requires the Signature, Expires and AWSAccessKeyId parameters'">
The authorization credentials received by the underlying system were only partially received, and thus were deemed invalid, possibly due to incorrect copying or improper modification.
</xsl:when>
<!-- todo: match access denied errors with specific but irrelevant to the end user messages? -->
<!-- todo: these should be virtually impossible except for hard bugs and tampering, so special handling may not even be required -->
<!-- todo: <Code>AccessDenied</Code><Message>Invalid date (should be seconds since epoch): 99999999999999999</Message> -->
<xsl:otherwise>
<xsl:value-of select="//Message"/>
</xsl:otherwise>
</xsl:choose>
</b></li>
</ul>
<xsl:choose>
<xsl:when test="//Expires != ''">
<xsl:if test="//ServerTime != ''">
<p>Our records indicate that the resource you have requested was marked as available for download only until <xsl:value-of select="substring(//Expires,1,10)"/> at <xsl:value-of select="substring(//Expires,12,8)"/> (UTC).</p>
<p>However, it is now <xsl:value-of select="substring(//ServerTime,1,10)"/> at <xsl:value-of select="substring(//ServerTime,12,8)"/> (UTC) according to the server.</p>
<p>This error indicates that the link you're using may be out of date, or that this content has not been updated as recently as you would have expected for it to be.</p>
<p>This condition usually occurs as the result of safety mechanism, designed to prevent out-of-date information from being inadvertently misinterpreted as though it were current information. This error does not mean the information is lost or that a serious error has occurred -- only that the authorization to view it via the link you are using was time-sensitive, and that authorization has expired.</p>
</xsl:if>
</xsl:when>
<xsl:otherwise> <!-- AccessDenied is only checked in the absence of Expires (and ServerTime) -->
<xsl:if test="//Code = 'AccessDenied'">
<p>It seems we may have provided you with a link to a resource to which you do not have access, or a resource which does not exist, or that our internal security mechanisms were unable to reach consensus on your authorization to view it.</p>
</xsl:if>
</xsl:otherwise>
</xsl:choose>
<xsl:if test="//Code = 'InternalError'">
<p>The system that stores the requested information is experiencing a momentary operational problem, which should be resolved shortly. You may wish to try again in a few moments.</p>
</xsl:if>
<xsl:if test="//Code = 'SignatureDoesNotMatch'">
<p>The request could not be processed because the security credentials exchanged between two or more of our systems were not in agreement with each other.</p>
<p>If you clicked a link on a web page to arrive here, this suggests a configuration issue on one of our systems that we'll need to investigate for you. If you clicked on a link from an email message or that was copied and pasted from another source, you may wish to check the link to be sure all the characters were copied successfully.</p>
</xsl:if>
<xsl:if test="//Code = 'NoSuchVersion'">
<p>We sometimes archive consecutive versions of the same object; usually, these consecutive versions are retained, but in some cases, portions of the archive may be purged. This is particularly the case, for example, in a daily report where one day's report turns out to contain exactly the same information as the prior day's report. When this occurs, we sometimes purge the older copies, since they are redundant. We may be able to provide you with a newer version of the file that contains identical data to the one you are unable to retrieve from this location. This issue can also arise during testing, when not all versions of the file may have been suitable for use.</p>
</xsl:if>
<xsl:if test="//Code = 'RequestTimeout'">
<p>We didn't receive all of the data that should have accompanied your request. This is an unusual condition, and the system that stores the requested information might be experiencing a momentary operational problem. If that is the case, this issue should be resolved shortly. You may wish to try again in a few moments.</p>
</xsl:if>
<xsl:if test="//Code = 'SlowDown'">
<p>The system that stores the resource you requested is experiencing a momentary capacity overload. Please try again in a moment.</p>
</xsl:if>
<xsl:if test="//Code = 'ServiceUnavailable'">
<p>The system that stores the requested information is experiencing an operational error or momentary capacity overload. Please try again in a moment; this situation is very unusual but should resolve itself momentarily.</p>
</xsl:if>
<xsl:if test="//Code = 'NoSuchKey'">
<p>Your appear to authorized to view the resource you requested; however, the resource you requested does not actually exist at the expected location... meaning that it's either been removed, or hasn't been created, or... we may simply have sent you to the wrong link to fetch it. In any event, this is unexpected.</p>
</xsl:if>
<xsl:if test="//Code = 'InvalidAccessKeyId'">
<p>Your request was handed off to a system that did not recognize the credentials offered to it for the purpose of authorizing access to the resource you are requesting.</p>
<p>This error is very unlikely to occur, but since it has indeed occurred, it suggests we have an issue in our systems that will require our intervention.</p>
</xsl:if>
<p>For assistance with your request, please contact your program administrator or client services representative. Providing the following information will be very helpful to us in identifying and correcting the cause of this error:</p>
<ul>
<li>Response Code: <xsl:value-of select="//Code"/> (<xsl:value-of select="//ProxyHTTPCode"/>)</li>
<xsl:if test="//Key != ''"><li>URI: /<xsl:value-of select="//Key"/></li></xsl:if>
<xsl:if test="//Resource != ''"><li>Resource: <xsl:value-of select="//Resource"/></li></xsl:if>
<xsl:if test="//Expires != ''"><li>Expiration: <xsl:value-of select="substring(//Expires,1,10)"/> at <xsl:value-of select="substring(//Expires,12,8)"/> (UTC)</li></xsl:if>
<xsl:if test="//ServerTime != ''"><li>Server Time: <xsl:value-of select="substring(//ServerTime,1,10)"/> at <xsl:value-of select="substring(//ServerTime,12,8)"/> (UTC)</li></xsl:if>
<xsl:if test="//ProxyTime != ''"><li>Request Time: <xsl:value-of select="substring(//ProxyTime,1,10)"/> at <xsl:value-of select="substring(//ProxyTime,12,8)"/> (UTC)</li></xsl:if>
<li>Request Identifier: <xsl:value-of select="//RequestId"/><xsl:if test="//ProxyIdentity != ''"> via <xsl:value-of select="//ProxyIdentity"/></xsl:if></li>
<!-- the host identifier does not contain any meaningful or sensitive -->
<!-- information, but it is suppressed because it is long and noisy, -->
<!-- and it is not especially useful to us; we can cross-reference it -->
<!-- from the request identifier if for any reason it is needed by the -->
<!-- upstream storage vendor for advanced troubleshooting -->
<!-- <li>Host Identifier: <xsl:value-of select="//HostId"/></li> -->
</ul>
</div>
<!-- XSL error handler for Amazon S3 REST responses by michael@sqlbot.net -->
</body>
</html>
</xsl:template>
</xsl:stylesheet>
Raw
s3-error-handler.lua
-- prettify the raw XML error messages from Amazon S3, using browser-based XSLT
-- requires HAProxy 1.6 built with embedded Lua
-- in the haproxy global config, load this lua file
-- global
-- lua-load /etc/haproxy/s3-error-handler.lua # that's this file, btw -- save it there
-- then, in the back-end facing Amazon S3, fire the Lua action if the response status code >= 400
-- backend my-s3-backend
-- http-response lua.s3_error_xsl if { status ge 400 }
-- remember that to proxy a request to S3, the Host: header sent to S3 must be something that
-- S3 can associate with the bucket; if the incoming host header doesn't match, just replace it;
-- this allows any request that your proxy sends to be recognized by S3:
-- http-request set-header host example-bucket.s3.amazonaws.com
-- the lua script will modify the resulting XML and call on the browser to render the page using
-- the XSL stylesheet at /error.xsl, in the bucket, which must be publicly-readable, and must
-- have Content-Type: text/xsl
-- for your own sanity, you may also want to (at least for testing) configure /error.xsl with...
-- Cache-Control: private, no-cache, no-store
-- so that you don't have to worry about the browser caching it while you're testing
function s3addxsl(txn)
-- primitive, yet effective.
-- modify haproxy's output buffer when an S3 error message is returned (typically with 403/404),
-- containing raw XMl -- add a link to an xsl stylesheet for a browser-implemented transformation
-- note that this is a rawwwwwwwwww http response buffer we're working with; HAProxy Lua support is
-- new, and evolving, so there may be simpler mechanisms in the future, but as written,
-- since S3 will be using Transfer-Encoding: chunked, we need to capture the first chunk size,
-- which is a number in hex immediatedly following the headers; we add the length of our new string,
-- and modify that chunk size directly in the buffer after converting back to hex
local buff = txn.res:get();
-- capture the http status code for inclusion in the XML
-- there should be a better way, but txn.sf:status() is not available here
local status_code = buff.gsub(buff,"^HTTP/1%.%d (%d+) .*","%1",1) or "Unknown";
local stylesheet = "<?xml-stylesheet type=\"text/xsl\" href=\"/error.xsl\"?>\n";
-- assumes your system clock is UTC, of course
local more_xml = "<ProxyTime>" .. os.date("%Y-%m-%dT%H:%M:%SZ") .. "</ProxyTime>" ..
"<ProxyHTTPCode>" .. status_code .. "</ProxyHTTPCode>";
local newstrlen = string.len(stylesheet) + string.len(more_xml);
-- find the length of the chunked body, a hex value immediately following the header;
-- add the length of our new payload to it, and replace it back into the buffer
buff = string.gsub(buff,"\r\n\r\n(%x+)\r\n", function (chk)
local chunklen = tonumber(chk,16) + newstrlen
return "\r\n\r\n" .. string.format("%x",chunklen) .. "\r\n"
end, 1);
-- before <Error> tag, insert the stylesheet directive, and after, the addtional XML
buff = string.gsub(buff,"<Error>",stylesheet .. "<Error>" .. more_xml);
txn.res:send(buff);
txn:done();
end
-- still part of the Lua script, we need to register our new action on HAProxy
-- startup to call the function above; the "http-response lua...." directive won't
-- parse as valid unless the action has been registered successfully
core.register_action('s3_error_xsl', { 'http-res' }, s3addxsl);
-- 2015-10-13 by michael@sqlbot.net -- provided as-is, no warranty -- consultation and custom development services are available
-- http://stackoverflow.com/questions/33107902/aws-s3-gracefully-handle-403-after-getsignedurl-expired/33109592#33109592
-- eof --
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment