Spring Cloud Gateway收到的是http请求,但schema却是https
问题描述
为了方便说明问题,这里把问题简化下(实际上通过下面简化后的描述就暗示了是部署环境导致的问题,实际情况并没有这么顺利,中间经历过崩溃、搞不懂、超出认知、持续折腾了一周,各种实验、怀疑过这种问题,比如ingress自动申请的证书问题、ingress没有正确终止ssl,甚至还重新装过k3s,也怀疑过事traefix nginx的问题,哎,反正各种问题都怀疑完了,然后各种验证,崩溃完了)
有个nginx,跑了一个前端代码,配置文件大概这样:
server
{
listen 80;
listen 443 ssl http2;
server_name xxxxx.top;
index index.php index.html index.htm default.php default.htm default.html;
root /www/wwwroot/xxxxx.top;
#SSL-START SSL相关配置,请勿删除或修改下一行带注释的404规则
#error_page 404/404.html;
#HTTP_TO_HTTPS_START
if ($server_port !~ 443){
rewrite ^(/.*)$ https://$host$1 permanent;
}
#HTTP_TO_HTTPS_END
#ssl_certificate /etc/letsencrypt/live/leyong.top/fullchain.pem;
#ssl_certificate_key /etc/letsencrypt/live/leyong.top/privkey.pem;
ssl_certificate /root/.acme.sh/xxxxx.top/fullchain.cer;
ssl_certificate_key /root/.acme.sh/xxxxx.top/leyong.top.key;
ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ciphers EECDH+CHACHA20:EECDH+CHACHA20-draft:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
add_header Strict-Transport-Security "max-age=31536000";
error_page 497 https://$host$request_uri;
#SSL-END
#ERROR-PAGE-START 错误页配置,可以注释、删除或修改
#error_page 404 /404.html;
#error_page 502 /502.html;
#ERROR-PAGE-END
#PHP-INFO-START PHP引用配置,可以注释或修改
include enable-php-00.conf;
#PHP-INFO-END
#REWRITE-START URL重写规则引用,修改后将导致面板设置的伪静态规则失效
include /www/server/panel/vhost/rewrite/xxxxx.top.conf;
#REWRITE-END
#禁止访问的文件或目录
location ~ ^/(\.user.ini|\.htaccess|\.git|\.env|\.svn|\.project|LICENSE|README.md)
{
return 404;
}
#一键申请SSL证书验证目录相关设置
location ~ \.well-known{
allow all;
}
#禁止在证书验证目录放入敏感文件
if ( $uri ~ "^/\.well-known/.*\.(php|jsp|py|js|css|lua|ts|go|zip|tar\.gz|rar|7z|sql|bak)$" ) {
return 403;
}
location ^~ /ly {
root /www/wwwroot/xxxxx.top;
try_files $uri /ly/index.html;
index index.html index.htm;
error_page 405 =200 http://$host:$server_port$request_uri;
error_page 404 = /404.html;
}
location /api {
proxy_set_header Host $host:$server_port;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 60;
proxy_read_timeout 600;
proxy_send_timeout 600;
proxy_pass http://10.1.12.7:8760/api;
#proxy_pass http://10.1.12.7:31760/api;
client_max_body_size 80m;
}
access_log /www/wwwlogs/xxxxx.top.log;
error_log /www/wwwlogs/xxxxx.top.error.log;
}
http://10.1.12.7:8760/api 指向的是一个spring cloud gateway服务,这个gateway是用docker-compose部署的,采用的镜像比如是:a.com/gateway:1.0,部署的服务器内网ip是:10.1.12.7
然后有一天我再同样的服务器上用同样版本的镜像:a.com/gateway:1.0,通过k3s部署了一下,然后通过service暴露到节点上,节点端口是31760,那么访问地址就是:http://10.1.12.7:31760/api
然后把上面的http://10.1.12.7:8760/api 改为 http://10.1.12.7:31760/api ,也就是指向了通过k3s部署的那个gateway服务。
然后通过https://xxxxx.top/api/xxxxx进行访问,奇怪的事情发生了,会出现这样的异常信息:
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.183[DEBUG] 1 [d63cddef1a164adf9f5eb91b63b9a375] [reactor-http-epoll-3:6022325] [reactor.netty.http.server.HttpServerOperations.?:?] [11dd715f, L:/10.42.0.56:8080 - R:/10.42.0.1:54128] New http connection, requesting read
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.183[DEBUG] 1 [d63cddef1a164adf9f5eb91b63b9a375] [reactor-http-epoll-3:6022325] [reactor.netty.transport.TransportConfig.?:?] [11dd715f, L:/10.42.0.56:8080 - R:/10.42.0.1:54128] Initialized pipeline DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.left.httpTrafficHandler = reactor.netty.http.server.HttpTrafficHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.184[DEBUG] 1 [d63cddef1a164adf9f5eb91b63b9a375] [reactor-http-epoll-3:6022326] [reactor.netty.http.server.HttpServerOperations.?:?] [11dd715f, L:/10.42.0.56:8080 - R:/10.42.0.1:54128] Increasing pending responses, now 1
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.184[DEBUG] 1 [d63cddef1a164adf9f5eb91b63b9a375] [reactor-http-epoll-3:6022326] [reactor.netty.http.server.HttpServer.?:?] [11dd715f-1, L:/10.42.0.56:8080 - R:/10.42.0.1:54128] Handler is being applied: org.springframework.http.server.reactive.ReactorHttpHandlerAdapter@38382b69
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.189[TRACE] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022331] [o.s.cloud.gateway.filter.WeightCalculatorWebFilter.?:?] Weights attr: {}
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.190[TRACE] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022332] [o.s.c.g.h.predicate.PathRoutePredicateFactory.?:?] Pattern "[/lypqs-oauth-server/**]" does not match against value "/oauth/resource/visible"
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.190[TRACE] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022332] [o.s.c.g.h.predicate.PathRoutePredicateFactory.?:?] Pattern "[/lypqs-gateway-server/**]" does not match against value "/oauth/resource/visible"
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.190[TRACE] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022332] [o.s.c.g.h.predicate.PathRoutePredicateFactory.?:?] Pattern "/oauth/**" matches against value "/oauth/resource/visible"
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.190[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022332] [o.s.c.gateway.handler.RoutePredicateHandlerMapping.?:?] Route matched: oauth
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.191[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022333] [o.s.c.gateway.handler.RoutePredicateHandlerMapping.?:?] Mapping [Exchange: GET https://xxxxx.top/oauth/resource/visible?ts=1747227403065&resourceType=ADMIN_RESOURCE&changeHttpsToHttp=false] to Route{id='oauth', uri=lb://lypqs-oauth-server, order=0, predicate=Paths: [/oauth/**], match trailing slash: true, gatewayFilters=[[[StripPrefix parts = 1], order = 1]], metadata={}}
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.191[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022333] [o.s.c.gateway.handler.RoutePredicateHandlerMapping.?:?] [11dd715f-1] Mapped to org.springframework.cloud.gateway.handler.FilteringWebHandler@539725a5
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.191[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022333] [o.s.cloud.gateway.handler.FilteringWebHandler.?:?] Sorted gatewayFilterFactories: [[GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.RemoveCachedBodyFilter@12d40609}, order = -2147483648], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.AdaptCachedBodyGlobalFilter@7dee835}, order = -2147482648], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.NettyWriteResponseFilter@7fb8bad0}, order = -1], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.ForwardPathFilter@193eb1ba}, order = 0], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.GatewayMetricsFilter@46320c9a}, order = 0], [[StripPrefix parts = 1], order = 1], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.RouteToRequestUrlFilter@76437e9b}, order = 10000], [GatewayFilterAdapter{delegate=top.tangyh.lypqs.gateway.filter.GrayscaleReactiveLoadBalancerClientFilter@8fd91d1}, order = 10150], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.WebsocketRoutingFilter@2be818da}, order = 2147483646], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.NettyRoutingFilter@52d0f583}, order = 2147483647], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.ForwardRoutingFilter@236ae13d}, order = 2147483647]]
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.191[TRACE] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022333] [o.s.cloud.gateway.filter.RouteToRequestUrlFilter.?:?] RouteToRequestUrlFilter start
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.193[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022335] [reactor.netty.resources.PooledConnectionProvider.?:?] [73863284] Created a new pooled channel, now: 0 active connections, 0 inactive connections and 0 pending acquire requests.
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.193[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022335] [reactor.netty.tcp.SslProvider.?:?] [73863284] SSL enabled using engine sun.security.ssl.SSLEngineImpl@3a9ca0f7 and SNI /10.42.0.50:8080
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.193[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022335] [reactor.netty.transport.TransportConfig.?:?] [73863284] Initialized pipeline DefaultChannelPipeline{(reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.sslReader = reactor.netty.tcp.SslProvider$SslReadHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.194[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022336] [reactor.netty.transport.TransportConnector.?:?] [73863284] Connecting to [/10.42.0.50:8080].
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.194[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022336] [r.netty.resources.DefaultPooledConnectionProvider.?:?] [73863284, L:/10.42.0.56:59796 - R:10.42.0.50/10.42.0.50:8080] Registering pool release on close event for channel
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.194[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022336] [reactor.netty.resources.PooledConnectionProvider.?:?] [73863284, L:/10.42.0.56:59796 - R:10.42.0.50/10.42.0.50:8080] Channel connected, now: 1 active connections, 0 inactive connections and 0 pending acquire requests.
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.198[ERROR] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022340] [o.s.b.a.w.r.error.AbstractErrorWebExceptionHandler.?:?] [11dd715f-1] 500 Server Error for HTTP GET "/api/oauth/resource/visible?ts=1747227403065&resourceType=ADMIN_RESOURCE&changeHttpsToHttp=false"
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 485454502f312e31203430302042616420526571756573740d0a436f6e74656e742d4c656e6774683a20300d0a436f6e6e656374696f6e3a20636c6f73650d0a0d0a
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1214)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.AccessLogFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ com.alibaba.csp.sentinel.adapter.spring.webflux.SentinelWebFluxFilter [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.TokenContextFilter [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.ContextPathFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.config.CorsConfiguration$$Lambda$811/0x00000008408ebc40 [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.TraceFilter [DefaultWebFilterChain]
*__checkpoint ⇢ HTTP GET "/api/oauth/resource/visible?ts=1747227403065&resourceType=ADMIN_RESOURCE&changeHttpsToHttp=false" [ExceptionHandlingWebHandler]
Original Stack Trace:
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1214)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.199[TRACE] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022341] [o.s.cloud.gateway.filter.GatewayMetricsFilter.?:?] spring.cloud.gateway.requests tags: [tag(httpMethod=GET),tag(httpStatusCode=500),tag(outcome=SERVER_ERROR),tag(routeId=oauth),tag(routeUri=lb://lypqs-oauth-server),tag(status=INTERNAL_SERVER_ERROR)]
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.200[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022342] [reactor.netty.http.server.HttpServerOperations.?:?] [11dd715f-1, L:/10.42.0.56:8080 - R:/10.42.0.1:54128] Decreasing pending responses, now 0
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.200[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022342] [reactor.netty.http.server.HttpServerOperations.?:?] [11dd715f-1, L:/10.42.0.56:8080 - R:/10.42.0.1:54128] Last HTTP packet was sent, terminating the channel
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.200[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022342] [reactor.netty.http.server.HttpServerOperations.?:?] [11dd715f-1, L:/10.42.0.56:8080 - R:/10.42.0.1:54128] Last HTTP response frame
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.201[ WARN] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022343] [reactor.netty.http.client.HttpClientConnect.?:?] [73863284, L:/10.42.0.56:59796 - R:10.42.0.50/10.42.0.50:8080] The connection observed an error
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 485454502f312e31203430302042616420526571756573740d0a436f6e74656e742d4c656e6774683a20300d0a436f6e6e656374696f6e3a20636c6f73650d0a0d0a
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1214)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.AccessLogFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ com.alibaba.csp.sentinel.adapter.spring.webflux.SentinelWebFluxFilter [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.TokenContextFilter [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.ContextPathFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.config.CorsConfiguration$$Lambda$811/0x00000008408ebc40 [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.TraceFilter [DefaultWebFilterChain]
*__checkpoint ⇢ HTTP GET "/api/oauth/resource/visible?ts=1747227403065&resourceType=ADMIN_RESOURCE&changeHttpsToHttp=false" [ExceptionHandlingWebHandler]
Original Stack Trace:
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1214)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.201[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022343] [reactor.netty.resources.PooledConnectionProvider.?:?] [73863284, L:/10.42.0.56:59796 ! R:10.42.0.50/10.42.0.50:8080] Channel closed, now: 0 active connections, 0 inactive connections and 0 pending acquire requests.
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.201[ WARN] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022343] [reactor.netty.http.client.HttpClientConnect.?:?] [73863284, L:/10.42.0.56:59796 ! R:10.42.0.50/10.42.0.50:8080] The connection observed an error
io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 485454502f312e31203430302042616420526571756573740d0a436f6e74656e742d4c656e6774683a20300d0a436f6e6e656374696f6e3a20636c6f73650d0a0d0a
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:477)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 485454502f312e31203430302042616420526571756573740d0a436f6e74656e742d4c656e6774683a20300d0a436f6e6e656374696f6e3a20636c6f73650d0a0d0a
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1214)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.AccessLogFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ com.alibaba.csp.sentinel.adapter.spring.webflux.SentinelWebFluxFilter [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.TokenContextFilter [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.ContextPathFilter [DefaultWebFilterChain]
*__checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.config.CorsConfiguration$$Lambda$811/0x00000008408ebc40 [DefaultWebFilterChain]
*__checkpoint ⇢ top.tangyh.lypqs.gateway.filter.TraceFilter [DefaultWebFilterChain]
*__checkpoint ⇢ HTTP GET "/api/oauth/resource/visible?ts=1747227403065&resourceType=ADMIN_RESOURCE&changeHttpsToHttp=false" [ExceptionHandlingWebHandler]
Original Stack Trace:
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1214)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.201[TRACE] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022343] [reactor.netty.channel.ChannelOperations.?:?] [11dd715f, L:/10.42.0.56:8080 - R:/10.42.0.1:54128] Disposing ChannelOperation from a channel
java.lang.Exception: ChannelOperation terminal stack
at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:465)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
[lypqs-gateway-server:8080:0000:1] 2025-05-18 21:33:41.201[DEBUG] 1 [711e1083010c4aeb9991b6ba60f174d2] [reactor-http-epoll-3:6022343] [r.netty.resources.DefaultPooledConnectionProvider.?:?] [73863284, L:/10.42.0.56:59796 ! R:10.42.0.50/10.42.0.50:8080] onStateChange(PooledConnection{channel=[id: 0x73863284, L:/10.42.0.56:59796 ! R:10.42.0.50/10.42.0.50:8080]}, [disconnecting])
排查过程
此处省略10000字,如果真的把整个过程写下来的,估计10000字都不够。
中途通过gateway 端口的入站请求包,发现是http请求,但日志里或者通过exchange.getRequest().getURI().getSchema()却返回https
和这个头有关:X-Forwarded-Proto: https
罪魁祸首
配置: server.forward-headers-strategy
由于配置文件中其实没有配置这个的参数的,所以导致了不同部署环境有不同的表现。
然后也正式因为没有配置这个参数,到最后定位到这个参数也是相当坎坷。
我们底层采用的事Undertow,undertow在没有配置这个参数的情况下不同的部署环境有不同的行为:


具体就请各位自己看代码了。
再我的这个情况下需要设置为 none . 尝试过fremework也不行
系统推荐
- getPath vs getAbsolutePath vs getCanonicalPath
- 常见问题
- 在没有 telnet 和 nc 的世界里,如何优雅地判断端口是否通?
- Btrace入门
- 微博关注关系如何实现
- JDK命令行工具
- 分库分表
- SQL优化
- Nginx的双向认证配置
- 批量替换文件名中的指定字符串
- 随机图片在线接口
- PostgreSQL高可用
- 随机毒鸡汤:学习过滤得了学渣,却过滤不了人渣。