Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xds: unify client and server handling HttpConnectionManager #8228

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
18060b4
Create HttpConnectionManager datatype to model HCM received as the AP…
voidzcy Jun 2, 2021
0883219
Update LdsUpdate definition to use HttpConnection data model.
voidzcy Jun 2, 2021
6753f15
Update existing usages for LdsUpdate with data populated from HttpCon…
voidzcy Jun 2, 2021
fbc3bd1
Move logics for parsing and validating TCP listener into ClientXdsCli…
voidzcy Jun 2, 2021
c825bd0
Fix tests.
voidzcy Jun 2, 2021
308da0b
Merge branch 'master' of github.com:grpc/grpc-java into refactor/use_…
voidzcy Jun 3, 2021
4837170
Revert excessive changes.
voidzcy Jun 3, 2021
57ec58d
Listener address port is optional for server side TCP listener.
voidzcy Jun 3, 2021
bb967b3
Minor polishment.
voidzcy Jun 3, 2021
f49edd0
Update misleading variable name.
voidzcy Jun 7, 2021
361096b
Eliminate redundant static method qualifier.
voidzcy Jun 7, 2021
2fa37bb
Unify the handling of data inside HttpConnectionManager for both clie…
voidzcy Jun 7, 2021
38a6846
Merge branch 'master' of github.com:grpc/grpc-java into refactor/use_…
voidzcy Jun 7, 2021
b6411ae
Simplify proto getter.
voidzcy Jun 7, 2021
4f779f7
Remove inapplicable TODO.
voidzcy Jun 9, 2021
90eab59
Change static factory method name from withXXX() to forXXX().
voidzcy Jun 9, 2021
d9c674b
Delete unncessary addresses in test data.
voidzcy Jun 10, 2021
b2ee048
Include FilterChain's name in the parsed form.
voidzcy Jun 11, 2021
d3ac75c
Fix tests, use some random FilterChain names as it does not matter fo…
voidzcy Jun 11, 2021
4ef3425
Include FilterChain's name in the parsed form, if no name is provided…
voidzcy Jun 11, 2021
003c535
Add Route with non_forwarding_action
voidzcy Jun 12, 2021
8cbb28f
Parse top-level HttpFilters based on client/server usages.
voidzcy Jun 14, 2021
9001c89
Merge branch 'master' of github.com:grpc/grpc-java into refactor/use_…
voidzcy Jun 14, 2021
ed23f12
Fix parsing top-level HttpFilter with parsing override HttpFilter con…
voidzcy Jun 14, 2021
163d636
Include RBAC filter parsing in test.
voidzcy Jun 14, 2021
8b845a3
Minor style issue.
voidzcy Jun 14, 2021
f052e72
Manage RDS resources referenced by HttpConnectionManagers in LDS resp…
voidzcy Jun 15, 2021
f735616
Fix XdsNameResolver handling Route without cluster, also add test cov…
voidzcy Jun 15, 2021
4b000a5
Add a test case for covering parsing RBACPerRoute.
voidzcy Jun 15, 2021
e3e885e
Merge branch 'master' of github.com:grpc/grpc-java into refactor/use_…
voidzcy Jun 16, 2021
bd26751
Minor style polish and code simplication.
voidzcy Jun 17, 2021
7136ec5
Fix bug in parsing HttpFilter config for checking if the filter imple…
voidzcy Jun 17, 2021
09ee39e
Delete redundant interface method for indicating if the filter implem…
voidzcy Jun 17, 2021
2ae6b34
Nack LDS response if HttpConnectionManager contains xff_num_trusted_h…
voidzcy Jun 17, 2021
9ad1e94
Simplify boolean zen.
voidzcy Jun 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
481 changes: 340 additions & 141 deletions xds/src/main/java/io/grpc/xds/ClientXdsClient.java

Large diffs are not rendered by default.

282 changes: 47 additions & 235 deletions xds/src/main/java/io/grpc/xds/EnvoyServerProtoData.java

Large diffs are not rendered by default.

68 changes: 68 additions & 0 deletions xds/src/main/java/io/grpc/xds/HttpConnectionManager.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/*
* Copyright 2021 The gRPC Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package io.grpc.xds;

import static com.google.common.base.Preconditions.checkNotNull;

import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableList;
import io.grpc.xds.Filter.NamedFilterConfig;
import java.util.List;
import javax.annotation.Nullable;

/**
* HttpConnectionManager is a network filter for proxying HTTP requests.
*/
@AutoValue
abstract class HttpConnectionManager {
// Total number of nanoseconds to keep alive an HTTP request/response stream.
abstract long httpMaxStreamDurationNano();

// Name of the route configuration to be used for RDS resource discovery.
@Nullable
abstract String rdsName();

// List of virtual hosts that make up the route table.
@Nullable
abstract ImmutableList<VirtualHost> virtualHosts();

// List of http filter configs. Null if HttpFilter support is not enabled.
@Nullable
abstract ImmutableList<NamedFilterConfig> httpFilterConfigs();

static HttpConnectionManager forRdsName(long httpMaxStreamDurationNano, String rdsName,
@Nullable List<NamedFilterConfig> httpFilterConfigs) {
checkNotNull(rdsName, "rdsName");
return create(httpMaxStreamDurationNano, rdsName, null, httpFilterConfigs);
}

static HttpConnectionManager forVirtualHosts(long httpMaxStreamDurationNano,
List<VirtualHost> virtualHosts, @Nullable List<NamedFilterConfig> httpFilterConfigs) {
checkNotNull(virtualHosts, "virtualHosts");
return create(httpMaxStreamDurationNano, null, virtualHosts,
httpFilterConfigs);
}

private static HttpConnectionManager create(long httpMaxStreamDurationNano,
@Nullable String rdsName, @Nullable List<VirtualHost> virtualHosts,
@Nullable List<NamedFilterConfig> httpFilterConfigs) {
return new AutoValue_HttpConnectionManager(
httpMaxStreamDurationNano, rdsName,
virtualHosts == null ? null : ImmutableList.copyOf(virtualHosts),
httpFilterConfigs == null ? null : ImmutableList.copyOf(httpFilterConfigs));
}
}
11 changes: 10 additions & 1 deletion xds/src/main/java/io/grpc/xds/RouterFilter.java
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,16 @@
import com.google.protobuf.Message;
import io.grpc.ClientInterceptor;
import io.grpc.LoadBalancer.PickSubchannelArgs;
import io.grpc.ServerInterceptor;
import io.grpc.xds.Filter.ClientInterceptorBuilder;
import io.grpc.xds.Filter.ServerInterceptorBuilder;
import java.util.concurrent.ScheduledExecutorService;
import javax.annotation.Nullable;

/**
* Router filter implementation. Currently this filter does not parse any field in the config.
*/
enum RouterFilter implements Filter, ClientInterceptorBuilder {
enum RouterFilter implements Filter, ClientInterceptorBuilder, ServerInterceptorBuilder {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not from this PR but why use enum when clearly this type is not used to enumerate constants? enum works given how this is used but I wouldn't think of the RouterFilter as an enum.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Java, enum types are considered to be a special type of class. When creating an enum class, the compiler will create instances (objects) of each enum constants. So enum can be an easy trick to create singletons.

INSTANCE;

static final String TYPE_URL =
Expand Down Expand Up @@ -66,4 +68,11 @@ public ClientInterceptor buildClientInterceptor(
ScheduledExecutorService scheduler) {
return null;
}

@Nullable
@Override
public ServerInterceptor buildServerInterceptor(
FilterConfig config, @Nullable Filter.FilterConfig overrideConfig) {
return null;
}
}
15 changes: 13 additions & 2 deletions xds/src/main/java/io/grpc/xds/VirtualHost.java
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,23 @@ public static VirtualHost create(
abstract static class Route {
abstract RouteMatch routeMatch();

@Nullable
abstract RouteAction routeAction();

abstract ImmutableMap<String, FilterConfig> filterConfigOverrides();

static Route create(
RouteMatch routeMatch, RouteAction routeAction,
static Route forAction(RouteMatch routeMatch, RouteAction routeAction,
Map<String, FilterConfig> filterConfigOverrides) {
return create(routeMatch, routeAction, filterConfigOverrides);
}

static Route forNonForwardingAction(RouteMatch routeMatch,
Map<String, FilterConfig> filterConfigOverrides) {
return create(routeMatch, null, filterConfigOverrides);
}

private static Route create(
RouteMatch routeMatch, @Nullable RouteAction routeAction,
Map<String, FilterConfig> filterConfigOverrides) {
return new AutoValue_VirtualHost_Route(
routeMatch, routeAction, ImmutableMap.copyOf(filterConfigOverrides));
Expand Down
95 changes: 13 additions & 82 deletions xds/src/main/java/io/grpc/xds/XdsClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@

import com.google.auto.value.AutoValue;
import com.google.common.base.MoreObjects;
import com.google.common.base.MoreObjects.ToStringHelper;
import com.google.common.collect.ImmutableList;
import com.google.protobuf.Any;
import io.grpc.Status;
Expand All @@ -29,7 +28,6 @@
import io.grpc.xds.Endpoints.LocalityLbEndpoints;
import io.grpc.xds.EnvoyServerProtoData.Listener;
import io.grpc.xds.EnvoyServerProtoData.UpstreamTlsContext;
import io.grpc.xds.Filter.NamedFilterConfig;
import io.grpc.xds.LoadStatsManager2.ClusterDropStats;
import io.grpc.xds.LoadStatsManager2.ClusterLocalityStats;
import java.util.ArrayList;
Expand All @@ -48,92 +46,25 @@
*/
abstract class XdsClient {

static final class LdsUpdate implements ResourceUpdate {
// Total number of nanoseconds to keep alive an HTTP request/response stream.
final long httpMaxStreamDurationNano;
// The name of the route configuration to be used for RDS resource discovery.
@Nullable
final String rdsName;
// The list virtual hosts that make up the route table.
@Nullable
final List<VirtualHost> virtualHosts;
// Filter instance names. Null if HttpFilter support is not enabled.
@Nullable final List<NamedFilterConfig> filterChain;
// Server side Listener.
@AutoValue
abstract static class LdsUpdate implements ResourceUpdate {
// Http level api listener configuration.
@Nullable
final Listener listener;

LdsUpdate(
long httpMaxStreamDurationNano, String rdsName,
@Nullable List<NamedFilterConfig> filterChain) {
this(httpMaxStreamDurationNano, rdsName, null, filterChain);
}

LdsUpdate(
long httpMaxStreamDurationNano, List<VirtualHost> virtualHosts,
@Nullable List<NamedFilterConfig> filterChain) {
this(httpMaxStreamDurationNano, null, virtualHosts, filterChain);
}

private LdsUpdate(
long httpMaxStreamDurationNano, @Nullable String rdsName,
@Nullable List<VirtualHost> virtualHosts, @Nullable List<NamedFilterConfig> filterChain) {
this.httpMaxStreamDurationNano = httpMaxStreamDurationNano;
this.rdsName = rdsName;
this.virtualHosts = virtualHosts == null
? null : Collections.unmodifiableList(new ArrayList<>(virtualHosts));
this.filterChain = filterChain == null ? null : Collections.unmodifiableList(filterChain);
this.listener = null;
}

LdsUpdate(Listener listener) {
this.listener = listener;
this.httpMaxStreamDurationNano = 0L;
this.rdsName = null;
this.filterChain = null;
this.virtualHosts = null;
}
abstract HttpConnectionManager httpConnectionManager();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the server side there is no such thing as one httpConnectionManager in the LDS because you can have more than one httpConnectionManager due to the filterChain. What's the plan for properly supporting the server side semantics?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See EnvoyServerProtoData.FilterChain: One FilterChain has one HttpConnectionManager. The server side will only use LdsUpdate.listener field, the rests are for client side.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now LdsUpdate is an oneof HttpConnectionManager (for client-side) or Listener (for server-side).


@Override
public int hashCode() {
return Objects.hash(
httpMaxStreamDurationNano, rdsName, virtualHosts, filterChain, listener);
}
// Tcp level listener configuration.
@Nullable
abstract Listener listener();

@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
LdsUpdate that = (LdsUpdate) o;
return httpMaxStreamDurationNano == that.httpMaxStreamDurationNano
&& Objects.equals(rdsName, that.rdsName)
&& Objects.equals(virtualHosts, that.virtualHosts)
&& Objects.equals(filterChain, that.filterChain)
&& Objects.equals(listener, that.listener);
static LdsUpdate forApiListener(HttpConnectionManager httpConnectionManager) {
checkNotNull(httpConnectionManager, "httpConnectionManager");
return new AutoValue_XdsClient_LdsUpdate(httpConnectionManager, null);
}

@Override
public String toString() {
ToStringHelper toStringHelper = MoreObjects.toStringHelper(this);
toStringHelper.add("httpMaxStreamDurationNano", httpMaxStreamDurationNano);
if (rdsName != null) {
toStringHelper.add("rdsName", rdsName);
} else {
toStringHelper.add("virtualHosts", virtualHosts);
}
if (filterChain != null) {
toStringHelper.add("filterChain", filterChain);
}
if (listener != null) {
toStringHelper.add("listener", listener);
}
return toStringHelper.toString();
static LdsUpdate forTcpListener(Listener listener) {
checkNotNull(listener, "listener");
return new AutoValue_XdsClient_LdsUpdate(null, listener);
}

}

static final class RdsUpdate implements ResourceUpdate {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ public void start() {
new XdsClient.LdsResourceWatcher() {
@Override
public void onChanged(XdsClient.LdsUpdate update) {
releaseOldSuppliers(curListener.getAndSet(update.listener));
releaseOldSuppliers(curListener.getAndSet(update.listener()));
reportSuccess();
}

Expand Down
27 changes: 18 additions & 9 deletions xds/src/main/java/io/grpc/xds/XdsNameResolver.java
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,10 @@ public Result selectConfig(PickSubchannelArgs args) {
return Result.forError(
Status.UNAVAILABLE.withDescription("Could not find xDS route matching RPC"));
}
if (selectedRoute.routeAction() == null) {
return Result.forError(Status.UNAVAILABLE.withDescription(
"Could not route RPC to Route with non-forwarding action"));
}
RouteAction action = selectedRoute.routeAction();
if (action.cluster() != null) {
cluster = action.cluster();
Expand Down Expand Up @@ -648,14 +652,17 @@ public void run() {
return;
}
logger.log(XdsLogLevel.INFO, "Receive LDS resource update: {0}", update);
List<VirtualHost> virtualHosts = update.virtualHosts;
String rdsName = update.rdsName;
HttpConnectionManager httpConnectionManager = update.httpConnectionManager();
List<VirtualHost> virtualHosts = httpConnectionManager.virtualHosts();
String rdsName = httpConnectionManager.rdsName();
cleanUpRouteDiscoveryState();
if (virtualHosts != null) {
updateRoutes(virtualHosts, update.httpMaxStreamDurationNano, update.filterChain);
updateRoutes(virtualHosts, httpConnectionManager.httpMaxStreamDurationNano(),
httpConnectionManager.httpFilterConfigs());
} else {
routeDiscoveryState = new RouteDiscoveryState(
rdsName, update.httpMaxStreamDurationNano, update.filterChain);
rdsName, httpConnectionManager.httpMaxStreamDurationNano(),
httpConnectionManager.httpFilterConfigs());
logger.log(XdsLogLevel.INFO, "Start watching RDS resource {0}", rdsName);
xdsClient.watchRdsResource(rdsName, routeDiscoveryState);
}
Expand Down Expand Up @@ -739,11 +746,13 @@ private void updateRoutes(List<VirtualHost> virtualHosts, long httpMaxStreamDura
Set<String> clusters = new HashSet<>();
for (Route route : routes) {
RouteAction action = route.routeAction();
if (action.cluster() != null) {
clusters.add(action.cluster());
} else if (action.weightedClusters() != null) {
for (ClusterWeight weighedCluster : action.weightedClusters()) {
clusters.add(weighedCluster.name());
if (action != null) {
if (action.cluster() != null) {
clusters.add(action.cluster());
} else if (action.weightedClusters() != null) {
for (ClusterWeight weighedCluster : action.weightedClusters()) {
clusters.add(weighedCluster.name());
}
}
}
}
Expand Down
Loading