-
Notifications
You must be signed in to change notification settings - Fork 64
feat: add client side logging with slf4j #3403
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 95 commits
59b5d73
d34ec88
b6417ae
0c169ad
9ced175
681f7d5
5c01ef2
6605ef3
ebbae15
543298d
aac66f2
487614c
cc3465b
211a3d1
2970210
32f8b7c
245c14d
8358497
97087b2
727a3e9
23bc111
bbe0700
3d7c7f9
b870d81
56a2870
83eedf0
d85c848
1919809
613b6c8
1169eaa
a6b5433
fb0966e
77939fe
04ef774
439e071
673c9fe
c2b607b
3df39fa
4190dc7
fefb436
7901d8a
20f9f5a
172701d
d25e742
a8d2f20
df97834
32dbd0e
37eb391
f95423b
8e01aa0
73210fa
b1386cb
ef26d30
0c466eb
7a23e2d
3821ec3
23f00e8
9f7d77c
86cc4d2
446910a
de01f37
e3862a8
427e820
6ff5479
82ead2b
b5b6e94
5db8884
56316d3
b7234e3
e7acafc
801cef6
cf8ecbc
19faf95
47f1212
420de1e
c222198
227857a
008048a
07888cb
20017cc
2d3ba3e
7c1ea76
7aa8333
d5c759d
1641f55
d86b7a0
f611c74
539031b
091ab38
b3b328e
6aece18
c3db3e7
a975fd5
b1af879
157921b
89da1b4
fad1865
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,136 @@ | ||
/* | ||
* Copyright 2025 Google LLC | ||
* | ||
* Redistribution and use in source and binary forms, with or without | ||
* modification, are permitted provided that the following conditions are | ||
* met: | ||
* | ||
* * Redistributions of source code must retain the above copyright | ||
* notice, this list of conditions and the following disclaimer. | ||
* * Redistributions in binary form must reproduce the above | ||
* copyright notice, this list of conditions and the following disclaimer | ||
* in the documentation and/or other materials provided with the | ||
* distribution. | ||
* * Neither the name of Google LLC nor the names of its | ||
* contributors may be used to endorse or promote products derived from | ||
* this software without specific prior written permission. | ||
* | ||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | ||
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | ||
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | ||
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | ||
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | ||
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | ||
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | ||
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | ||
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
*/ | ||
|
||
package com.google.api.gax.grpc; | ||
|
||
import static com.google.api.gax.logging.LoggingUtils.executeWithTryCatch; | ||
import static com.google.api.gax.logging.LoggingUtils.logRequest; | ||
import static com.google.api.gax.logging.LoggingUtils.logResponse; | ||
import static com.google.api.gax.logging.LoggingUtils.recordResponseHeaders; | ||
import static com.google.api.gax.logging.LoggingUtils.recordResponsePayload; | ||
import static com.google.api.gax.logging.LoggingUtils.recordServiceRpcAndRequestHeaders; | ||
|
||
import com.google.api.core.InternalApi; | ||
import com.google.api.gax.logging.LogData; | ||
import com.google.api.gax.logging.LoggerProvider; | ||
import io.grpc.CallOptions; | ||
import io.grpc.Channel; | ||
import io.grpc.ClientCall; | ||
import io.grpc.ClientInterceptor; | ||
import io.grpc.ForwardingClientCall; | ||
import io.grpc.ForwardingClientCallListener.SimpleForwardingClientCallListener; | ||
import io.grpc.Metadata; | ||
import io.grpc.MethodDescriptor; | ||
import io.grpc.Status; | ||
import java.util.HashMap; | ||
import java.util.Map; | ||
|
||
@InternalApi | ||
public class GrpcLoggingInterceptor implements ClientInterceptor { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This can be package private. Same thing for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. With the current test setup, these needs to be public for testing purposes. I'll look into the test setup again There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. bumping this now again. Does this still need to be public with InternalApi? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Unfortunately, as conversations diverged to other topics since then, I did not got the chance to revamp the testing. As of now, these still need to be public with InternalApi. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If this class has to be public due to testing purposes, would it make sense to add a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. unfortunately, adding |
||
|
||
private static final LoggerProvider LOGGER_PROVIDER = | ||
LoggerProvider.forClazz(GrpcLoggingInterceptor.class); | ||
|
||
ClientCall.Listener<?> currentListener; // expose for test setup | ||
|
||
@Override | ||
public <ReqT, RespT> ClientCall<ReqT, RespT> interceptCall( | ||
MethodDescriptor<ReqT, RespT> method, CallOptions callOptions, Channel next) { | ||
|
||
return new ForwardingClientCall.SimpleForwardingClientCall<ReqT, RespT>( | ||
next.newCall(method, callOptions)) { | ||
LogData.Builder logDataBuilder = LogData.builder(); | ||
|
||
@Override | ||
public void start(Listener<RespT> responseListener, Metadata headers) { | ||
recordServiceRpcAndRequestHeaders( | ||
method.getServiceName(), | ||
method.getFullMethodName(), | ||
null, // endpoint is for http request only | ||
metadataHeadersToMap(headers), | ||
logDataBuilder, | ||
LOGGER_PROVIDER); | ||
SimpleForwardingClientCallListener<RespT> responseLoggingListener = | ||
new SimpleForwardingClientCallListener<RespT>(responseListener) { | ||
@Override | ||
public void onHeaders(Metadata headers) { | ||
recordResponseHeaders( | ||
metadataHeadersToMap(headers), logDataBuilder, LOGGER_PROVIDER); | ||
super.onHeaders(headers); | ||
} | ||
|
||
@Override | ||
public void onMessage(RespT message) { | ||
recordResponsePayload(message, logDataBuilder, LOGGER_PROVIDER); | ||
super.onMessage(message); | ||
} | ||
|
||
@Override | ||
public void onClose(Status status, Metadata trailers) { | ||
logResponse(status.getCode().toString(), logDataBuilder, LOGGER_PROVIDER); | ||
super.onClose(status, trailers); | ||
} | ||
}; | ||
currentListener = responseLoggingListener; | ||
super.start(responseLoggingListener, headers); | ||
} | ||
|
||
@Override | ||
public void sendMessage(ReqT message) { | ||
logRequest(message, logDataBuilder, LOGGER_PROVIDER); | ||
super.sendMessage(message); | ||
} | ||
}; | ||
} | ||
|
||
// Helper methods for logging | ||
zhumin8 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
private static Map<String, String> metadataHeadersToMap(Metadata headers) { | ||
|
||
Map<String, String> headersMap = new HashMap<>(); | ||
executeWithTryCatch( | ||
() -> { | ||
for (String key : headers.keys()) { | ||
// grpc header values can be either ASCII strings or binary | ||
// https://grpc.io/docs/guides/metadata/#overview | ||
// this condition identified binary headers and skip for logging | ||
if (key.endsWith(Metadata.BINARY_HEADER_SUFFIX)) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is more future proof to do something like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you add a comment on why we need this There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added comment in 091ab38 |
||
continue; | ||
} | ||
Metadata.Key<String> metadataKey = | ||
Metadata.Key.of(key, Metadata.ASCII_STRING_MARSHALLER); | ||
String headerValue = headers.get(metadataKey); | ||
|
||
headersMap.put(key, headerValue); | ||
} | ||
}); | ||
|
||
return headersMap; | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -681,6 +681,7 @@ private ManagedChannel createSingleChannel() throws IOException { | |
builder = | ||
builder | ||
.intercept(new GrpcChannelUUIDInterceptor()) | ||
.intercept(new GrpcLoggingInterceptor()) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. just a thought (I don't know if there are any downsides or if this would work): Could we gate adding the logging interceptor here? i.e. Check if the logging env var exists + is configured and only add an interceptor if so. I think there was some mention about not using interceptors (potentially) so it may not work in the future. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess there is no harm in adding a gate here for now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was just thinking that having it gate there might be a way to remove all the |
||
.intercept(headerInterceptor) | ||
.intercept(metadataHandlerInterceptor) | ||
.userAgent(headerInterceptor.getUserAgentHeader()) | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
/* | ||
* Copyright 2025 Google LLC | ||
* | ||
* Redistribution and use in source and binary forms, with or without | ||
* modification, are permitted provided that the following conditions are | ||
* met: | ||
* | ||
* * Redistributions of source code must retain the above copyright | ||
* notice, this list of conditions and the following disclaimer. | ||
* * Redistributions in binary form must reproduce the above | ||
* copyright notice, this list of conditions and the following disclaimer | ||
* in the documentation and/or other materials provided with the | ||
* distribution. | ||
* * Neither the name of Google LLC nor the names of its | ||
* contributors may be used to endorse or promote products derived from | ||
* this software without specific prior written permission. | ||
* | ||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | ||
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | ||
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | ||
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | ||
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | ||
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | ||
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | ||
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | ||
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
*/ | ||
|
||
package com.google.api.gax.grpc; | ||
|
||
import static org.mockito.ArgumentMatchers.any; | ||
import static org.mockito.Mockito.mock; | ||
import static org.mockito.Mockito.spy; | ||
import static org.mockito.Mockito.verify; | ||
import static org.mockito.Mockito.when; | ||
|
||
import com.google.api.gax.grpc.testing.FakeMethodDescriptor; | ||
import io.grpc.CallOptions; | ||
import io.grpc.Channel; | ||
import io.grpc.ClientCall; | ||
import io.grpc.ClientInterceptors; | ||
import io.grpc.Metadata; | ||
import io.grpc.MethodDescriptor; | ||
import io.grpc.Status; | ||
import org.junit.jupiter.api.Test; | ||
import org.junit.jupiter.api.extension.ExtendWith; | ||
import org.mockito.Mock; | ||
import org.mockito.Mockito; | ||
import org.mockito.junit.jupiter.MockitoExtension; | ||
|
||
@ExtendWith(MockitoExtension.class) | ||
class GrpcLoggingInterceptorTest { | ||
@Mock private Channel channel; | ||
|
||
@Mock private ClientCall<String, Integer> call; | ||
|
||
private static final MethodDescriptor<String, Integer> method = FakeMethodDescriptor.create(); | ||
|
||
@Test | ||
void testInterceptor_basic() { | ||
when(channel.newCall(Mockito.<MethodDescriptor<String, Integer>>any(), any(CallOptions.class))) | ||
.thenReturn(call); | ||
GrpcLoggingInterceptor interceptor = new GrpcLoggingInterceptor(); | ||
Channel intercepted = ClientInterceptors.intercept(channel, interceptor); | ||
@SuppressWarnings("unchecked") | ||
ClientCall.Listener<Integer> listener = mock(ClientCall.Listener.class); | ||
ClientCall<String, Integer> interceptedCall = intercepted.newCall(method, CallOptions.DEFAULT); | ||
// Simulate starting the call | ||
interceptedCall.start(listener, new Metadata()); | ||
// Verify that the underlying call's start() method is invoked | ||
verify(call).start(any(ClientCall.Listener.class), any(Metadata.class)); | ||
|
||
// Simulate sending a message | ||
String requestMessage = "test request"; | ||
interceptedCall.sendMessage(requestMessage); | ||
// Verify that the underlying call's sendMessage() method is invoked | ||
verify(call).sendMessage(requestMessage); | ||
} | ||
|
||
@Test | ||
void testInterceptor_responseListener() { | ||
when(channel.newCall(Mockito.<MethodDescriptor<String, Integer>>any(), any(CallOptions.class))) | ||
.thenReturn(call); | ||
GrpcLoggingInterceptor interceptor = spy(new GrpcLoggingInterceptor()); | ||
Channel intercepted = ClientInterceptors.intercept(channel, interceptor); | ||
@SuppressWarnings("unchecked") | ||
ClientCall.Listener<Integer> listener = mock(ClientCall.Listener.class); | ||
ClientCall<String, Integer> interceptedCall = intercepted.newCall(method, CallOptions.DEFAULT); | ||
interceptedCall.start(listener, new Metadata()); | ||
|
||
// Simulate respond interceptor calls | ||
Metadata responseHeaders = new Metadata(); | ||
responseHeaders.put( | ||
Metadata.Key.of("test-header", Metadata.ASCII_STRING_MARSHALLER), "header-value"); | ||
interceptor.currentListener.onHeaders(responseHeaders); | ||
|
||
interceptor.currentListener.onMessage(null); | ||
|
||
Status status = Status.OK; | ||
interceptor.currentListener.onClose(status, new Metadata()); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you know
dependencies.properties
(or the bazel build) also have some form of optional dependency? Would this be something that we also need to add?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can look into this. But I don't think it really matters as bazel build here is only used for integration tests (which is irrelevant to logging feature), and we do not publish artifact with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you know if self-service also uses the bazel build? If so, would it make sense for them to also have slfj be optional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. Correct me if I am wrong, I am adding slf4j optional dependency to Gax. For self-service, if even if they use bazel build and build with slf4j (non-optional). Their published artifact depends on a published version of GAX? If so, we are good and do not need to alter the bazel process.
For the original question: from a quick search, bazel does not have a easy equivalent of maven's optional.
cc. @blakeli0 if you know any details on the self-service process?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in person discussion: We are not adding to gradle templates and should be good here.