Skip to content

fix: provider has random disconnects #99

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 24, 2025
Merged

Conversation

fubuloubu
Copy link
Member

@fubuloubu fubuloubu commented Feb 20, 2025

What I did

fixes: #98

Was getting really weird random disconnect errors w/ Alchemy in prod use, so needs a workaround

Example 1:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 534, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 516, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/http/client.py", line 1395, in getresponse
    response.begin()
  File "/usr/local/lib/python3.11/http/client.py", line 325, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/http/client.py", line 294, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 474, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/util.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 534, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 516, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/http/client.py", line 1395, in getresponse
    response.begin()
  File "/usr/local/lib/python3.11/http/client.py", line 325, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/http/client.py", line 294, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/ape_ethereum/provider.py", line 762, in _eth_call
    result = self.make_request("eth_call", arguments)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/ape_alchemy/provider.py", line 280, in make_request
    result = request_with_retry(
             ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/ape/utils/rpc.py", line 134, in request_with_retry
    return func()
           ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/ape_alchemy/provider.py", line 281, in <lambda>
    lambda: self.web3.provider.make_request(RPCEndpoint(rpc), parameters),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/web3/_utils/caching/caching_utils.py", line 261, in wrapper
    return func(provider, method, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/web3/providers/rpc/rpc.py", line 169, in make_request
    raw_response = self._make_request(method, request_data)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/web3/providers/rpc/rpc.py", line 146, in _make_request
    return self._request_session_manager.make_post_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/web3/_utils/http_session_manager.py", line 145, in make_post_request
    with self.get_response_from_post_request(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/web3/_utils/http_session_manager.py", line 127, in get_response_from_post_request
    return session.post(endpoint_uri, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 637, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 682, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/taskiq/receiver/receiver.py", line 271, in run_task
    returned = await target_future
               ^^^^^^^^^^^^^^^^^^^
  File "/app/bot.py", line 40, in redeemed
    params=dict(amount=log.value // 10 ** stable.decimals()),
                                          ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/ape/contracts/base.py", line 273, in __call__
    return ContractCall(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/ape/contracts/base.py", line 123, in __call__
    raw_output = self.provider.send_call(txn, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/ape_ethereum/provider.py", line 708, in send_call
    return self._eth_call(arguments, raise_on_revert=raise_on_revert, skip_trace=skip_trace)
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/ape_ethereum/provider.py", line 785, in _eth_call
    raise vm_err.with_ape_traceback() from err
ape.exceptions.VirtualMachineError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

I have another example but I want to check that this works first

How I did it

How to verify it

Checklist

  • Passes all linting checks (pre-commit and CI jobs)
    - [ ] New test cases have been added and are passing
    - [ ] Documentation has been updated
  • PR title follows Conventional Commit standard (will be automatically included in the changelog)

Copy link
Member Author

@fubuloubu fubuloubu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoopsie

@fubuloubu fubuloubu force-pushed the fix/random-disconnect branch 2 times, most recently from fc18cc3 to 2059bf4 Compare February 20, 2025 17:33
@fubuloubu fubuloubu marked this pull request as ready for review February 20, 2025 18:16
@fubuloubu fubuloubu requested a review from antazoey February 20, 2025 18:16
@fubuloubu
Copy link
Member Author

Needs rebase for weird alchemy issue present on main, but good to go

@fubuloubu fubuloubu force-pushed the fix/random-disconnect branch from 1d440ca to e832333 Compare February 23, 2025 05:04
@fubuloubu fubuloubu enabled auto-merge (squash) February 23, 2025 05:04
@fubuloubu fubuloubu requested a review from antazoey February 23, 2025 05:04
@fubuloubu fubuloubu merged commit 3c630d3 into main Feb 24, 2025
25 checks passed
@fubuloubu fubuloubu deleted the fix/random-disconnect branch February 24, 2025 04:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Alchemy drops some requests where eth_call reverts
2 participants