dvc push doesn't work with PUT method

[tomasz-wezyk]

Bug Report

Description

For dvc version 2.43.1 pushing to remote doesn't work. It push first file with put method and then try to use post method.

Here there is our dvc config file:

[core]
    remote = test_data
[cache]
    type = hardlink
['remote "test_data"']
    url = https://artifactory.company_name.com/artifactory/testDataRepo/
    method = PUT
    auth = basic
    jobs = 4

We tested the same configuration with 2.41.1 version and it works as expected.

Reproduce

with dvc ver. 2.43.1 installed:

1. dvc init
2. dvc remote add test_data <artifactory_storage>
3. dvc remote modify test_data method put
4. dvc remote modify test_data auth basic
5. Copy at least two files to the directory
6. dvc add
7. dvc push

Expected

Files pushed to artifactory general storage

Environment information

Output of dvc doctor:

---------------------------------
Platform: Python 3.8.0 on Windows-10-10.0.19041-SP0
Subprojects:
        dvc_data = 0.35.1
        dvc_objects = 0.19.0
        dvc_render = 0.0.17
        dvc_task = 0.1.11
        dvclive = 1.4.0
        scmrepo = 0.1.6
Supports:
        http (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.8.3, aiohttp-retry = 2.8.3)
Cache types: hardlink, symlink
Cache directory: NTFS on C:\
Caches: local
Remotes: https
Workspace directory: NTFS on C:\
Repo: dvc, git

Additional Information (if any):

It worked when we changed method in https://github.com/fsspec/filesystem_spec/blob/master/fsspec/implementations/http.py
from:

async def _put_file(
        self,
        lpath,
        rpath,
        chunk_size=5 * 2**20,
        callback=_DEFAULT_CALLBACK,
        method="post",
        **kwargs,
    ):

to:

async def _put_file(
        self,
        lpath,
        rpath,
        chunk_size=5 * 2**20,
        callback=_DEFAULT_CALLBACK,
        method="put",
        **kwargs,
    ):

but we treat this more like quick hack not a solution.

Here there are logs with verbose:
logs.txt

[skshetry]

Looks like this happens because we started using underlying fsspec filesystem instead of our wrapper for async implementations, which used to pass method to httpfs from fsspec.

dvc-objects/src/dvc_objects/fs/generic.py

Lines 149 to 152 in 8bdd8de

	if to_fs.fs.async_impl:
	put_coro = callback.wrap_and_branch_coro(
	to_fs.fs._put_file # pylint: disable=protected-access
	)

[skshetry]

@efiop, this is not a dvc-http specific issue, it’s related to dvc-objects or dvc in large.

[efiop]

CC @pmrowla

[pmrowla]

@tomasz-wezyk you should be able to test the fix with

pip install git+https://github.com/iterative/dvc-http.git@refs/pull/45/merge#egg=dvc-http

(from your existing DVC virtualenv)

[tomasz-wezyk]

@pmrowla it works with your patch