Documenting that the new (3.14) pathlib copy functionality uses Copy-on-Write

[opk12]

Documentation

(edited)

The PR 119058 and 122369 added pathlib.Path.copy(), with Copy-on-Write support. CoW should be documented, because it has distinctive, user-requested properties on huge files.

• Copying is instantaneous, does negligible I/O, requires no disk space for the data and negligible disk space for the metadata.
• Reading the original and the copy does half the I/O and requires half the RAM (page cache) than a traditional copy (think booting VM images)

In the context of a Linux VM manager, CoW is an explicit desired property. Disk image copying is the slowest part of snapshotting a VM. Users expect CoW snapshots nowadays, and intentionally set up a CoW filesystem for the disk image directory.

For clarity, I'm not asking to mention FICLONE specifically. I'm not asking to mention copy_file_range, a micro-optimization on the traditional copy algorithm. Instead, my point is that switching from O(file size) to zero is a user-visible feature.

Keywords: reflink copy

Linked PRs

• GH-124985: Document that pathlib.Path.copy() uses copy-on-write. #125861

[picnixz]

I'm not sure to follow what you want to do. What I understood is that you want us to document how we support CoW filesystems (by the way, the project you linked only support Python 3.7 according to pypi and I don't know why it would be a "keyword").

cc @barneygale as the PR's author

[opk12]

Please document that copies are CoW if the filesystem is CoW and list the supported filesystems, so that one is certain to provide CoW to the end user. Please leave out the implementation details (the exact syscalls).

Sorry for the confusion. reflink copy / shallow copy is jargon for CoW, and the library was named after that. I now reworked the ticket title.

[barneygale]

I'm cautious about touting the speed benefits of copy(), because the implementation of PathBase.copy() makes too many system calls when walking directories. We ought to add an implementation of Path.copy() that uses os.DirEntry.is_symlink() and is_dir(), rather than calling these methods on path objects. Or we could open a large can of worms, and consider whether PathBase.iterdir() might be allowed generate path objects with a special dir_entry attribute that grants public access to a os.DirEntry-like object, and then implement this in Path.iterdir(), and consult the attribute from PathBase.copy().

[opk12]

Very good point. Without touching on copy()'s performance, how about only saying that it does CoW and on which filesystems? As said above, CoW is an explicit user request, has much better free space requirement and future read performance.

[opk12]

(Off-topic: there even was an actual proposal at btrfs, the most used CoW fs on Linux, to CoW-copy an arbitrary directory with a single syscall. The proposal stalled, but showed that this is doable.)

[barneygale]

#125419 will solve the known copy() performance problems. When it lands I'll start working on this issue.

[barneygale]

This is now documented! Turns out I didn't need to wait for #125419. Thanks for the report, @opk12