Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump datasets from 3.2.0 to 3.4.0 #136

Closed
wants to merge 1 commit into from

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Mar 14, 2025

Bumps datasets from 3.2.0 to 3.4.0.

Release notes

Sourced from datasets's releases.

3.4.0

Dataset Features

  • Faster folder based builder + parquet support + allow repeated media + use torchvideo by @​lhoestq in huggingface/datasets#7424

    • /!\ Breaking change: we replaced decord with torchvision to read videos, since decord is not maintained anymore and isn't available for recent python versions, see the video dataset loading documentation here for more details. The Video type is still marked as experimental is this version
    from datasets import load_dataset, Video
    dataset = load_dataset("path/to/video/folder", split="train")
    dataset[0]["video"]  # <torchvision.io.video_reader.VideoReader at 0x1652284c0>

    • faster streaming for image/audio/video folder from Hugging Face
    • support for metadata.parquet in addition to metadata.csv or metadata.jsonl for the metadata of the image/audio/video files
  • Add IterableDataset.decode with multithreading by @​lhoestq in huggingface/datasets#7450

    • even faster streaming for image/audio/video folder from Hugging Face if you enable multithreading to decode image/audio/video data:
    dataset = dataset.decode(num_threads=num_threads)
  • Add with_split to DatasetDict.map by @​jp1924 in huggingface/datasets#7368

General improvements and bug fixes

New Contributors

Full Changelog: huggingface/datasets@3.3.2...3.4.0

3.3.2

Bug fixes

Other general improvements

New Contributors

... (truncated)

Commits
  • 14fb15a release: 3.4.0 (#7453)
  • 97ff626 minor docs changes (#7452)
  • e8ee24a Fix resuming after ds.set_epoch(new_epoch) (#7451)
  • 7ad7379 Add IterableDataset.decode with multithreading (#7450)
  • f09db01 Fix small bugs with async map (#7445)
  • 67ffdfb Refactor string_to_dict to return None if there is no match instead of ra...
  • f693f4e Add with_split to DatasetDict.map (#7368)
  • 5c8869f Faster folder based builder + parquet support + allow repeated media + use t...
  • 26379d5 Use pyupgrade --py39-plus (#7428)
  • 6631dc0 fix: None default with bool type on load creates typing error (#7426)
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Bumps [datasets](https://github.com/huggingface/datasets) from 3.2.0 to 3.4.0.
- [Release notes](https://github.com/huggingface/datasets/releases)
- [Commits](huggingface/datasets@3.2.0...3.4.0)

---
updated-dependencies:
- dependency-name: datasets
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot bot added dependencies Pull requests that update a dependency file python Pull requests that update Python code labels Mar 14, 2025
Copy link
Contributor Author

dependabot bot commented on behalf of github Mar 17, 2025

Superseded by #139.

@dependabot dependabot bot closed this Mar 17, 2025
@dependabot dependabot bot deleted the dependabot/pip/datasets-3.4.0 branch March 17, 2025 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file python Pull requests that update Python code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants