Add BufRead::read_while #70772
Closed
Add BufRead::read_while #70772
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
r? @sfackler (rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
|
I am confused why CI is failing. Perhaps someone with more knowledge on Rust's CI could point me in the right direction? |
|
#69898 will hopefully unblock the CI, right now mingw-check fails on outdated toolstate script that doesn't take renamed rustc-guide into account properly |
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
sfackler
reviewed
Apr 4, 2020
sfackler
reviewed
Apr 4, 2020
| /// assert_eq!(buf, b"-ipsum"); | ||
| /// ``` | ||
| #[unstable(feature = "buf_read_read_while", issue = "none")] | ||
| fn read_while<P>(&mut self, buf: &mut Vec<u8>, predicate: P) -> Result<usize> |
There was a problem hiding this comment.
Probably worth updating read_until to delegate to this method.
There was a problem hiding this comment.
read_until uses memchr to scan for the token; it seems likely this operation is vectorized in many cases, and I'm afraid moving to read_while's linear scan could inadvertently cause a slowdown.
|
I'm a bit worried that BufRead will have both read_until and read_while with different arguments but the naming doesn't make it super clear which is which. |
|
@sfackler That's a fair concern. My reasoning for naming this method I think I think you're right that the naming of these methods could confuse people who aren't already familiar with Rust. However both methods seem useful and reasonably named. I think the best resolution here would be to acknowledge that both methods indeed have a close resemblance, and document how they differ. For example we could add something like following to
|
sfackler
reviewed
Apr 5, 2020
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
|
I don't quite follow the motivation. As for the second paragraph I don't see the problem of taking |
@HeroicKatora Ah yes, fair enough. I didn't realize that this was implemented for
Perhaps adding more details could be helpful here. A while ago I was implementing a parser and encoder for the MIME spec and figured we could abstract it to take any
This means: "read until you encounter either // Get the param name.
//
// ```txt
// text/html; charset=utf-8;
// ^^^^^^^
// ```
let mut param_name = vec![];
s.read_while(&mut param_name, |b| !matches!(b, b';' | b'='))?;Neither |
|
It seems clear that the existing methods are insufficient, adding this convenience would be good complement. Perhaps this was not very clear but I wasn't trying to questioning the need per-se, but the need of that solution in particular. Which alternatives were considered and why was this one chosen? The adaptor solution that I was trying to hint at would look like: let mut param_name = vec![];
s
.by_ref()
.take_while(|b| !matches!(b, b';' | b'='))
.read_to_end(&mut param_name)?;which parallels |
@HeroicKatora Oh I really like that suggestion! That's a design I hadn't considered, and I'll be the first to admit that it ticks all the same boxes as what I had, but with fewer downsides (as @sfackler mentioned: argument ordering of I think the right next step is to close this PR and file an issue to implement |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch implements
BufRead::read_while, a counterpart toBufRead::read_untilthat takes a predicate instead of a single byte. This allows delimiting byte streams using more refined patterns, which in particularly works well with the newly stabilizedmatchesmacro.Usage
Motivation
The
ReadandBufReadtraits provide far fewer adapters thanIteratordoes. The idea is that in most cases it should be possible to convert aReadinto aBufRead, and subsequently calllinesorsplitto convert it into anIterator.However both
BufRead::splitandBufRead::linesconsumeselfwhich is problematic for streams that want to parse only a section of a stream but leave the rest of the stream intact. In such cases it can be incredibly convenient to be able to directly operate on the stream (a common example are "headers" sections). And adapters such asBufRead::read_whilemake such operations faster to author and less prone to errors.Differences with BufRead::read_until
BufRead::read_untilprovides an inclusive read,BufRead::read_whileprovides a non-inclusive read. When thepredicateno longer returnstrue, the operation finishes and the buffer becomes available with all bytes read up until that point.References
BufRead::read_untilIterator::take_whileomnom::BufReadExt::read_while