Skip to content

Rollup of 6 pull requests #89703

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Oct 9, 2021
Merged
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
853ffc7
stack overflow handler specific openbsd fix.
devnexen Jul 27, 2021
af83a96
std: Stabilize command_access
lf- Aug 28, 2021
4be574e
Add 'core::array::from_fn' and 'core::array::try_from_fn'
c410-f3r Sep 30, 2021
fdccc7d
Use reference instead of raw pointer
c410-f3r Sep 30, 2021
325025e
Improve previous commit
steffahn Sep 30, 2021
13bfcb7
Merge pull request #2 from steffahn/collect_into_array_fix_ub
c410-f3r Sep 30, 2021
355c7e9
Remove an unnecessary use of unwrap_unchecked
steffahn Sep 30, 2021
91ad91e
Skip platforms without unwinding support
c410-f3r Oct 3, 2021
5e1941c
Apply suggestions from code review
yaahc Oct 5, 2021
e159d42
Redo #81358 in unicode-table-generator
cuviper Oct 6, 2021
6b0b417
Let unicode-table-generator fail gracefully for bitsets
cuviper Oct 7, 2021
459a7e3
Regenerate tables for Unicode 14.0.0
cuviper Oct 7, 2021
6a52fb7
Add documentation to boxed conversions
timClicks Oct 8, 2021
85c4a52
Also cfg flag auxiliar function
c410-f3r Oct 8, 2021
fa5a212
Simplify wording
timClicks Oct 9, 2021
020ec0a
Remove unnecessary hyphen
timClicks Oct 9, 2021
3214253
Fix invalid HTML generation for higher bounds
GuillaumeGomez Oct 8, 2021
86bf3ce
Rollup merge of #75644 - c410-f3r:array, r=yaahc
GuillaumeGomez Oct 9, 2021
3e4f956
Rollup merge of #87528 - :stack_overflow_obsd, r=joshtriplett
GuillaumeGomez Oct 9, 2021
703cb97
Rollup merge of #88436 - lf-:stabilize-command-access, r=yaahc
GuillaumeGomez Oct 9, 2021
21a5101
Rollup merge of #89614 - cuviper:unicode-14, r=joshtriplett
GuillaumeGomez Oct 9, 2021
9f32ab8
Rollup merge of #89664 - timClicks:51430-document-boxed-conversions, …
GuillaumeGomez Oct 9, 2021
3e93472
Rollup merge of #89700 - GuillaumeGomez:fix-rustdoc-higher-bound-html…
GuillaumeGomez Oct 9, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Let unicode-table-generator fail gracefully for bitsets
The "Alphabetic" property in Unicode 14 grew too big for the bitset
representation, panicking "cannot pack 264 into 8 bits". However, we
were already choosing the skiplist for that anyway, so this doesn't need
to be a hard failure. That panic is now a returned `Err`, and then in
`emit_codepoints` we automatically defer to skiplist.
cuviper committed Oct 7, 2021
commit 6b0b41729939c3f7520e9ed86b36fba2524c7970
10 changes: 6 additions & 4 deletions src/tools/unicode-table-generator/src/raw_emitter.rs
Original file line number Diff line number Diff line change
@@ -23,7 +23,7 @@ impl RawEmitter {
writeln!(&mut self.file).unwrap();
}

fn emit_bitset(&mut self, ranges: &[Range<u32>]) {
fn emit_bitset(&mut self, ranges: &[Range<u32>]) -> Result<(), String> {
let last_code_point = ranges.last().unwrap().end;
// bitset for every bit in the codepoint range
//
@@ -44,7 +44,7 @@ impl RawEmitter {
let unique_words =
words.iter().cloned().collect::<BTreeSet<_>>().into_iter().collect::<Vec<_>>();
if unique_words.len() > u8::MAX as usize {
panic!("cannot pack {} into 8 bits", unique_words.len());
return Err(format!("cannot pack {} into 8 bits", unique_words.len()));
}
// needed for the chunk mapping to work
assert_eq!(unique_words[0], 0, "has a zero word");
@@ -105,6 +105,8 @@ impl RawEmitter {
writeln!(&mut self.file, " &BITSET_MAPPING,").unwrap();
writeln!(&mut self.file, " )").unwrap();
writeln!(&mut self.file, "}}").unwrap();

Ok(())
}

fn emit_chunk_map(&mut self, zero_at: u8, compressed_words: &[u8], chunk_length: usize) {
@@ -154,12 +156,12 @@ pub fn emit_codepoints(emitter: &mut RawEmitter, ranges: &[Range<u32>]) {
emitter.blank_line();

let mut bitset = emitter.clone();
bitset.emit_bitset(&ranges);
let bitset_ok = bitset.emit_bitset(&ranges).is_ok();

let mut skiplist = emitter.clone();
skiplist.emit_skiplist(&ranges);

if bitset.bytes_used <= skiplist.bytes_used {
if bitset_ok && bitset.bytes_used <= skiplist.bytes_used {
*emitter = bitset;
emitter.desc = String::from("bitset");
} else {