Comment line error: expected type expression, found 'invalid token'

[chrboesch]

Zig Version

0.14.0-dev.839+a931bfada

Steps to Reproduce and Observed Behavior

In the ziglings exercises "103_tokenization.zig" the compiler throws an error from a comment line.

Here is an extract of the exercise to reproduce the error:

// 	My name is Ozymandias, King of Kings;
// 	Look on my Works, ye Mighty, and despair!
// 	 by Percy Bysshe Shelley
//
//
const std = @import("std");
const print = std.debug.print;

pub fn main() !void {

    // our input
    const poem =
        \\My name is Ozymandias, King of Kings;
        \\Look on my Works, ye Mighty, and despair!
    ;

    // now the tokenizer, but what do we need here?
    var it = std.mem.tokenizeAny(u8, poem, " ,;!\n");

    // print all words and count them
    var cnt: usize = 0;
    while (it.next()) |word| {
        cnt += 1;
        print("{s}\n", .{word});
    }

    // print the result
    print("This little poem has {d} words!\n", .{cnt});
}

‹test› » zig run 103_tokenization.zig 
103_tokenization.zig:1:1: error: expected type expression, found 'invalid token'
//  My name is Ozymandias, King of Kings;

Expected Behavior

No error.

[alexrp]

#20928

[chrboesch]

After I took a look in the binary version of the source code, I found the reason for this error. There is a TAB in the comment.

[chrboesch]

@alexrp Thanks. Didn't find that issue because I searched for the error message.

[plaukiu]

zig could at least say that it's a tab, not just 'invalid token'. quite unhelpful with such invisible characters.

[mpfaff]

After I took a look in the binary version of the source code, I found the reason for this error. There is a TAB in the comment.

Why is that a valid reason to error? Comments should have no restrictions on content.

[chrboesch]

Yes and no. Comments can be used to automatically generate documentation. In this case, it's also content.

[vladimir-popov]

But for documentation zig has special syntax of comments. Why the regular comments should have any restrictions?

[relector-tuxnix]

This is not good design or considerate to the developer due to a white character being quite difficult to debug. Some additional context around the invalid "token" would be much better and likely save thousands of cumulative man hours of debugging and unraveling the mystery of Where's Wally.

[Vexu]

This is a duplicate of #20900