Folder names and restructuring

[d-kleine]

Hey there!

I've been thinking about this repository's folder structure in light of PR #113, and I had some thoughts on how to enhance this for better organization.

• First off, regarding the ".1st_setup" folder, this is not an internal Git folder, so I think it would be more intuitive to rename it without starting with a period.
• Also, I noticed that the appendices are placed at the top of the folder structure. Typically, they're found at the end of a chapter or book. Perhaps moving them to the end would make navigation smoother and more consistent with standard practices.
• Lastly, instead of using generic labels like "ch01," etc., it would be better to consider using actual chapter names. This would not only personalize the structure but also make it easier for everyone to locate the specific content they're looking for.

Here's what I suggest in the following order:

1. Git-internal folders

   • .git
   • .github/workflows
   • (optional) .devcontainer
   • (optional) .vscode
     etc.

2. Chapter folder (numbers indicating order, each renamed with topic)
   2.1 00_Setup (this would be the renamed ".1st_setup" folder)
   2.2 01_Understanding_LLMs
   2.3 02_Working_with_Text_Data
   etc.

3. Appendix folders
   3.1 Appendix_A
   3.2 Appendix_B
   3.3 Appendix_C
   etc.

4. Regular project files at root

   • README.md
   • .gitignore
   • etc.

I believe this arrangement would make navigating this repository a breeze and keep things nicely organized. How does that sound? Let me know your thoughts!

[rasbt]

Thanks a lot for taking the time to lay out your thoughts and suggestions so elaborately. It's sometimes a bit hard to find good balance making it work well for various types of readers. Wit h that, I mean keeping it as simple as possible for those who find too many folders overwhelming, but then also having useful features for the more "power user" audience like you, who is more experienced.

Honestly, I think the best way is to have all the setup instructions in one Readme that is preferably not too long so that readers can skim it and then decide which approach to use (and then read about that in more detail).

By the way, I like your chapter naming approach. I thought about that in the very beginning as well. The big challenge with that is that chapter names may still change as per publisher preferences. It's also still possible that chapters may get split ... (although I hope they stay as they are, because I put a lot of thought into it, and I prefer the current chapter layout.)

I agree though that a short name would make the chapters easier to identify ... something to think about when the book is finalized and the chapters won't change anymore. However, at the same time, I am a bit worried about breaking hyperlinks (I remember there were people who linked certain chapter notebooks, and changing the names could create annoyances there).

I also thought about naming them 01_ch, 02_ch, ... but I think that would be awkward. However, I do agree that it's a bit annoying that the appendices are listed before the main chapters. I hope there won't be too many / any additional appendices though, so that's maybe not the end of the world.

Based on your feedback, I thought about the organization a bit more, and took another knack at it in #115

In short, the main changes would be:

• There's one folder that collects all setup recommendations: 1st_setup
• The README there lists all approaches in short, so readers can get a quick overview first
• The subfolders then contain more information on each of these approaches
• In addition, I also added a note on top of the main README's table of contents -- I believe that's something you advocated a few months ago

How do you like this approach?

[d-kleine]

Thank you sincerely for your thoughtful feedback. In response to your ideas, I would like to address several key points:

I agree though that a short name would make the chapters easier to identify ... something to think about when the book is finalized and the chapters won't change anymore. However, at the same time, I am a bit worried about breaking hyperlinks (I remember there were people who linked certain chapter notebooks, and changing the names could create annoyances there).

I see your point here too, especially about relative paths (not just in the READMEs, but also in the Jupyter notebooks). As an idea, we could also set up a new CI workflow that checks for broken links leading to a 404 status? This would at least automate the process of looking for broken links.

About fixing hyperlinks, I don't see any possibility to fix hyperlinks automatically. But at least in VS Code, you could search all files in a repository for those paths (also with regular expressions) and replace them quite easily:

I also thought about naming them 01_ch, 02_ch, ... but I think that would be awkward. However, I do agree that it's a bit annoying that the appendices are listed before the main chapters. I hope there won't be too many / any additional appendices though, so that's maybe not the end of the world.

Agree, 01_ch, 02_ch,... sounds awkward for me too. My idea of naming the folders like 01_Understanding_LLMs, 02_Working_with_Text_Data,... was because of the sorting of the files GitHub does:

• Files and folders in a repository are sorted alphabetically
• Folders always appear before files
• Numbers and periods come before letters in the sorting order

Then the appendices appendix_A, appendix_B, ... would be listed after the chapters.

In short, the main changes would be:

There's one folder that collects all setup recommendations: 1st_setup
The README there lists all approaches in short, so readers can get a quick overview first
The subfolders then contain more information on each of these approaches
In addition, I also added a note on top of the main README's table of contents -- I believe that's something you advocated a few months ago

My thoughts on that:

• With reference to the READMEs, I really like the changes as they improve navigation and purpose for the folders and files provided by the repo.
• About the change to the ".1st_setup" folder, I really appreciate the adjustment made by removing the period before the name. However, I'm not quite fond of the name itself. "1st_setup" may cause confusion or misinterpretation due to its numeric prefix, especially as there are other folders labeled with numbers like "01" (e.g. ch01) inside.
• I think it would be the best to list the appendices after the chapters as this would make the repository consistent with the book and easier to navigate for users.
• Right now, I'm not sure how to rename ch01 to something like 01_ch in a way that keeps it similar without including the content (like Understanding_LLMs) in the name. Additionally, I feel like the '_ch' tag in the chapter folder names is unnecessary.
• As a side note, it seems a little odd to me too that chapters are abbreviated as ch01 while appendices are named in full as appendix_A.

I see your point about the potential challenges with chapter names and layout changes. It's understandable to prefer the current setup, especially after putting already so much effort into it. Keeping the current chapter layout sounds like a good plan for now in the draft.

I completely agree that making major changes to the repository's structure while it's still in the draft stage might not be the best approach. However, once there is a final book's structure, it may be beneficial to consider adjusting the folder structure and names to reflect their contents, aligning the repository's structure more closely with the organization of the book, before the book comes to the final review from the publisher and then goes to print. From my point of view, this could greatly improve readability and user-friendliness when navigating and enhance the overall experience.

[rasbt]

I see your point here too, especially about relative paths (not just in the READMEs, but also in the Jupyter notebooks). As an idea, we could also set up a new CI workflow that checks for broken links leading to a 404 status? This would at least automate the process of looking for broken links.

Sorry, I should have said external hyperlinks. But this is a great point, I'll try to set this up as part of the CI!

"1st_setup" may cause confusion or misinterpretation due to its numeric prefix

Totally. I was mainly trying to have it show up higher up. But maybe with the note in the README.md this is not necessary anymore and it can just be setup

[d-kleine]

Sorry, I should have said external hyperlinks.

If one would reorganize the content, this won't have any impact on hyperlinks, for instance when they are pointing at external websites. An automated check would be beneficial to check for broken links, especially as sometimes URLs or websites contents can change over time. But the critical components are rather the absolute and relative paths in the files then, when it comes renaming or restructuring folder and files.

Maybe Manning also has some tools to check for broken links or paths?

What do you think about the above mentioned idea, discussing the folder naming once there is a final structure, before the book will be finalized and goes into print?

[rasbt]

Oh I actually meant external links in terms of external websites, social media posts, etc pointing to specific notebooks in this repository.

I do like having more descriptive folder names similar to what you suggested. It's something I also had in mind in the very beginning when I set up the code repo. But yeah, I then decided against it since I wasn't/am sure how much the chapter names will change etc.

For the moment I need to get back focusing more on the actual contents of the book but that's something to perhaps revisit at a later stage.

[d-kleine]

Oh I actually meant external links in terms of external websites, social media posts, etc pointing to specific notebooks in this repository.

Oh, I see what you're saying now. Well, in a draft, changes are bound to happen, that's just the nature of the process, unfortunately.

Alright, thanks! 👍

[d-kleine]

@rasbt As an additional idea for the finalized book repo, maybe it would be useful to store pivotal files like previous_chapters.py and gpt_download.py more centrally for importing them. Currently, there are multiplied over the all chapters, making it hard to manage them efficiently.

[rasbt]

Yes, I agree, from a developer perspective, this looks messy and even annoying in case you update something that should be reflected in all chapters. If this were my own private project, I'd have them in one place and then symlink them (or just add the relative folder to the Python path). However, based on my previous books and workshop teaching experience, there are lots of people who would run into trouble with that because they'd move certain files, which would then break this. So, this is why I made each folder self-contained (e.g. you can just drag and drop the whole folder contents into Colab and it should work).