refactor: consolidate syntax crates, introduce type syntax & core syntax

[azjezz]

📌 What Does This PR Do?

This PR introduces a significant refactoring of the crates related to syntax processing within the Mago workspace, aiming for better organization, improved maintainability, and clearer API boundaries.

🔍 Context & Motivation

Currently, several small crates (mago_ast, mago_token, mago_lexer, mago_parser, mago_walker, mago_ast_utils) handle different aspects of PHP syntax processing. These crates are tightly coupled:

• The parser needs the lexer, AST definitions, and tokens.
• The lexer needs token definitions.
• The walker needs the AST.
• AST utils need the AST. This strong interdependency means users often need to import most or all of these crates together, and internal changes can require updates across multiple places.

Additionally, the PHP docblock type parser, previously developed within mago_type_checker ( private for now ), is a valuable component that could be useful independently. Common utilities are also shared between the main PHP lexer/parser and this type parser.

🛠️ Summary of Changes

• Consolidated mago_syntax Crate:

  • Merged the functionality of mago_ast, mago_ast_utils, mago_token, mago_lexer, mago_parser, and mago_walker into a single, unified crate named mago_syntax.
  • This crate now serves as the primary interface for PHP tokenization, parsing into an AST, AST definitions, and AST traversal utilities.
  • Rationale: Reflects the practical reality that these components are almost always used together. Simplifies dependency management for consumers of the PHP parser.

• New mago_type_syntax Crate:

  • Introduced a new crate dedicated to parsing PHP docblock type strings.
  • Contains the type-specific lexer (TypeLexer), token definitions (TypeToken, TypeTokenKind), parser, AST, and error types.
  • Extracted and refined from the work-in-progress mago_type_checker.
  • Rationale: Exposes the type parser as a reusable component for other tools that might need to understand docblock types without needing the full PHP parser or type checker.

• New mago_syntax_core Crate:

  • Introduced a new utility crate to hold common, low-level components shared between different lexing/parsing tasks.
  • Contains shared fundamentals like lexer input handling (e.g., the Input struct), position/span management helpers (if not already in a dedicated span crate), error primitives, and common lexing utilities (e.g., read_digits_of_base, character classification macros/functions).
  • Rationale: Avoids code duplication between mago_syntax and mago_type_syntax by providing a shared foundation (DRY principle).

📂 Affected Areas

• Linter
• Formatter
• CLI
• Composer Plugin
• Dependencies
• Documentation
• Other (please specify): Parser, Lexer, Ast, Walker, Ast Utilities, and Token.

🔗 Related Issues or PRs

📝 Notes for Reviewers