Skip to content

fix(misc): fix erroring debug statements #663

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 10, 2025
Merged

fix(misc): fix erroring debug statements #663

merged 1 commit into from
Mar 10, 2025

Conversation

BRNMan
Copy link

@BRNMan BRNMan commented Mar 10, 2025

Changed two debug statements that were supposed to print a number argument from printing a list to correctly printing the length of the list.

Related Issues

When calling source.parse_categories() with a debug level logger, I was seeing this error. This was messing up my "look at how fast the logs go" method of performance profiling.

TypeError: %d format: a real number is required, not list
Call stack:
  File "/usr/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/.../scraper.py", line 413, in safe_scrape
    self.scrape_site(site)
  File "/home/.../scraper.py", line 346, in scrape_site
    paper.parse_categories()
  File "/home/.../venv/lib/python3.12/site-packages/newspaper/source.py", line 326, in parse_categories
    log.debug("We are extracting from %d categories", self.categories)
Message: 'We are extracting from %d categories'
Arguments: ([Category(url='...', html='<!DOCTYPE html>

Proposed Changes:

Looks like we weren't printing the length of the lists in a few places. This change corrects that.

How did you test it?

Run something like this before you scrape a site.

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.StreamHandler()
    ]
)

Then create a source and run through paper.parse_categories() and paper.parse_feeds().

Now it doesn't error!

Checklist

  • I have updated the related issue with new insights and changes
  • I added unit tests and updated the docstrings
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
  • I documented my code
  • I ran pre-commit hooks and fixed any issue

@BRNMan
Fix erroring debug statements
c62b4ff 
Changed two debug statements that were supposed to print a number argument
from printing a list to correctly printing the length of the list.
@AndyTheFactory AndyTheFactory changed the base branch from master to dev-0.9.4 March 10, 2025 05:33
@AndyTheFactory AndyTheFactory merged commit 2ab8208 into AndyTheFactory:dev-0.9.4 Mar 10, 2025
6 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@BRNMan @AndyTheFactory